Skip to content

truffle: add AArch64 wide path using vqtbl2q - #406

Open
a-eridani wants to merge 1 commit into
VectorCamp:developfrom
a-eridani:truffle-wide-path
Open

truffle: add AArch64 wide path using vqtbl2q#406
a-eridani wants to merge 1 commit into
VectorCamp:developfrom
a-eridani:truffle-wide-path

Conversation

@a-eridani

@a-eridani a-eridani commented Aug 12, 2026

Copy link
Copy Markdown

The default NEON path uses 3 pshufb and bit-ops, even though AArch64 has a 32-byte tbl. So, this PR fixes it. Also, blockSingleMaskWide uses vshlq_u8(vdupq_n(1), ...) instead of tbl. This gave a speedup of about 30%

Also, before in /internal/TruffleWide was a gate by HAVE_SVE instead of CAN_USE_WIDE_TRUFFLE.

| Matcher              | max_matches |      size | loops | total_sec | avg_time |    max_bw |    avg_bw |
|:---------------------|------------:|----------:|------:|----------:|---------:|----------:|----------:|
| Truffle              |           0 |     16000 | 62500 |     0.068 |    1.087 | 14040.935 |         0 |
| Truffle              |           0 |     64000 | 15625 |     0.034 |    2.200 | 27744.867 |         0 |
| Truffle              |           0 |    256000 |  3906 |     0.034 |    8.777 | 27815.111 |         0 |
| Truffle              |           0 |   1024000 |   976 |     0.034 |   35.207 | 27737.763 |         0 |
| Truffle              |           0 |   4096000 |   244 |     0.035 |  142.434 | 27424.901 |         0 |
| Truffle              |           0 |  16384000 |    61 |     0.034 |  565.049 | 27652.460 |         0 |
| Truffle              |           0 |  65536000 |    15 |     0.035 | 2306.267 | 27100.075 |         0 |
| Truffle              |           0 | 262144000 |     3 |     0.027 | 9026.000 | 27697.762 |         0 |
| Reverse Truffle      |           0 |     16000 | 62500 |     0.107 |    1.712 |  8914.344 |         0 |
| Reverse Truffle      |           0 |     64000 | 15625 |     0.053 |    3.403 | 17937.335 |         0 |
| Reverse Truffle      |           0 |    256000 |  3906 |     0.039 |    9.975 | 24476.099 |         0 |
| Reverse Truffle      |           0 |   1024000 |   976 |     0.035 |   36.370 | 26850.861 |         0 |
| Reverse Truffle      |           0 |   4096000 |   244 |     0.034 |  141.225 | 27659.683 |         0 |
| Reverse Truffle      |           0 |  16384000 |    61 |     0.035 |  569.836 | 27420.167 |         0 |
| Reverse Truffle      |           0 |  65536000 |    15 |     0.034 | 2256.467 | 27698.171 |         0 |
| Reverse Truffle      |           0 | 262144000 |     3 |     0.027 | 9131.667 | 27377.259 |         0 |
| Truffle Wide         |           0 |     16000 | 62500 |     0.061 |    0.977 | 15621.969 |         0 |
| Truffle Wide         |           0 |     64000 | 15625 |     0.028 |    1.763 | 34617.384 |         0 |
| Truffle Wide         |           0 |    256000 |  3906 |     0.026 |    6.729 | 36282.513 |         0 |
| Truffle Wide         |           0 |   1024000 |   976 |     0.027 |   27.908 | 34992.474 |         0 |
| Truffle Wide         |           0 |   4096000 |   244 |     0.027 |  112.475 | 34729.813 |         0 |
| Truffle Wide         |           0 |  16384000 |    61 |     0.028 |  455.377 | 34312.226 |         0 |
| Truffle Wide         |           0 |  65536000 |    15 |     0.027 | 1822.867 | 34286.655 |         0 |
| Truffle Wide         |           0 | 262144000 |     3 |     0.022 | 7262.000 | 34425.778 |         0 |
| Reverse Truffle Wide |           0 |     16000 | 62500 |     0.098 |    1.569 |  9724.425 |         0 |
| Reverse Truffle Wide |           0 |     64000 | 15625 |     0.043 |    2.764 | 22085.508 |         0 |
| Reverse Truffle Wide |           0 |    256000 |  3906 |     0.029 |    7.381 | 33074.822 |         0 |
| Reverse Truffle Wide |           0 |   1024000 |   976 |     0.025 |   25.967 | 37607.521 |         0 |
| Reverse Truffle Wide |           0 |   4096000 |   244 |     0.024 |   99.902 | 39100.960 |         0 |
| Reverse Truffle Wide |           0 |  16384000 |    61 |     0.025 |  405.918 | 38492.993 |         0 |
| Reverse Truffle Wide |           0 |  65536000 |    15 |     0.025 | 1633.867 | 38252.815 |         0 |
| Reverse Truffle Wide |           0 | 262144000 |     3 |     0.019 | 6469.000 | 38645.849 |         0 |
| Truffle              |           5 |     16000 | 62500 |     0.072 |     0.466 | 27173.112 | 20930.097 |
| Truffle              |           5 |     64000 | 15625 |     0.070 |     1.790 | 27585.431 | 21821.484 |
| Truffle              |           5 |    256000 |  3906 |     0.069 |     7.087 | 27665.021 | 22050.189 |
| Truffle              |           5 |   1024000 |   976 |     0.069 |    28.234 | 27816.285 | 22150.179 |
| Truffle              |           5 |   4096000 |   244 |     0.069 |   113.306 | 27979.598 | 22085.727 |
| Truffle              |           5 |  16384000 |    61 |     0.069 |   454.928 | 28086.784 | 22023.373 |
| Truffle              |           5 |  65536000 |    15 |     0.068 |  1818.973 | 27592.804 | 21975.848 |
| Truffle              |           5 | 262144000 |     3 |     0.054 |  7260.000 | 27849.981 | 22069.129 |
| Reverse Truffle      |           5 |     16000 | 62500 |     0.071 |     0.679 | 27175.370 | 21288.299 |
| Reverse Truffle      |           5 |     64000 | 15625 |     0.069 |     2.664 | 27780.968 | 21946.189 |
| Reverse Truffle      |           5 |    256000 |  3906 |     0.069 |    10.618 | 27773.796 | 22101.429 |
| Reverse Truffle      |           5 |   1024000 |   976 |     0.069 |    42.468 | 28004.260 | 22161.031 |
| Reverse Truffle      |           5 |   4096000 |   244 |     0.069 |   170.344 | 27765.640 | 22060.683 |
| Reverse Truffle      |           5 |  16384000 |    61 |     0.070 |   687.203 | 27867.115 | 21913.221 |
| Reverse Truffle      |           5 |  65536000 |    15 |     0.068 |  2722.853 | 27989.252 | 22120.801 |
| Reverse Truffle      |           5 | 262144000 |     3 |     0.055 | 10928.467 | 27958.993 | 22100.953 |
| Truffle Wide         |           5 |     16000 | 62500 |     0.058 |    0.377 | 33780.804 | 25889.973 |
| Truffle Wide         |           5 |     64000 | 15625 |     0.056 |    1.429 | 34533.085 | 27323.010 |
| Truffle Wide         |           5 |    256000 |  3906 |     0.055 |    5.666 | 34790.707 | 27607.786 |
| Truffle Wide         |           5 |   1024000 |   976 |     0.054 |   22.223 | 36434.442 | 28200.155 |
| Truffle Wide         |           5 |   4096000 |   244 |     0.055 |   90.159 | 34970.648 | 27756.995 |
| Truffle Wide         |           5 |  16384000 |    61 |     0.055 |  362.977 | 34690.628 | 27577.485 |
| Truffle Wide         |           5 |  65536000 |    15 |     0.055 | 1458.573 | 34547.883 | 27401.251 |
| Truffle Wide         |           5 | 262144000 |     3 |     0.043 | 5782.400 | 34802.784 | 27669.569 |
| Reverse Truffle Wide |           5 |     16000 | 62500 |     0.051 |    0.491 | 37505.626 | 29484.573 |
| Reverse Truffle Wide |           5 |     64000 | 15625 |     0.049 |    1.894 | 38953.306 | 30835.465 |
| Reverse Truffle Wide |           5 |    256000 |  3906 |     0.049 |    7.558 | 39577.227 | 31063.787 |
| Reverse Truffle Wide |           5 |   1024000 |   976 |     0.049 |   30.186 | 39763.246 | 31286.766 |
| Reverse Truffle Wide |           5 |   4096000 |   244 |     0.049 |  121.105 | 39177.571 | 31027.281 |
| Reverse Truffle Wide |           5 |  16384000 |    61 |     0.049 |  481.967 | 39078.516 | 31118.562 |
| Reverse Truffle Wide |           5 |  65536000 |    15 |     0.048 | 1928.080 | 39172.673 | 30887.119 |
| Reverse Truffle Wide |           5 | 262144000 |     3 |     0.039 | 7768.733 | 38900.415 | 30817.083 |

Btw I also added return nullptr in each branch with assert(false) to truffle_simd.hpp and template SuperVector<16> SuperVector<16>::vshr_8_imm<5>() const into impl.cpp.

@markos markos left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi, thanks for the patch. Although you are correct about the optimization, I'm actually not very happy with how the SVE port is handled and I will be making heavy changes in this part of the code over the next months. I will defer merging this PR, until I figure out what's the best route to avoid this kind of single-arch optimizations -Wide Truffle seems to work only for SVE, which is something I should not have accepted in the first place. An optimization like that should be provided for all capable architectures, eg. AVX2/AVX512/etc. Or at least the proposal should leave open/unimplemented stubs for other architectures to be filled.

@markos markos added the enhancement New feature or request label Aug 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants