> There is no reason a portable_simd relu_dot implemention should need to specify the SIMD width.
What should it choose then? I have a Zen 3 processor, and benchmarking some simd I did recently says 32 byte or 64 byte chunks was fastest. But I'm sure I'd get a different result on a different Zen, and different again on Intel's.
How would the library decide what SIMD width I should use?
It'd need some kind of compile-time hardware-feature-detection, yea? That seems probably feasible since proc macros can do essentially anything they like (worryingly).
Only if the end-user is the one compiling the software, on the same very system they'll be running it on. Which is true of GPU shader kernels, due to how GPU drivers work; but isn't generally true of CPU object code (unless you're on Gentoo.)
What you'd actually want is a matrix of variant implementations burned into the binary, with runtime (or process-boot-time) hardware detection that swaps symbols out to point to the correct variant.
If you want the library to perform that selection, you also need the "correct" / most efficient implementation to be independent of your workload. I'm not that familiar with SIMD performance characteristics, but I wouldn't be surprised if that's not always the case.
From how I understand it, there'd likely only be a single SIMD function impl per uarch that'd actually be fully legally executable without hitting undefined instructions. Plus increasingly-more-generic function impls compiled for lower and lower common-denominator subsets of SIMD functionality. (Ultimately grounding in a non-SIMD impl.)
If that's the case, then the selection logic would be trivial: figure out the full hierarchical ID of the uarch you're running on, then search for the longest prefix match in the table of available impls.
If things work more like you're imagining, though, then I suppose the process-boot impl-selector would narrow down the impl matrix to just the subset that are legal on the running uarch; pick one arbitrarily to be active at first; and then wrap the calls in a handler that gradually re-works the called function in a way reminiscent of a profile-guided JIT, but without the need to actually synthesize any code at runtime — instead, it'd just be a multi-armed bandit passing-through-to and re-ranking competitor impls, with decreasing sampling of the non-first-ranked impls as confidence-in-score-separation increases.
It's definitely a 80% solution where you occasionally need to drop down to intrinsics (at zero runtime perf cost) for CPU specific instructions.
But just having vector types, arithmetic, swizzling, loads and stores will go a long way for basic tasks.
And with generics you can write code that is type and width agnostic. No need to rewrite your code of you want to go from SSE to AVX512, just change from f32x4 to f32x16 (or use generics) and you are done.
Because there are platform vendors. And SIMD performance very much depends on the use-case, which is a balance of practicalities and specifications and intended deployment targets ..
I also think this is a deployment problem, not a build problem, but okay ..
> also think this is a deployment problem, not a build problem ...
This I agree with, deploying and running code for the correct cpu is a problem with no established solution.
As for actually writing the code, portable_simd is great. You need to adjust simd width and compiler config for the cpu you deploy to and fill in the blanks with intrinsics. Which is much less work per target than writing it all with raw intrinsics if you are deploying to more than one target.
I wrote a multi-target SIMD-using synthesizer, and I don't think I would've had as much success if I were just using someone else's library - its been especially important to have the SIMD instructions for both architectures I'm supporting (ARM and x86) directly available in the code, since a synthesizer necessarily involves a working pipeline from one stage to the other. I wonder how much easier/better the code would have been to write with portable_simd ..
> I wonder how much easier/better the code would have been to write with portable_simd ..
I had a quick glance of the code and it seems to use mostly basic arithmetic instructions on fixed width simd vectors using some helper macros like SIMD_MUL(x, y) for _mm_mul_ps, etc. Some explicit simd intrinsics code for stuff like sin.
That would've been pretty easy to write with portable_simd in Rust or the equivalent C/C++ language extensions (or maybe C++26 std::simd).
You'd just use f32x4 (or add a typedef with attribute in C++) and then use x*y instead of SIMD_MUL.
For basic stuff like this, you should get the exact same generated code.
SIMD seems to me, to be very platform specific. Maybe there are times one SIMD unit is not anothers' SIMD unit?