Rust Error: Could not find `avx` in `x86`

Let’s start this article with an error message right off the bat:¹

~> cargo install ripgrep --features 'avx-accel simd-accel'
    Updating registry `https://github.com/rust-lang/crates.io-index`
  Installing ripgrep v0.9.0
…
error[E0432]: unresolved import `simd::x86::avx`
  --> /home/mqudsi/.cargo/registry/src/github.com-1ecc6299db9ec823/bytecount-0.3.2/src/lib.rs:49:16
   |
49 | use simd::x86::avx::{LowHigh128, u8x32};
   |                ^^^ Could not find `avx` in `x86`
error: aborting due to previous error
For more information about this error, try `rustc --explain E0432`.
error: Could not compile `bytecount`.

Does the error look familiar? It’s a common one. The rust compiler does not avail itself of processor-specific features and capabilities needed to perform advanced optimizations or to use vectorized instructions like the SSE{,2,3,4{,.1,.2}}² or AVX intrinsics. But some cargo crates have features that take advantage of these features — and depending on your platform and architecture, compilation will fail with an error like the one above.

While bytecount is a common culprit, it’s by no means the only one, and you can encounter this error (or one like it) with pretty much any crate trying to use lower-level CPU instructions. Fortunately in this case if we were to try compiling/installing ripgrep normally without the --features ... bit we would not encounter this error at all, but what’s the use of using the fastest code search utility on the planet on an advanced CPU you paid good money for if you’re not planning on milking every last drop of performance you can get out of it?

The solution is to explicitly instruct the rustc compiler to use all the capabilities of your CPU, including those not necessarily found on other machines in your platform (which will cause the binaries you build to no longer be guaranteed to run on another ABI-compatible machine).³ If you set the RUSTC_FLAGS environment variable prior to invoking cargo install ... to instruct the compiler to use the native capabilities of the CPU, that’s exactly what we get:

~> env RUSTFLAGS="-C target-cpu=native" \
       cargo install ripgrep --features 'avx-accel simd-accel'
    Updating registry `https://github.com/rust-lang/crates.io-index`
  Installing ripgrep v0.9.0
    Finished release [optimized + debuginfo] target(s) in 58.88s
   Replacing /home/mqudsi/.cargo/bin/rg

Note that you may see a message like the following:

‘native’ is not a recognized processor for this target (ignoring processor)

This is perfectly normal — some crates try to introspect the current value of target-cpu to take differing build actions, and are unable to identify native as a valid CPU target (which makes sense, since it’s really a meta target and not a real one).

1. I’m primarily blogging this as I’ve run into – and worked around – this error several times before running into it again and not being able to remember how I resolved this problem the last time around. ↩

2. fish’s nested parameter expansion ftw! ↩

3. There is a proposal to add a crate-level profiling option to enable this automatically over at rust-lang/cargo#2535. ↩