Using build.rs to integrate rust applications with system libraries like a pro

I’m happy to announce the release of version 0.2 of the rsconf crate, with new support for informing Cargo about the presence of custom cfg keys and values (to work around a major change that has resulted in hundreds of warnings for many popular crates under 1.80 nightly).

rsconf itself is a newer crate that was born out of the need (in the fish-shell transition from C++ to rust) for a replacement for some work that’s traditionally been relegated to the build system (e.g. CMake or autoconf) in order to “feature detect” various native system capabilities in the kernel, selected runtime (e.g. libc), or installed libraries. It (optionally) integrates with the popular cc crate so you can test and configure the build toolchain for various underlying features or behavior, and then unlock conditional compilation of native rust code that interops with the system or external libraries accordingly.

While Cargo is an impressive build tool and normally more than sufficient for the needs of the majority of rust crates shipping standalone, contained libraries or packages, for those of us transitioning “more brittle” system software or libraries that rely on functionality of the native kernel, libc, or external libraries – and often have to support older versions thereof, lacking in features or enhancements – it doesn’t offer feature parity with some of the build tools we’ve been traditionally using in the C and C++ world.

Fortunately, the language and tooling devs behind rust recognized early on the need for a more flexible and involved approach to building certain crates and came up with the build.rs approach that lets developers run what is effectively a rust script prior to the traditional build steps invoked by Cargo, influencing how Cargo works, what flags are passed to rustc (the actual rust compiler), and what libraries Cargo asks your linker to include when generating the final binary. Importantly, at this stage devs can inspect the host/target systems and tell the compiler which Cargo features and rust cfgs should be enabled, letting you conditionally compile (or not) various bits and pieces of regular rust code to take advantage of functionality discovered to be present at build time or work around capabilities found to be missing or lacking.

In addition to supporting external libraries (like gettext and, once upon a time, curses), it also runs on quite a number of different unixy systems, starting with Linux, macOS, and the BSDs, and with a long tail of support for various other fun posix-compatible operating systems and projects.¹ As you can imagine, as a shell, fish does a lot of stuff outside the purview of the rust standard library and a lot of the codebase deals with low-level integration with the operating system – and the details of this change quite a bit just from one kernel version to the other, let alone across different OS kernels and distributions altogether.

A lot of the crates in the rust ecosystem seem to rely on OS detection to determine what feature should or shouldn’t be available, but the C/C++ world moved from that to feature detection a long time ago. Fish uses build.rs to determine what low-level operating system features are available (regardless of what OS we are targeting) and enables or disables the compilation of rust code sitting behind #[cfg(key = "value")] depending on the results.

Cargo exposes a very bare-bones mechanism for build.rs to influence which cfg or feature will be enabled/disabled at build-time by means of intercepting specially prefixed stdout messages. Generally, this would look like

1. In build.rs, check for the presence/absence of some feature somehow,
2. In response, execute println!("cargo:rustc-cfg=foo");

Which prints to stdout a line of text Cargo will intercept when it is running the compiled build.rs binary and, based off of that, tell rustc to enable the cfg foo. This “unlocks” code behind a #[cfg(foo)], allowing rustc to see and compile it (normally it would be as if it weren’t there at all).

This is all well and good, but it has a few obvious drawbacks. The first is that somehow that glares at us from point number one above. How exactly does one check if a system feature is present or not? Why doesn’t Cargo help us in this regard? In the world of legacy build systems, this is part and parcel of what a build system does and, in fact, a raison d’être for their existence in a world that predates package managers, semantic versioning, and all the other nice stuff we can now take for granted in a rust-native ecosystem.

It’s from this need that rsconf was born. The crate offers some of the functionality typically made available by “legacy” build systems, wrapped in an easy-to-use and rust-friendly api. Some examples of the available functionality:

• system.has_library(libname)
• system.has_header(name)
• system.has_symbol(symbol)
• system.has_symbol_in(symbol, &[libname])
• system.has_type(type_name)
• system.get_macro_value(name)
• system.get_{u32,u64,i32,i64}_value(ident)
• system.ifdef(define)
• system.if(expr)
• add_library_search_path(path)
• link_library(libname, LinkType::Static/Dynamic)

The names of these methods should make what they do quite clear, and there are various convenience functions to simplify some common patterns around these same principles. The methods themselves are largely implemented as build-time attempts to compile or link minimal C source code to determine the truthiness of the expressions, while the latter two direct Cargo as to how it should attempt find and use external libraries pre-installed on the build host/target.²

In addition, there are methods that make it easier to perform “regular” Cargo stuff in a build.rs script, offering a “strongly typed” api instead of the usual println!() stuff that is prone to typos, mangled types, and more:

• enable_cfg(cfg_name)
• enable_feature(feature_name)
• set_cfg_value(cfg_name, cfg_value)
• rebuild_if_env_changed(env_var_name)
• rebuild_if_path_changed(path)

New to the 0.2.0 rsconf release (as hinted at above) are variations on enable_cfg() and set_cfg_value(), necessitated by changes to the rust compiler that will land in 1.80.³ The compiler will begin checking expressions such as #[cfg(foo)], #[cfg(foo = "bar")], and #[cfg(feature = "baz")], to make sure that all of foo, bar, and baz are valid constraints (not typos or hallucinations). As you can imagine, if the compiler comes across the attribute cfg!(hello) while the hello cfg is enabled, it knows that it’s a valid cfg name. But what about when it’s not enabled? So now we have to let rustc know up front not only which cfg or feature names/values are valid and active, we also have to let it know which are valid even when they’re inactive. To that end, rsconf 0.2.0 introduces the following:

• declare_cfg(cfg_name: &str, enabled: bool)
• declare_cfg_values(cfg_name: &str, valid_values: &[&str])
• declare_feature(feature: &str, enabled: bool)

The first can be used directly in lieu of the old enable_cfg() to both declare a cfg and specify that it is to be enabled (or disabled), but, for now, the second needs to be used in conjunction with the existing set_cfg_value(cfg_name, value) to let the compiler know in advance all the valid values, regardless of whether they’re defined for the current compilation or not.⁴

If you’re curious, you can take a quick look at fish-shell’s current build.rs to get an idea of what real-world rsconf usage in the wild looks like. It’s a short build script, and quite easy to understand.

The rsconf crate itself is still in its early days and will, DV, continue to evolve and see new features. If there are build-related tests or tasks that you feel would fall under its purview, do open an issue in the repository and let us know. Feedback about the proposed builder api for declaring cfg values is also welcome! Otherwise, please give it a try and see if it can help you make your build.rs sane and easier to understand. It’s intentionally written to be fast and lite, with only a dependency on the cc-rs crate (which you’ll almost certainly already be taking a dependency on if you’re compiling against system libraries or headers), so you only stand to benefit from making the switch!

Looking for something else to read or learn? Take a look at my other rust posts, especially this one about designing truly safe semaphores in rust or learn about using simd to speed up rust applications significantly. Sign up below to get emails about open source rust stuff or add my blog to your RSS reader!

1. Unfortunately, as a part of the transition from C++ to rust, we have currently lost support for the vast majority of the legacy OSes and other non-tier-1 unix platforms we used to support due to issues with availability of the rust toolchain and incompatible dependencies like the libc crate, but we are keen to get them back. ↩

2. One thing I really appreciate about Cargo is that it goes to great lengths to support cross-compilation out-of-the-box, clarifying operations and configurations taken from/applying to the machine you are building on (the host) vs the machine you are building for (the target). rsconf is written in a way that similarly respects this divide. ↩

3. You can also refer to this rustc documentation page for more on how this new feature works. ↩

4. A separate “builder-style” api will be added at some point, but there are decisions to make about its shape. For example, it could look like add_cfg("cfg_name").with_values(&["value1", "value2"]).set_value("value1") or along the lines of add_cfg("cfg_name").set_values("active_value", &["other value 1", "other value 2"]) ↩