Benchmarking rust compilation speedups and slowdowns from sccache and -Zthreads

Posted on by Mahmoud Al-Qudsi
2

Just a PSA from one rust developer to another: if you use sccache, take a moment to benchmark a clean build1 of your favorite or current project and verify whether or not having RUSTC_WRAPPER=sccache is doing you any favors.

I’ve been an sccache user almost from the very start, when the Mozilla team first introduced it in the rust discussions on GitHub maybe seven years back or so, probably because of my die-hard belief in the one-and-only ccache earned over long years of saving my considerable time and effort on C++ development projects (life pro-tip: git bisect on large C or C++ projects is a night-and-day difference with versus without ccache). At the same time, I was always painfully aware of just how little sccache actually cached compared to its C++ progenitor, and I was left feeling perpetually discontented ever since learning to query its hit rate stats with sccache -s (something I never needed to do for ccache).

But my blind belief in the value of build accelerators led me to complacency, and I confess that with sccache mostly chugging away without issue in the background, I kind of forgot that I had RUSTC_WRAPPER set at all. But I recently remembered it and in a bout of procrastination, decided to benchmark how much time sccache was actually saving me… and the results were decidedly not great.

Continue reading →

  1. sccache does not cache nor speed up incremental builds, and recent versions try to more or less bypass the caching pipeline altogether in an attempt to avoid slowing down incremental builds. ↩

Using build.rs to integrate rust applications with system libraries like a pro

Posted on by Mahmoud Al-Qudsi
Reply

I’m happy to announce the release of version 0.2 of the rsconf crate, with new support for informing Cargo about the presence of custom cfg keys and values (to work around a major change that has resulted in hundreds of warnings for many popular crates under 1.80 nightly).

rsconf itself is a newer crate that was born out of the need (in the fish-shell transition from C++ to rust) for a replacement for some work that’s traditionally been relegated to the build system (e.g. CMake or autoconf) in order to “feature detect” various native system capabilities in the kernel, selected runtime (e.g. libc), or installed libraries. It (optionally) integrates with the popular cc crate so you can test and configure the build toolchain for various underlying features or behavior, and then unlock conditional compilation of native rust code that interops with the system or external libraries accordingly.

Continue reading →

Embed only the video from another post on X or Twitter

Posted on by Mahmoud Al-Qudsi
2

Twitter has a new-ish feature that lets you embed only the video from another post or tweet in a post/tweet of your own (without quote-replying the source tweet itself). Only the video is then embedded in your post, and a small attribution appears at the bottom identifying where the video came from:

In the screenshot above, Sarah is sharing a video that was originally shared by Luc, but she’s not embedding/quoting Luc’s tweet itself – only the video. This post will cover how to do that yourself, both on the desktop/web and in the iOS Twitter app on iPhone.

Continue reading →

Increment only numbers matching regex in Vim

Posted on by Mahmoud Al-Qudsi
5

Long-time vim or neovim users are probably already aware that visually selecting a block of text then pressing CTRL + A in vim will result in any numbers in the selected block of text being incremented by 1. This works even if the block contains non-numeric text: each group of digits gets treated as a number and is incremented.1

For example, here’s a video that shows what happens when you select some text in vim and then use CTRL + A to increment the values:

Continue reading →

  1. To be pedantic, only the first group-of-digits/number on each line gets incremented; like many vim commands this only works on the first match per line of text unless some sort /g global modifier is used. ↩

tcpproxy 0.4 released

Posted on by Mahmoud Al-Qudsi
Reply

Image courtesy of Hack A Day

This blog post was a bit delayed in the pipeline, but a new release of tcproxy, our educational async (tokio) rust command line proxy project, is now available for download (precompiled binaries or install via cargo).

I was actually surprised to find that we haven’t written about tcpproxy before (you can see our other rust-related posts here), but it’s a command line tcp proxy “server” written with two purposes in mind: a) serving as a real-world example of an async (tokio-based) rust networking project, and b) serving as a minimal but-still-useful tcp proxy you can run and use directly from the command line, without needing complex installation or configuration procedures. (You can think of it as being like Minix, but for rust and async networking.)

The tcpproxy project has been around for quite some time, originally published in 2017 before rust’s async support was even stabilized. At the time, it manually chained futures to achieve scalability without relying on the thread-per-connection model – but today its codebase is a lot easier to follow and understand thanks to rust’s first-class async/await support.

Continue reading →

CallerArgumentExpression and extension methods don’t mix

Posted on by Mahmoud Al-Qudsi
Reply

This post is for the C# developers out there and takes a look at the interesting conjunction of [CallerArgumentExpression] and static extension methods – a mix that at first seems too convenient to pass up.

A quick recap: [CallerArgumentExpression] landed as part of the C# 10.0 language update and helps to reduce the (often brittle!) boilerplate involved in, among other uses, creating useful error messages capturing the names of variables or the text of expressions. You tag an optional string method parameter with [CallerArgumentExpression("argName")] where argName is the name of the method argument you want stringified, and the compiler does the rest.

Continue reading →

Implementing truly safe semaphores in rust

Posted on by Mahmoud Al-Qudsi
4

Discuss this article on r/rust or on Hacker News.

Low-level or systems programming languages generally strive to provide libraries and interfaces that enable developers, boost productivity, enhance safety, provide resistance to misuse, and more — all while trying to reduce the runtime cost of such initiatives. Strong type systems turn runtime safety/sanity checks into compile-time errors, optimizing compilers try to reduce an enforced sequence of api calls into a single instruction, and library developers think up of clever hacks to even completely erase any trace of an abstraction from the resulting binaries. And as anyone that’s familiar with them can tell you, the rust programming language and its developers/community have truly embraced this ethos of zero-cost abstractions, perhaps more so than any others.

I’m not going to go into detail about what the rust language and standard library do to enable zero-cost abstractions or spend a lot of time going over some of the many examples of zero-cost interfaces available to rust programmers, though I’ll just quickly mention a few of my favorites: iterators and all the methods the Iterator trait exposes have to be at the top of every list given the amount of black magic voodoo the compiler has to do to turn these into their loop-based equivalents, zero-sized types make developing embedded firmware in rust a dream and it’s really crazy to see how all the various peripheral abstractions can be completely erased giving you small firmware blobs despite all the safety abstractions, and no list is complete the newest member of the team, async/await and how rust manages to turn an entire web server api into a single state machine and event loop. (And to think this can be used even on embedded without a relatively heavy async framework like tokio and with even zero allocations to boot!)

Continue reading →

SecureStore 0.100: KISS, git-versioned secrets management for rust

Posted on by Mahmoud Al-Qudsi
Reply

A few days ago, we published a new version of both the securestore library/crate and the ssclient CLI used to create, manage, and retrieve secrets from SecureStore vaults, an open and cross-language protocol for KISS secrets management. SecureStore vaults provide a more secure and far more reliable solution to storing secrets in environment variables and a simpler and less error prone alternative to network-based secrets management solutions, and make setting up development environments a breeze.

For some background, the SecureStore protocol (first published in 2017) is an open specification and cross-language library/frontend for securely storing encrypted secrets versioned in git, alongside your code. We have implementations available in rust (crate, cli) and for C#/.NET (api and cli, nuget) and the specification is purposely designed to be both easy-to-use and easy-to-port to other languages or frameworks.

This is the first update with (minor) breaking changes to the securestore public api, although pains have been taken to ensure that most common workflows won’t break. The changes are primarily to improve ergonomics when retrieving secrets from rust, and come with completely rewritten docs and READMEs (for the project, the lib, and the cli).

Continue reading →

C# file size formatting library PrettySize 3.1 released

Posted on by Mahmoud Al-Qudsi
Reply

PrettySizeHot on the heels of an update to our rust port of PrettySize we have a new release of PrettySize.NET that brings new features and capabilities to the best .NET library for formatting file sizes for human-readable output and display.

PrettySize 3.1, available on GitHub and via Nuget, has just been released and contains a number of improvements and requested features and newfound abilities to make handling file sizes (and not just formatting them) easier and more enjoyable.

Continue reading →

PrettySize 0.3 release and a weakness in rust’s type system

Posted on by Mahmoud Al-Qudsi
2

PrettySizeI’m happy to announce that a new version of size, the PrettySize.NET port for rust, has been released and includes a number of highly requested features and improvements.

The last major release of the size crate was 0.1.2, released in December of 2018. It was feature complete with regards to its original purpose: the (automatic) textual formatting of file sizes for human-readable printing/display purposes. It would automatically take a file size, pick the appropriate unit (KB, MB, GB, etc) to display the final result in, and choose a suitable precision for the floating numeric component. It had support for both base-10 (KB, MB, GB, etc) and base-2 (KiB, MiB, GiB, etc) types, and the user could choose between them as well as override how the unit was formatted. In short, it did one thing and did it right.

Continue reading →