Tag Archives: programming

Using build.rs to integrate rust applications with system libraries like a pro

Posted on by Mahmoud Al-Qudsi
Reply

I’m happy to announce the release of version 0.2 of the rsconf crate, with new support for informing Cargo about the presence of custom cfg keys and values (to work around a major change that has resulted in hundreds of warnings for many popular crates under 1.80 nightly).

rsconf itself is a newer crate that was born out of the need (in the fish-shell transition from C++ to rust) for a replacement for some work that’s traditionally been relegated to the build system (e.g. CMake or autoconf) in order to “feature detect” various native system capabilities in the kernel, selected runtime (e.g. libc), or installed libraries. It (optionally) integrates with the popular cc crate so you can test and configure the build toolchain for various underlying features or behavior, and then unlock conditional compilation of native rust code that interops with the system or external libraries accordingly.

Continue reading →

CallerArgumentExpression and extension methods don’t mix

Posted on by Mahmoud Al-Qudsi
Reply

This post is for the C# developers out there and takes a look at the interesting conjunction of [CallerArgumentExpression] and static extension methods – a mix that at first seems too convenient to pass up.

A quick recap: [CallerArgumentExpression] landed as part of the C# 10.0 language update and helps to reduce the (often brittle!) boilerplate involved in, among other uses, creating useful error messages capturing the names of variables or the text of expressions. You tag an optional string method parameter with [CallerArgumentExpression("argName")] where argName is the name of the method argument you want stringified, and the compiler does the rest.

Continue reading →

SecureStore 0.100: KISS, git-versioned secrets management for rust

Posted on by Mahmoud Al-Qudsi
Reply

A few days ago, we published a new version of both the securestore library/crate and the ssclient CLI used to create, manage, and retrieve secrets from SecureStore vaults, an open and cross-language protocol for KISS secrets management. SecureStore vaults provide a more secure and far more reliable solution to storing secrets in environment variables and a simpler and less error prone alternative to network-based secrets management solutions, and make setting up development environments a breeze.

For some background, the SecureStore protocol (first published in 2017) is an open specification and cross-language library/frontend for securely storing encrypted secrets versioned in git, alongside your code. We have implementations available in rust (crate, cli) and for C#/.NET (api and cli, nuget) and the specification is purposely designed to be both easy-to-use and easy-to-port to other languages or frameworks.

This is the first update with (minor) breaking changes to the securestore public api, although pains have been taken to ensure that most common workflows won’t break. The changes are primarily to improve ergonomics when retrieving secrets from rust, and come with completely rewritten docs and READMEs (for the project, the lib, and the cli).

Continue reading →

PrettySize 0.3 release and a weakness in rust’s type system

Posted on by Mahmoud Al-Qudsi
2

PrettySizeI’m happy to announce that a new version of size, the PrettySize.NET port for rust, has been released and includes a number of highly requested features and improvements.

The last major release of the size crate was 0.1.2, released in December of 2018. It was feature complete with regards to its original purpose: the (automatic) textual formatting of file sizes for human-readable printing/display purposes. It would automatically take a file size, pick the appropriate unit (KB, MB, GB, etc) to display the final result in, and choose a suitable precision for the floating numeric component. It had support for both base-10 (KB, MB, GB, etc) and base-2 (KiB, MiB, GiB, etc) types, and the user could choose between them as well as override how the unit was formatted. In short, it did one thing and did it right.

Continue reading →

Compare streams quickly and efficiently in C# and .NET

Posted on by Mahmoud Al-Qudsi
Reply

Have you ever needed to compare the contents of two files (or other streams), and it mattered how quickly you got it done? To be frank, it doesn’t normally come up in the list of things you may need on a daily code-crunching basis, but that rather depends on what kind of programs you tend to write. In our world, let’s just say it’s not an uncommon task.

At a first blush, it would seem to be no harder than comparing two arrays. A pointer reading from each file, compare bytes as you come across them, and bail when things differ. And it would be that easy if you were to use memory-mapped files and let the OS map a file on disk to a range in memory, but that has some drawbacks that may not always be OK depending on what you’re trying to do with the files (or streams) in question. It also requires having a physical path on the filesystem that you can pass in to the kernel, and it unduly burdens the kernel with some not insignificant workloads that aren’t (in practice) subject to the same scheduling and fairness guarantees that user code would be, and they can tend to slow down older machines significantly1.

Continue reading →

  1. Especially under Windows ↩

Faster lookups, insertions, and in-order traversal than a red-black or AVL tree

Posted on by NeoSmart Technologies
Reply

NeoSmart Technologies is pleased to announce the immediate availability of its open source NeoSmart.Collections library/package of specialized containers and collections designed to eke out performance beyond that which is available in typical libraries and frameworks, by purposely optimizing for specific (common!) use cases at the cost of pretty much everything else.

In many regards, the data structures/containers/collections that ship with pretty much any given framework or standard library are a study in compromise. Language and framework developers have no idea what sort of data users will throw at their collections, or in what order. They have no clue whether or not a malicious third party would ultimately be at liberty to insert carefully crafted values into a container, or even what the general ratio of reads to writes would look like. The shipped code must be resilient, fairly performant, free of any obvious pathological cases with catastrophic memory or computation complexities, and above all, dependable.

It isn’t just language and framework developers that are forced to make choices that don’t necessarily align with your own use case. Even when attempting to identify what alternative data structure you could write up and use for your needs, you’ll often be presented with theoretical 𝒪 numbers that don’t necessarily apply or even have any relevance at all in the real world. An algorithm thrown out the window for having a horrible 𝒪 in Algorithms and Data Structures 101 may very well be your best friend if you can reasonably confine it to a certain subset of conditions or input values – and that’s without delving into processor instruction pipelines, execution units, spatial and temporal data locality, branch prediction, or SIMD processing.

Continue reading →

rsevents: manual and auto reset events for rust

Posted on by Mahmoud Al-Qudsi
Reply

One of the unique characteristics of Rust (both the language and the community that has evolved around it) is a strong acknowledgement of multithreading, synchronization, and concurrency, as witnessed in the design of the core language (which acknowledges OS concepts of threads with sync and send) and the presence of various structures in the standard library aimed at simplifying development of correct, multithreaded code.

rsevents is a new crate that should be immediately familiar to anyone that has done multithreaded programming under Windows: it exposes a synchronization primitive, namely, an event for use where Mutex – intended to exclusively marshall access to a variable or region of code – either does not convey the correct semantics or is not the right tool for the job. For those that didn’t come fleeing to rust from a Win32 background, in Windows an event is the lowest-level kernel synchronization primitive, which can be thought of as a “waitable bool” – it is either set or reset (on or off, true or false) and if it isn’t set, you can wait on it until it becomes set.

Continue reading →

Transparent encryption and decryption in rust with cryptostreams

Posted on by Mahmoud Al-Qudsi
2

C# developers have long been spoiled when it comes to quickly and easily getting up and running with encryption thanks to the .NET CryptoStream  class, which wraps a Stream instance in a second Stream that automatically encrypts/decrypts anything read/written to/from it before passing it along to the underlying Stream. Long story short, it makes it ridiculously easy to add encryption or decryption facilities to an existing pipeline, as after setting up the CryptoStream instance you can just treat it like any other Stream object and read or write to it normally.

Encryption has been somewhat of a sore spot in the rust ecosystem after a few false starts with “native” rust encryption libraries that went nowhere, but today the rust community has fortunately adopted the OpenSSL bindings as the approach of choice, and the rust-openssl crate makes it easy to both bundle and consume the openssl bindings from rust in a cross-platform manner. What it doesn’t do is make encryption and decryption any easier than OpenSSL itself does.

Continue reading →

PrettySize for rust

Posted on by Mahmoud Al-Qudsi
Reply

We’ve just published a rust port of our PrettySize.NET library, now available via cargo and github. Like its .NET predecessor, PrettySize-rs aims to provide a comprehensive API for dealing with file sizes, covering both manipulation and human-readable formatting.

Continue reading →

Modern C++ isn’t memory safe, either

Posted on by Mahmoud Al-Qudsi
11

A recurring theme in just about all discussions revolving around the comparison of programming languages – apart from using the wrong tool for the job, adamantly pushing a language objectively/demonstrably inferior at x out of blind loyalty, bashing on languages you’ve never used or studied simply because you’ve seen firsthand how well received such comments can be, and worse – is acting off of stale information that no longer necessarily holds true.

At NeoSmart Technologies, we don’t just have one dog in the race; our software is developed in a multitude of languages, ranging from C/C++ to both desktop/web C#/ASP.NET, rust, [JS|TypeScript]/HTML/[LESS|CSS], (ba)sh scripting, and more.1 So it’s always interesting to observe these discussions (sometimes up close and personal and sometimes disinterestedly from afar) and observe what arguments remain standing once the dust has settled and the troops have gone home for the day.

Continue reading →

  1. Gasp, yes, even PHP! ↩