Compare streams quickly and efficiently in C# and .NET

Posted on by Mahmoud Al-Qudsi
Reply

Have you ever needed to compare the contents of two files (or other streams), and it mattered how quickly you got it done? To be frank, it doesn’t normally come up in the list of things you may need on a daily code-crunching basis, but that rather depends on what kind of programs you tend to write. In our world, let’s just say it’s not an uncommon task.

At a first blush, it would seem to be no harder than comparing two arrays. A pointer reading from each file, compare bytes as you come across them, and bail when things differ. And it would be that easy if you were to use memory-mapped files and let the OS map a file on disk to a range in memory, but that has some drawbacks that may not always be OK depending on what you’re trying to do with the files (or streams) in question. It also requires having a physical path on the filesystem that you can pass in to the kernel, and it unduly burdens the kernel with some not insignificant workloads that aren’t (in practice) subject to the same scheduling and fairness guarantees that user code would be, and they can tend to slow down older machines significantly1.

Continue reading →

  1. Especially under Windows ↩

Easy Window Switcher 1.2.2

Posted on by Mahmoud Al-Qudsi
6

Just a quick heads-up: Easy Window Switcher 1.2.2 has just been released and it brings correct hotkey support for the following keyboard layouts:

  • Swedish
  • German Swiss (QWERTZ)
  • International Spanish

Continue reading →

.NET Standard and new .NET Framework Logo and Banner

Posted on by Mahmoud Al-Qudsi
2

For yesterday’s NeoSmart.Collections release announcement, we had need of an updated logo for Microsoft’s .NET, and were unable to find something corresponding to .NET Standard.

For those of us that were around when C# was first introduced, this is probably the logo that most represents the .NET Framework:1

Continue reading →

  1. I tried extremely hard to find this in a higher resolution or even just losslessly resized, but to no avail. Give me a holler if you have something better we can preserve! ↩

Faster lookups, insertions, and in-order traversal than a red-black or AVL tree

Posted on by NeoSmart Technologies
Reply

NeoSmart Technologies is pleased to announce the immediate availability of its open source NeoSmart.Collections library/package of specialized containers and collections designed to eke out performance beyond that which is available in typical libraries and frameworks, by purposely optimizing for specific (common!) use cases at the cost of pretty much everything else.

In many regards, the data structures/containers/collections that ship with pretty much any given framework or standard library are a study in compromise. Language and framework developers have no idea what sort of data users will throw at their collections, or in what order. They have no clue whether or not a malicious third party would ultimately be at liberty to insert carefully crafted values into a container, or even what the general ratio of reads to writes would look like. The shipped code must be resilient, fairly performant, free of any obvious pathological cases with catastrophic memory or computation complexities, and above all, dependable.

It isn’t just language and framework developers that are forced to make choices that don’t necessarily align with your own use case. Even when attempting to identify what alternative data structure you could write up and use for your needs, you’ll often be presented with theoretical 𝒪 numbers that don’t necessarily apply or even have any relevance at all in the real world. An algorithm thrown out the window for having a horrible 𝒪 in Algorithms and Data Structures 101 may very well be your best friend if you can reasonably confine it to a certain subset of conditions or input values – and that’s without delving into processor instruction pipelines, execution units, spatial and temporal data locality, branch prediction, or SIMD processing.

Continue reading →

ignore-result: A simple crate to make ignoring rust Results safe and easy

Posted on by Mahmoud Al-Qudsi
3

Rust, both by design and by convention, has a fairly strongly defined model for strict error handling, designed to force developers to deal with errors up front rather than assume they don’t exist. If you stick to a few conventions and principles for best practice, error handling becomes fairly straight-forward (although what you ultimately do with the errors is a different question) if you are living in an all-rust world. The problems start when you step foot outside of the comfortable world of crates and Results, such as when dealing with FFI to interface with C libraries or using rust in an embedded context.

The typical approach for dealing with errors in rust in 2018 is to have any function that can encounter a scenario wherein it is unable to return a valid value declare a return type of Result<T, E> where T is the expected result type in normal cases and E is a type covering possible errors that can arise during execution. When calling from one such function into other functions with non-guaranteed success, the typical control flow in the event of an error is almost always “early out”:

Continue reading →

rsevents: manual and auto reset events for rust

Posted on by Mahmoud Al-Qudsi
Reply

One of the unique characteristics of Rust (both the language and the community that has evolved around it) is a strong acknowledgement of multithreading, synchronization, and concurrency, as witnessed in the design of the core language (which acknowledges OS concepts of threads with sync and send) and the presence of various structures in the standard library aimed at simplifying development of correct, multithreaded code.

rsevents is a new crate that should be immediately familiar to anyone that has done multithreaded programming under Windows: it exposes a synchronization primitive, namely, an event for use where Mutex – intended to exclusively marshall access to a variable or region of code – either does not convey the correct semantics or is not the right tool for the job. For those that didn’t come fleeing to rust from a Win32 background, in Windows an event is the lowest-level kernel synchronization primitive, which can be thought of as a “waitable bool” – it is either set or reset (on or off, true or false) and if it isn’t set, you can wait on it until it becomes set.

Continue reading →

An unhandled exception in the English language

Posted on by Mahmoud Al-Qudsi
1

While no one expects a language that formed organically via iteration and evolution to be free of logical errors, over time conventions have been formed to deal with most of them such that for the most part there is a “right” way of saying something and a “wrong” way, even when there’s no obvious rule to cover the use case in mind.

The BBC has an excellent article with some examples and even past attempts at creating rules to deal with these cases.

“Adjectives in English absolutely have to be in this order: opinion-size-age-shape-colour-origin-material-purpose Noun. So you can have a lovely little old rectangular green French silver whittling knife. But if you mess with that word order in the slightest you’ll sound like a maniac. It’s an osadd thing that every English speaker uses that list, but almost none of us could write it out.”

My personal favorite “it’ll blow your mind” rule is with regards to ablaut reduplication:

You are utterly familiar with the rule of ablaut reduplication. You’ve been using it all your life. It’s just that you’ve never heard of it. But if somebody said the words zag-zig, or ‘cross-criss you would know, deep down in your loins, that they were breaking a sacred rule of language. You just wouldn’t know which one. All four of a horse’s feet make exactly the same sound. But we always, always say clip-clop, never clop-clip. Every second your watch (or the grandfather clock in the hall makes the same sound) but we say tick-tock, never tock-tick. You will never eat a Kat Kit bar. The bells in Frère Jaques will forever chime ‘ding dang dong’.

But what do you do when two rules or conventions clash? And what does that have to do with programming or computer science?

Continue reading →

Transparent encryption and decryption in rust with cryptostreams

Posted on by Mahmoud Al-Qudsi
2

C# developers have long been spoiled when it comes to quickly and easily getting up and running with encryption thanks to the .NET CryptoStream  class, which wraps a Stream instance in a second Stream that automatically encrypts/decrypts anything read/written to/from it before passing it along to the underlying Stream. Long story short, it makes it ridiculously easy to add encryption or decryption facilities to an existing pipeline, as after setting up the CryptoStream instance you can just treat it like any other Stream object and read or write to it normally.

Encryption has been somewhat of a sore spot in the rust ecosystem after a few false starts with “native” rust encryption libraries that went nowhere, but today the rust community has fortunately adopted the OpenSSL bindings as the approach of choice, and the rust-openssl crate makes it easy to both bundle and consume the openssl bindings from rust in a cross-platform manner. What it doesn’t do is make encryption and decryption any easier than OpenSSL itself does.

Continue reading →

Could not find `avx` in `x86`

Posted on by Mahmoud Al-Qudsi
Reply

Let’s start this article with an error message right off the bat:1

~> cargo install ripgrep --features 'avx-accel simd-accel'
    Updating registry `https://github.com/rust-lang/crates.io-index`
  Installing ripgrep v0.9.0
…
error[E0432]: unresolved import `simd::x86::avx`
  --> /home/mqudsi/.cargo/registry/src/github.com-1ecc6299db9ec823/bytecount-0.3.2/src/lib.rs:49:16
   |
49 | use simd::x86::avx::{LowHigh128, u8x32};
   |                ^^^ Could not find `avx` in `x86`
error: aborting due to previous error
For more information about this error, try `rustc --explain E0432`.
error: Could not compile `bytecount`.

Continue reading →

  1. I’m primarily blogging this as I’ve run into – and worked around – this error several times before running into it again and not being able to remember how I resolved this problem the last time around. ↩