{"id":5351,"date":"2026-01-28T13:26:43","date_gmt":"2026-01-28T19:26:43","guid":{"rendered":"https:\/\/neosmart.net\/blog\/?p=5351"},"modified":"2026-01-28T13:29:12","modified_gmt":"2026-01-28T19:29:12","slug":"sharding-uuidv7-and-uuid-v3-v4-and-v5-with-one-function","status":"publish","type":"post","link":"https:\/\/neosmart.net\/blog\/sharding-uuidv7-and-uuid-v3-v4-and-v5-with-one-function\/","title":{"rendered":"Sharding UUIDv7 (and UUID v3, v4, and v5) values with one function"},"content":{"rendered":"

UUIDv7 (wiki link<\/a>) is seeing strong and eager adoption as a solution to problems that have long plagued the tech industry, providing a solution for generating collision-free IDs on the backend that could still be sorted chronologically to play nicer with database indexes and other needs.1<\/a><\/sup> As a quick briefer, a UUIDv7 is essentially composed of two parts: a timestamp half and a randomized bytes half, and they’re sorted by the timestamp:<\/p>\n

\"\"<\/a><\/p>\n

Let’s say you want to shard based off a UUIDv7 key: maybe to assign data across a horizontally-distributed database, or to avoid having a couple of million files in the same directory. You can’t use the timestamp portion to derive a shard key, as not even the microseconds are necessarily uniformly distributed, so you need to shard based off the random portion of the UUID.<\/p>\n

Lucky for us, the last eight bytes of the UUID contain random data not just in the case of a UUIDv7 \u2013 those bytes\u00a0also<\/em> contain random data in the case of UUID v3 (MD5-hashed), UUID v4 (randomized), and UUID v5 (like v3 but using SHA-1), letting us use one approach to hash all four of these UUID types.<\/p>\n

With some judicious use of the stabilized subset of const generics, we can generate a shard key N<\/code>-chars long without any allocation, and provide a wrapper ShardKey<\/code> type to simplify both creating the shard key and passing it around in a way that lets it decompose to a &str<\/code> in a way that wouldn’t have been possible if we had returned a [char; N]<\/code> instead.2<\/a><\/sup><\/p>\n

\/\/\/ Derives a deterministic N-char sharding key from a UUID.\r\n\/\/\/\r\n\/\/\/ It extracts entropy starting from the last byte (the most random \r\n\/\/\/ part of a UUIDv7) moving backwards, converting nibbles to \r\n\/\/\/ hexadecimal characters. It uses only the last 8 bytes (the `rand_b`\r\n\/\/\/ section) to ensure the timestamp (first 6 bytes) is never touched, \r\n\/\/\/ preventing \"hot spot\" sharding issues based on time.\r\n\/\/\/\r\n\/\/\/ # Panics\r\n\/\/\/ * Panics if the provided UUID is not shardable based off the last \r\n\/\/\/   8 bytes (i.e. if the UUID is not one a v3, v4, v5, or v7 UUID).\r\n\/\/\/ * Panics if the requested shard key length `N` exceeds the \r\n\/\/\/   available entropy.\r\npub fn shard<const N: usize>(uuid: &uuid::Uuid) -> ShardKey<N> {\r\n    \/\/ Ensure UUID random bytes assumption\r\n    if !matches!(uuid.get_version_num(), 3 | 4 | 5 | 7) {\r\n        panic!(\"Provided UUID cannot be sharded with this interface!\");\r\n    }\r\n    \/\/ Validate shard key length\r\n    if N > 15 {\r\n        panic!(\"Requested shard key length exceeds available entropy!\");\r\n    }\r\n\r\n    const HEX: &[u8; 16] = b\"0123456789abcdef\";\r\n    let bytes = uuid.as_bytes();\r\n    let mut result = [b'\\0'; N];\r\n\r\n    \/\/ Generate N-char shard key\r\n    for i in 0..N {\r\n        \/\/ We extract data from the tail (byte 15) backwards to byte 8.\r\n        \/\/ Bytes 8-15 in UUIDv7 contain 62 bits of randomness + 2 bits \r\n        \/\/ to indicate variant. We avoid bytes 0-7 (Timestamp + Version) \r\n        \/\/ to ensure uniform distribution.\r\n\r\n        \/\/ Calculate byte index, consuming two nibbles per byte.\r\n        let inverse_offset = i \/ 2;\r\n        let byte_index = 15 - inverse_offset;\r\n\r\n        let byte = bytes[byte_index];\r\n\r\n        \/\/ Even indices take the low nibble; odd indices take the high nibble.\r\n        let nibble = if i % 2 == 0 {\r\n            byte & 0x0F\r\n        } else {\r\n            (byte >> 4) & 0x0F\r\n        };\r\n\r\n        result[i] = HEX[nibble as usize];\r\n    }\r\n\r\n    ShardKey { key: result }\r\n}\r\n\r\n#[repr(transparent)]\r\n#[derive(Copy, Clone, PartialOrd, Ord, PartialEq, Eq, Hash)]\r\npub struct ShardKey<const N: usize> {\r\n    key: [u8; N],\r\n}\r\n\r\nimpl<const N: usize> ShardKey<N> {\r\n    pub fn new(uuid: &uuid::Uuid) -> Self {\r\n        shard(uuid)\r\n    }\r\n\r\n    pub fn as_str(&self) -> &str {\r\n        \/\/ Safe because only we have access to this field and we always\r\n        \/\/ initialize it fully with ASCII-only characters.\r\n        unsafe { std::str::from_utf8_unchecked(&self.key) }\r\n    }\r\n}\r\n<\/code><\/pre>\n
The shard()<\/code> function can be used standalone, or the ShardKey<\/code> wrapper type can be created from a reference to a &Uuid<\/code> and then passed around until the point where you need to retrieve the key with a quick .as_str()<\/code> call. As promised, the length of the shard key itself is variable and parameterized, by providing different values for N<\/code> you can generate shard keys from one to fifteen (or sixteen, if you prefer) characters long (past which point the available uniformly-distributed entropy has been exhausted and the function will panic to ensure contract is upheld).<\/p>\n
  1. Of course, non-standardized solutions abound and UUIDv7 itself takes a lot of inspiration from predecessors like Ulid<\/a> and others. ↩<\/a><\/p><\/li>
  2. As rust is famously one of the very few languages that uses different-sized units\/types to represent a standalone char<\/code> versus the smallest type used to compose a String<\/code>; in rust, char<\/code> is 4-bytes long and has UTF-32 encoding while a String<\/code> or &str<\/code> uses an underlying 1-byte u8<\/code> type and are UTF8-encoded. This means you can’t go from a String<\/code> to a [char]<\/code> without allocating, nor go from a [N; char]<\/code> to an &str<\/code> without looping and allocating, either. ↩<\/a><\/p><\/li><\/ol>","protected":false},"excerpt":{"rendered":"
    UUIDv7 (wiki link) is seeing strong and eager adoption as a solution to problems that have long plagued the tech industry, providing a solution for generating collision-free IDs on the backend that could still be sorted chronologically to play nicer … Continue reading →<\/span><\/a><\/p>\n","protected":false},"author":505,"featured_media":5359,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_feature_clip_id":0,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_post_was_ever_published":false},"categories":[1],"tags":[936,1044,1042,1043],"class_list":["post-5351","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-software","tag-rust","tag-sharding","tag-uuid","tag-uuidv7"],"aioseo_notices":[],"aioseo_head":"\n\t\t\n\t\n\t\n\t\n\t\n\t\t\n\t\t