{"id":3894,"date":"2017-02-23T13:42:54","date_gmt":"2017-02-23T19:42:54","guid":{"rendered":"http:\/\/neosmart.net\/blog\/?p=3894"},"modified":"2017-02-24T06:43:18","modified_gmt":"2017-02-24T12:43:18","slug":"rewrite-a-rust-powered-in-place-file-rewrite-utility","status":"publish","type":"post","link":"https:\/\/neosmart.net\/blog\/rewrite-a-rust-powered-in-place-file-rewrite-utility\/","title":{"rendered":"rewrite: a rust-powered, in-place file rewrite utility"},"content":{"rendered":"

<\/a>Let’s say you’ve got a terminal open and you want to sort the contents of a file before you email it to a friend. The file can contain anything and it could be of any length, it doesn’t matter. What do you do?<\/p>\n

The obvious answer is to use sort<\/code>. Sorting the file is as easy as sort myfile<\/code> – except it doesn’t actually sort the file<\/em>, it sorts the *contents* of the file and dumps them to the command line (via stdout<\/code>). So how do you sort the file “in-place,” so-to-speak? Again, the obvious answer would be sort myfile > myfile<\/code>,1<\/a><\/sup> redirecting the output of the sort command back to the file you want to ultimately send sorted.<\/p>\n

<\/p>\n

The only problem is that if you did the command above, you’d likely lose your data entirely. You see, sort<\/code> is too smart to read the entire file into memory, sort it, then spit it out on stdout<\/code>, which is how it can manage sorting huge amounts of data without completely trashing your system memory. It keeps a handle to the input file open and reads even as it writes to stdout<\/code>.<\/del>2<\/a><\/sup> You see, the use of the ><\/code> to redirect instructs the shell to replace the file following it with the output of the previous command – truncating it entirely. If you did<\/em> try to sort your file that way, you’re going to regret it very dearly quite shortly thereafter when it comes time to email that file and you realize that it’s now completely blank<\/strong> and all your work has been lost.<\/p>\n

The “right” way of doing this would be to read from one file and write to another. You could do something like this:<\/p>\n

#randomly select 2048 words from the dictionary\nshuf -n 2048 \/usr\/share\/dict\/words > words.txt\nsort words.txt > sorted.txt\nmv sorted.txt words.txt<\/pre>\n
And that would do what you want. Except it’s a bit of a mouthful, and it gets a lot harder when it’s not just a one-off command that needs to read and write from\/to the same file but rather part of a larger workflow or bash script. Enter rewrite<\/code>.<\/p>\n
rewrite<\/code> is a dead-simple no-brainer of a command line utility that simply buffers content from stdin<\/code> and only when the upstream command or commands have finished does it write to the output file. Sorting a file in-place becomes as simple as<\/p>\n
sort words.txt | rewrite words.txt<\/pre>\n
And that’s it. You don’t have to worry about creating or deleting temporary files, you don’t need to generate random file names if you’re batch processing a thousand files as part of a script, etc. rewrite<\/code> takes care of the entire thing for you, and it couldn’t be easier to use. And for those that care about such things, rewrite<\/code> is written in rust for safety and reliability combined with performance and out-of-the-box cross-platform support.<\/p>\n
Installing rewrite<\/code> is as as simple as<\/p>\n
cargo install rewrite<\/pre>\n
(Assuming you have rust’s cargo<\/code> utility installed.) If you’re on Windows or Mac, pre-built binaries are also provided below. Of course, rewrite<\/code> is fully open-source (MIT licensed, no less) and available on GitHub<\/a> for your forking pleasure.<\/p>\n