Imagine the following scenario: one day, you’re using your computer as you would on any other day. Then, on impulse, you happen to take a look inside the case. Instead of some neatly (or not-so-neatly) organised disk drives, there are just a bunch of photographs. Illegible colourful scrawls of pixelated dust fill every inch of them, each a kaleidoscope, magnificent as much in their beauty as in their complexity. Instinctively you watch carefully as you create a new file and save it to the desktop, pleased to discover that as you expected, the spattered colours shift and change along with the flow of new data. It is this world that I wish to bring to you today.
Okay, so it’s not quite so deliriously grand as all that. But I did have the idea, a few weeks after watching a particularly engaging talk by tom7, to dive into the world of Network Block Device, or NBD for short. NBD is a network protocol to control block devices remotely. Block devices are a kind of abstraction provided by Unix-like operating systems, and represent some access to hardware. So, NBD is a protocol that lets you remotely control hardware, like disk drives, over a network. Well, that was the original intent, anyway.
Turns out, with just a little bit of computer magic, you can use the NBD protocol to provide just about anything as a block device. You could plug a potato into a Raspberry Pi, and with enough effort, make the potato accessible as a block device. It wouldn’t be good for very much - you could perhaps read the voltage of the potato and use it as an input device for something, like a second battery meter on you laptop that shows you how dead your potato is. Or maybe you could make it so that you could use a BMP image file as a virtual hard drive, so that you can watch it fill with pretty colours! And as I said, all it takes is just a tiiiiny bit of computer magic.
… did you know that the NBD protocol implementation paper is over 2000 lines long?
computer magic made simple (nbdkit)
Well, I’m not going to read through all of that and write my own implementation from scratch, tempting as it is. Why do such a thing when the greatest strength of humanity itself is co-operation, to build upon the foundations we lay for those who come after?
There’s a useful utility called
nbdkit. You can check the full description out over on its GitLab page, but to summarize, it provides a simple API layer that you can use to directly interact with the NBD protocol, by writing plugins in a variety of languages. All sorts of languages are supported, from C, to Python, to Rust. You can inform
nbdkit which operations your plugin supports (e.g, reading data, writing data, if it can be multithreaded), and provided everything has been set up according to the documentation, you can use an NBD server of your choosing to host
nbdkit as a device with your plugin loaded. Many Linux distributions have an
nbd kernel module to do some of the heavy lifting here, and you can use
nbd-client to connect your computer to the NBD service once
nbdkit is running, either locally or over the network.
Whew! Now that the basics are out of the way, let’s talk about
bmpdisk is a plugin that I wrote for
nbdkit that allows you to use a .BMP format image file as a block device. It can make a blank one for you, or you can provide an existing file. Once it’s hosted as a block device, you can do whatever you like with it - you can create a partition table on it and add a filesystem, just like any disk drive. You can store things on it (even other .BMPs, and thus, other drives), you can read things from it, you can copy and paste the image file to share it with someone else (who can then host it as a block device and mount the disk on their own OS). But, best of all, you can look at it. You can see your data filling it up, a colourful cascade filling up the blank spaces, giving way and disappearing when you reformat the disk.
For example, here’s a literal disk image: It contains a copy of the
bmpdisk source code and a compiled plugin for 64-bit Linux. It has a capacity of ~12 MiB, and is in ext4 format.
bmpdisk loaded, you can use it just like any other drive, in the ways you’d expect.
[autumn@melonarch doctormelon]$ cd /mnt/bmpdisk [autumn@melonarch bmpdisk]$ ls -la total 3689 drwxr-xr-x 3 autumn root 1024 Jun 28 22:02 . drwxr-xr-x 1 root root 20 Jun 17 16:01 .. -rwxr-xr-x 1 autumn autumn 3773880 Jun 18 15:15 libbmpdisk.so -rw-r--r-- 1 autumn autumn 592 Jun 17 16:11 README.md drwxr-xr-x 2 autumn autumn 1024 Jun 17 13:35 src [autumn@melonarch bmpdisk]$ echo "Hello there!" > hi.txt [autumn@melonarch bmpdisk]$ ls -la total 3690 drwxr-xr-x 3 autumn root 1024 Jun 28 22:09 . drwxr-xr-x 1 root root 20 Jun 17 16:01 .. -rw-r--r-- 1 autumn autumn 13 Jun 28 22:09 hi.txt -rwxr-xr-x 1 autumn autumn 3773880 Jun 18 15:15 libbmpdisk.so -rw-r--r-- 1 autumn autumn 592 Jun 17 16:11 README.md drwxr-xr-x 2 autumn autumn 1024 Jun 17 13:35 src [autumn@melonarch bmpdisk]$ cat hi.txt Hello there! [autumn@melonarch bmpdisk]$ _
how bmpdisk works
It’s surprisingly simple; BMP files are pretty much just straight up uncompressed blobs of data, with a couple of headers at the start to tell anything reading the file what its dimensions are and how many colours it supports, etc. This makes them trivial to connect to
nbdkit, as the API expects to be able to read and write buffers of plain old bytes. By writing the bytes into the R, G, and B channels of the BMP image data, which are each 8-bit values, and keeping the BMP file headers intact, the solution practically writes itself.
In my case, I simplified this even further; I used Rust, and used the
nbdkit crates and basically just kind of squished them together. Bam, instant image-masquerading-as-a-block-device!
You can see the (admittedly somewhat messy) source over at my GitHub page, along with short instructions on how to use
so, uh, what was the point of this, exactly?
For fun, mainly! But I just wanted to see how NBD worked. Now that I know how simple it is, I’m constantly thinking up weird things to do with it. There’s fundamentally no reason why I couldn’t make my own version of a tape drive, or why I couldn’t use something like a geiger counter to use background radiation as a source of random noise for a dice-rolling application. And I hope that maybe this post will inspire others to do Weird Things With Computer…