Monthly Archives: January 2012

Introducing ‘usbraid’ – for efficient USB RAID management.

Those of you who know me well also know that I’ve been doing geeky stuff for a long time, so it shouldn’t come as a surprise to learn that (while I wasn’t the first person to do so) I have been using USB RAID arrays for a few years. Unlike the linked articles however, I have generally had a practical reason for using one.

The first practical USB RAID array I ran was in RAID0 – attached with tape to the back of my Asus EeePC 701 laptop screen. The USB RAID storage was actually considerably faster than the 4Gb of internal non-upgradable flash the netbook came with.

Currently however, I use a USB RAID array to store my most confidential files on – things like my BitCoin wallet, password manager databases, important documents and the like. Why would I do that? Security and convenience, primarily. I wanted a backup solution with redundancy in case one of the drives failed, so that rules out my spare laptops which all only house a single HDD (without reaching for a soldering iron, anyway). I also don’t want to store such confidential information on my home server which is running 24×7 and always connected to the Internet – it exposes this data to unnecessary risk. No, ideally the storage device to be used for these specific backups should be only powered up when the data is actively being used.

Most USB HDDs you can buy would fail to meet the ‘redundancy’ requirement, but there are devices such as the Western Digital My Book RAID1 enclosures and the like. Unfortunately these generally house 3.5″ HDDs – overkill for the few small files I need to store securely. There are other non-apparent problems with these too:

  • The sheer bulk and weight of some of those solutions would make them very susceptible to damage if accidentally dropped.
  • They tend to rely on proprietary software and/or HDD controller chipsets which may not be easy to replace if they fail.
  • Generally, such devices are not terribly cheap.
  • In my experience, putting much trust in consumer-grade external hardware devices is just asking for trouble.

So there you have it – a very practical reason why I require a USB RAID array. Running five 1Gb sticks in RAID6, permanently duck-taped to a cheap USB hub solves all of the above problems, is silent, tougher, smaller, lighter, cheaper, more easy to replace (can just buy any other USB hub off the shelf – or in a pinch not even use a hub if a desktop has enough USB ports), and would require at least 3 drives (more than half the array in my case) to fail before losing data. As far as the hardware part of the solution goes, it’s perfect!

Of course, the software side of the story is a little more tedious. I actually run LVM to manage my partitions on top of my RAID device, so having to manually start a RAID array by specifying the device nodes of each USB key, setting the LVM volume group to ‘available’, creating mount points and then mounting each filesystem I’m interested in each time I want to use my array is actually quite a lot of work. After a bit of practice you can go from connecting the device to having the filesystems mounted in about a minute, but even that is far too long IMO – especially when you consider that you also need to do a number of steps to reverse all of this when you’re finished with the filesystems later.

A few months ago, I bit the bullet and spent a few hours writing my own solution which I now license to all (under the GPLv3): usbraid. I’ve spent most of this morning updating it to be less specific to my system and adding the included documentation, so hopefully it’s useful to somebody who might be in a similar situation. You need to know a bit about mdadm and LVM2 if you are considering making your own USB RAID setup and using this tool, but hopefully it’s not too difficult. Once setup as described in the included README file, you should just be able to simply run:

$ sudo usbraid -m
$ sudo usbraid -u

to mount and unmount your USB RAID filesystems.

Giving up fglrx in Debian Wheezy

The title says it all. A recent update has once again killed fglrx direct rendering from working with Xorg, so I’ve decided to just switch over to the free software Gallium driver entirely. This means no Amnesia, but I’ve since finished that game. It probably goes without saying that CrossFire won’t work now too, so… I would like to say that three of my GPUs are just doing nothing, but there are still power management issues with the radeon driver so the fans are sending my wife and I deaf while my cards cook at around 80-90 degrees, and it heats up my apartment noticeably – an annoyance since we’re heading towards the middle of summer here. It also means no OpenCL support since the AMD APP SDK depends on fglrx, although fortunately I haven’t been using that lately either.

The uninstallation of fglrx did not go smoothly. There have been times since I first performed my current desktop OS install where I manually ran the installer downloaded from AMD’s website, which spread files all over the place. These had to be cleaned up. The following two links were the most useful I came across which deal with this problem:

However, the final issue I had was documented on neither of those. The AMD installer created a file on my system in /etc/profile.d/ati-fglrx.sh which set an environment variable which caused direct rendering fail ($LIBGL_DRIVERS_PATH IIRC). Removing that file, logging out and in again got everything back to normal… well, “normal” as described above. :/

I’m still keeping fglrx on my laptop though (which I haven’t updated in a while)… for now. I don’t want my laptop run into the same power management issues leading up to Linux.conf.au 2012.

Here’s something I’ll be taking away from this experience. Proprietary software might sometimes be better than free software, but generally there can be no expectation of it becoming any better in the future than it is today. In the future it may become incompatible, may add new restrictions upon you, may not support new formats, may force you to upgrade (sometimes at cost) to continue functioning properly, etc. The issue I have experienced in this post was the former. With free software however, I can generally expect that the software I have today will never become worse over time – that is, it only gets better. Even in cases where ‘better’ is debatable (eg. GNOME 3), it can be (and often is) forked by anyone. That’s one of the reasons I love it.

To show my support of free software and software freedom, I have finally done something I feel guilty for not doing a long time ago – and became an associate member of the Free Software Foundation.
[FSF Associate Member]