System Saviour

There’s an issue I’ve been wanting to sort out for over a year, but it’s one of those niggling annoyances that’s just hard enough to find an elegant solution for that encourages me to keep putting it off. Well no more! I’ve finally got this problem licked.

So to clarify my situation, I have an external USB HDD for my laptop with a bunch of large games on it and the like, which won’t fit on my laptop internal SSDs. I run Xfce, and I have the option under Removable Drives and Media labeled Mount removable drives when hot-plugged ticked, and this works as the developers intended.

Mount removable drives when hot-plugged.

The problem is that I don’t lug this largish laptop around too much with me, so the USB HDD remains connected most of the time. When I power up I can see the device under Thunar and Nautilus, however it is not mounted. I need to manually click on the drive for that first. The reason being is that the device was not hot-plugged after Xfce was loaded – it was already connected when I logged in. Having to open a file manager and click the drive before I can use it after each reboot is, well… not ideal.

I’m aware one option could probably be to just add an entry to my /etc/fstab file to automount this if the device exists on boot, but I don’t like that for two reasons. Firstly, I might want to use a different HDD (or multiple HDDs) in the future. I don’t want to have to edit my /etc/fstab file for every HDD, SD card, USB stick or whatever. Basically, if a device is already inserted, and I’ve given it a filesystem label (so the filesystem is able to be mounted with a fixed mountpoint name under /media/ as per usual hot-plug USB mounting), I want it automatically mounted by the time I’m logged in. In the event a device does not have a label, I don’t want it automatically mounted since it may not have an obvious name or even a fixed mountpoint automatically created for any kind of automount to be meaningful. Since I don’t know what devices I’ll connect in the future, simply adding /etc/fstab entries won’t suffice.

Secondly, I want filesystems that do not have permissions (or permission support under GNU/Linux) to be mounted as the user currently logged in. If my spouse (for example) logs into my laptop with her own account and wants to plug in an NTFS or FAT32 formatted device, she should be able to do so without permission trouble. If /etc/fstab had mount permissions set to allow only my user account access, it would present problems. Conversely if she did have permission, it would mean either /etc/fstab also allowed my login access to the device as well (via group permissions) – probably not ideal for privacy, or permissions were so relaxed that any user on the system could access the device (eg. a 0000 umask) – a significant security risk!

After a bit of searching around the web, I decided the udisks command in the udisks Debian package was the way to go. As this package is a dependency for the xfce4-power-manager package, as an Xfce user I already found this to be installed. I also looked into pmount (which did not create entries under /media/ automatically using the device filesystem label), and usbmount which is no longer maintained, and (according to the Debian wiki page) should not be used if you want a desktop icon, and also apparently has the same issue pmount has (ignoring filesystem labels for use as mountpoint names). I wanted the behaviour of manually clicking the drive icon in the file manager mimicked as closely as possible, and udisks seems to do just that.

Unfortunately, udisks does not have some kind of “mount all” option. It does tell you which devices are connected via USB (via the --dump argument) but that did not look so easy to parse (and I wouldn’t be surprised if this output formatting changed when upgrading or replacing distributions that might include a new udisks version). Instead, I noticed looking under /dev/disk/by-path/ that USB devices had -usb- as part of the symlink name – be it the raw block device or a partition. This looked good enough to me, so I used that.

$ find /dev/disk/by-path -name '*usb*' -exec readlink -f {} \;
/dev/sdc1
/dev/sdc
$ 

I typically partition all my devices, including USB sticks. Still, I wanted a solution that would detect the correct device to mount regardless. I thought about using file -s <devices> but that requires either raw block device access (which seems too risky) or having the ability to automatically run the file command via sudo without a password. Running file on untrusted code is in some ways even more risky, given this can trigger code execution, as I recall. I would also prefer to have a self-contained solution – and by that I mean no changes outside of my home directory, and not something that changes my setup globally. I should be able to understand everything going on just by having common knowledge of how a distribution is put together and looking in the one spot.

In the end, I determined blkid would be helpful. It does not require root privileges, should exist on pretty much any system (as it’s included in the util-linux package), and can easily identify block devices with a filesystem label – which is all I’m actually interested in anyway. So here’s the solution we end up with:

# Mount all USB block devices that have a filesystem label.

for device in $(find /dev/disk/by-path -name '*usb*' -exec readlink -f {} \;)
do
    if [ -b "${device}" ] && blkid "${device}" | grep -q LABEL
    then
        if ! mount | grep -q "^${device} on "
        then
            udisks --mount "${device}"
        fi
    fi
done

We identify all USB-attached block devices, loop over them checking for devices with a LABEL entry, verify they are not already mounted (in case this code is ever executed multiple times so as to avoid mount warnings being printed), and finally if everything checks out the device in question is mounted. Beautiful.

Where do I stick this? I could put it in a script under ~/bin/ and point to it under the Xfce Session and Startup -> Application Autostart section. However, I don’t always have Xfce running. Sometimes I log in directly from agetty on a virtual console eg. when I’m running the Nvidia driver installer, which fails when Xorg is running. If I have the Nvidia driver downloaded to my external hard drive, it would be convenient to have that device automatically mounted during login even without Xfce.

When you login through a display manager such as LightDM, /etc/X11/Xsession is executed. On Debian systems at least, this in turn calls all scripts placed under /etc/X11/Xsession.d/, which are often dropped there by various packages. eg. gnupg-agent, xbindkeys, etc. One of the script is called 40×11-common_xsessionrc (included as part of the standard x11-common package) and it sources ${HOME}/.xsessionrc. Since ~/.xsessionrc is sourced after Xorg has already started and logged us in (but have not quite yet ran x-session-manager – a symlink to xfce4-session managed via update-alternatives in my case), it gives us the opportunity to do all kinds of neat things. I already use it to detect external displays I have connected (via xrandr) and setup the monitor configurations according to a series of predefined profiles. eg. If there is one HDMI LCD with 1920×1080 as the max res, assume the LCD is to the right of my laptop and adjust my Xorg screen layout accordingly. I also use it to launch xmodmap, which is useful for disabling my Caps Lock key (although as the name implies it only works with X).

But ~/xsessionrc won’t be sourced if logging in from agetty. Instead, /etc/profile, followed by ~/.bash_profile, ~/.bash_login, or ~/.profile will be sourced (and of the three I only use ~/.profile). Likewise, ~/.profile won’t be sourced from a display manager (or at least it shouldn’t be – I have a vague recollection of GDM doing this, or having done it in the past). Anyway, let’s fix that. In ~/.xsessionrc we’ve now got:

# Send expanded command output to ~/.xsession-errors for debugging.
set -x

# source profile data
for file in "/etc/profile" "${HOME}/.profile"
do
    if [ -f "${file}" ]
    then
        . "${file}"
    fi
done
unset file

Since this file is sourced, it does not require executable permissions.

So now we can just stick our USB mount code in ~/.profile, right? Well yes, but I prefer something more elegant. Towards the end of my ~/.profile file, I have the following:

if [ -d "${HOME}/.profile.d" ]
then
    for script in ~/.profile.d/*.sh
    do
        if [ -f "${script}" ]
        then
            . "${script}"
        fi
    done
fi
unset script

I then have a directory called ~/.profile.d and I put various files under it that I want executed when I login, regardless if logging in from a display manager or agetty. Any time I have environment variables required for specific functionality or a specific application, I add them to a separate file here. For example, I have dh_make.sh which I use to export the DEBEMAIL environment variable, and wine.sh which I used to export debugging environment variables, driver tweaks (also applied through environment variables), and other things related to Wine. For the purposes of USB automount at login functionality, I created the file usb_mass_mount.sh and put the code there.

And that’s all there is to it (and in fact slightly more than is strictly necessary). No sudo privileges required, no tweaks to udev scripts, fstab, or anything specific to the current session-manager – or even anything dependent on Xorg even running. If there were a more elegant way to determine which devices are USB attached, without udev changes and without complex parsing of udisks --dump or the contents of /sys/block, it would be darn near perfect.

Anyway, that was a very long-winded explanation for something which turned out to be relatively simple. I think I probably got way too excited over this. Anyway, I hope somebody else finds this useful.

It’s been a long time since I have seen any new hardware truly excite me. The last time I can recall such an event was perhaps the release of largish affordable SSDs, or perhaps the release of high-resolution displays, which sadly are only now starting to become readily available to those of us outside of Apple’s ecosystem. Unfortunately most hardware improvements are so incremental that it’s hard to feel truly excited about something.

That all changed today, with the release of the MSI GS30 Shadow.

I’m currently the owner of an Asus G55VW Republic of Gamers i7 laptop. It’s a good mid-range “laptop”, however it’s actually a desktop replacement due to the sheer size and weight of it (and I did indeed use it to replace a mid-tower desktop – the first time I’ve ever used a laptop as my primary computer). I’ve maxed out the RAM to 32Gb, and added in a 1Tb m-SATA SSD and replaced the 1Tb mechanical SATA drive with another SSD. It also has a GTX660M which is powerful enough to run any game on the market, and thankfully doesn’t require me to deal with software to switch between the Intel integrated graphics and the Nvidia GPU – Intel graphics are not available. Unfortunately it is no longer powerful enough to run everything in the highest detail settings on recent titles as evidenced during a recent play-through of Far Cry 4. A graphics card upgrade may be in order in the near future, although that’s usually not possible on a laptop – a replacement is generally necessary. I usually use the laptop propped up on a stand with an external keyboard, HDMI-connected LCD, mouse and external Creative X-Fi 5.1 sound card.

In addition to this, I have an old Sony Vaio 11.3″ laptop with a measly AMD E-350 APU and 1366×768 resolution display. This is the laptop I use as an actual laptop – I take it everywhere with me since it’s quite light and is tough enough to take a few drops or knocks while in my bike pannier on my commute to work. Since it has some work files on it, it uses full disk encryption via LUKS. This is painfully slow on the E-350 (even with an SSD) as the APU lacks an Advanced Encryption Standard Instruction Set implementation and is a very underpowered processor in general. However it has proven tolerable for light workloads.

Interestingly the E-350 has a relatively high powered graphics processing capability. Many games such as Killing Floor actually run slower when game-play settings are set to the lowest graphics configuration due to the false assumption by the game developers that a GPU would be the bottleneck, as this option transfers more work from a GPU and onto a CPU (at the expense of graphical quality). Boosting the graphics quality up a bit shifts more work to the graphics component of the APU and results in a slight performance improvement! Ultimately, a game such as Killing Floor still runs too poorly to be playable as more enemies appear in later waves. The E-350 is best suited to simple games like the original Counter-Strike (yes, the one from 1999), and even then probably not at 1920×1080 if using an external monitor.

Since I don’t use the Sony Vaio for gaming (mostly just SSH and a basic instant messaging client), up until recently it’s worked out fine for me. It’s 100% compatible with free software drivers (although I think I may have replaced the wireless/bluetooth module at some point to avoid needing a blob), and has survived a lot of rough treatment over the years, largely due to a single fan being the only moving part. However now my workplace wishes to depend more and more on SaaS applications. I despise the direction things are heading in in this regard – where websites such as Slack, Trello and PassPack have become the norm – perhaps the subject of another post/rant. However this is the company data that is being dealt with – not my own. These proprietary SaaS applications are not being forced onto other people as is the case with traditional proprietary application vendors. It only hurts the companies which choose to use it (well, in addition to employees such as myself), so maybe I can live with it… however my computer certainly can’t. The E-350 just isn’t up to the challenge of these CPU-intensive websites.

Since I’m always on call to deal with any possible infrastructure emergencies which may arrive, I need to always have a computer with me. The E-350 was the perfect small cheap lightweight machine I could slip into my backpack or bike pannier and take with me anywhere. It is relatively fast at booting to an Xfce desktop and opening Terminator (the most time-consuming aspect is getting past the lengthy LUKS passphrase and LightDM login password). However the second I need to open Firefox to log into Slack, the browser loading times can be longer than the entire boot-up time! It’s ridiculous, especially given how PSI+ or Pidgin loads almost instantly on the same machine and provides the same functionality only with a different interface and open standards.

So now I’m in the market for a new computer. I can’t lug around my Asus G55VM as that’s too big to even fit in my bike pannier and would make my back sore if I had to carry it around in a backpack all the time. But if I’m going to get a new laptop, I’m going to want something fast, light, with a high resolution screen. It might be asking a bit much, but I was hoping to be able to use this as an opportunity to replace the G55VM as well if I could find something portable with a graphics card more powerful than an Nvidia GTX 660M (eg. perhaps a GTX 860M). This might be possible since I’d happily forego the Blu-Ray drive – I have external USB Blu-Ray drives which are faster and don’t have the firmware issues playing DVDs which the Matshita UJ160 drive (built into the Asus laptop) has.

You might be starting to get the impression I’m quite fussy with choosing a laptop, and you would be correct. In addition to the above, I also require a 1Gb ethernet port, upgradable RAM and SSD, EFI which allows manual upload of custom signing keys for Secure Boot (which I imagine means restricting my options to the three big-name Taiwanese computer manufacturers, since those are the companies that tend to market more towards people who know what they are doing and expose all possible options), a HDMI port for an external monitor (as I’m yet to see an LCD in person that uses DisplayPort), and a dedicated 3.5mm mic-in port for use with my Sennheiser PC 360 G4ME headset when I don’t have my external sound card with me.

Ideally I would also like I/O MMU virtualization (AMD-Vi or Intel VT-d) support, so I can use the laptop as a new home server when it is time to retire it from laptop use (I use Xen and would make use of PCIe pass-through to guest for the NIC in a dedicated firewall VM), a backlit keyboard, no USB 2.0 ports (everything should be USB 3.0 these days), support for a second SSD, and a matte screen. I also don’t want an Asus Transformer laptop, since I’m not interested in tablet or touch-screen functionality (which are usually glossy anyway), and I’m not confident the hinges (with the detachable keyboard) would be able to withstand much punishment.

Now, I’ve been looking for something that fits all of the above as best as I can, but everything I have found requires compromising somewhere. Maybe something comes close, but doesn’t have an Ethernet port, or is too heavy, or doesn’t have VT-d or AES CPU support. Maybe it only has a GT 840M GPU, which wouldn’t really be an upgrade over a GTX 660M. It’s almost impossible to find something in a small form factor with support for two SSD storage devices. Everything I have seen has failed to impress, so I’ve been procrastinating on making a decision about what to do.

Well, today I was browsing the PC Case Gear website, and noticed something strange in the gaming laptop section which I had not seen before. That laptop is the MSI GS30 Shadow, and it amazingly ticks all the boxes with ease due to an impressive feature. A feature so simple, I can’t believe it hasn’t been done before. In addition to being a kick-ass Ultrabook, albeit one without a dedicated GPU, it supports and includes an external enclosure which can be used to connect a real PCIe 16x graphics card to it of your choice! This external enclosure doubles as speakers (which I probably won’t use since I already have a nice 5.1 surround sound setup), a 4-port USB 3.0 hub (so I can leave my keyboard, mouse, external sound card, external optical drives, etc. permanently connected) and includes a 3.5″ SATA expansion port for an SSD to store all those games on that require a powerful GPU. The enclosure is actually a docking station which the laptop sits on top of, so I could use this to replace my existing laptop stand as well.

I’m absolutely stoked. This is almost exactly what I’ve been looking for. I’ve wanted an Ultrabook-like machine which has upgradable RAM and storage (this provides two SSDs connected via two M.2 SATA connectors), and yet is lightweight and portable and has a high-end quad core processor with IO/MMU and AES CPU extensions. But the main feature is easily having the ability to finally upgrade the graphics card in a laptop. I don’t need portable GPU power since I mainly use my laptop away from home for work purposes, but I do want it at home for personal use. And as if all this wasn’t enough, the external GPU enclosure includes an additional 1Gb ethernet NIC – perfect for re-purposing this machine as my home server in the distant future. I also wonder if I might one day find alternate uses for that PCIe slot, such as a SAS controller. It’s exciting to think about the possibilities.

Although this laptop comes close, it isn’t perfect. It seems the laptop will only support up to 16Gb of RAM. I’m used to 32Gb (mostly for when I’m experimenting with various virtual machines or using it as cache for slower external mechanical/optical drives), but I’m confident 16Gb will still be enough to not make the decrease too noticeable. I would have liked to see a QHD display resolution option, although omitting it is probably reasonable given it would only be used with the integrated Iris Pro 5200 graphics. I also have the usual complaints about Windows being forcefully bundled (and a version of Windows that I especially hate), but given how fussy I am with hardware specifications (and my dabbles with Wine which sometimes uses licensed Microsoft components), I’m not as upset about it here as other people would be. It would also be nice if the dock supported more than just one 3.5″ SSD and PCIe slot. I also wonder if other future laptops will be released by MSI that will be compatible with this dock, although that seems doubtful given the dock size doesn’t look big enough to support larger laptops.

Upon reading reviews of the GS30 Shadow, I saw references to another recently-released device with similar functionality compatible with some Alienware series of laptops – Alienware’s Graphics Amplifier (which I’ll hereafter refer to as the AGA). This is something purchased separately to the Alienware laptops it is compatible with, and has the advantage of being able to pass the external GPU graphics onto the laptop display – a feature I can’t imagine myself using – but my gut feeling is that this might complicate GNU/Linux compatibility. The Alienware solution also has some significant drawbacks:

• The AGA connects to the laptop through a cable instead of via a dock. Although the AGA includes a PCIe 16x slot, the cable limits the bus bandwidth to PCIe 4x speeds, which will surely hinder the ability to upgrade down the road as performance will be compromised on faster graphics cards. I also feel Dell is misleading people about this as the limitation is not mentioned anywhere I could find on the website, although I can’t say I’m surprised.
• It doesn’t include a SATA controller or mounting brackets. This is a great feature of the MSI solution in my opinion, and something I’d like to make use of.
• It doesn’t look like something I could sit my laptop on reliably. That means I’ll have to find more desk space, and I’m not sure I would be able to comfortably find the room. I also can’t imagine cable length would be great.
• Alienware is nowadays owned by Dell. Dell and I have a bit of history, and I’d like to avoid that company wherever possible.
• Perhaps most importantly, there is no comparable Dell laptop compatible with the AGA – no 4th Generation i7 CPU with VT-d support in a 13″ machine, and the 13″ laptops on offer are almost twice as heavy as the MSI.

So there we go, the MSI GS30 Shadow is a winner. It isn’t actually released in Australia for another two days, but I’m convinced that the GS30 will be my next laptop sometime soon. I’ve never owned a MSI laptop before, but now I’m certainly looking forward to it.