This is Part 4 of a series of blog posts on building a Home NAS & Server using Linux. For the full index and other details, please see the Index
EDIT – April 2013: Since originally writing this segment, ZFS on Linux has reached it’s first non-RC version – 0.6.1. With that release, a Debian Wheezy repositories are available that handle the install process. If upgrading from these instructions, the previous packages must be purged fully before installing from the repo (dpkg –purge zfs zfs-devel zfs-dracut zfs-modules zfs-modules-devel zfs-test spl spl-modules spl-modules-devel). Up-to-date Debian-specific instructions are at http://zfsonlinux.org/debian.html)
Fortunately, the good folks involved with ZFS on Linux have got both useful instructions on compiling your own packages along with a lovely Ubuntu PPA repository for both stable and daily releases. Conveniently the Lucid PPA works perfectly with Debian Squeeze, although I’ve settled on Wheezy for the HP ProLiant Microserver, so for that I’m building from source.
Compiling and installing ZFS
In testing (within a virtual environment) I followed the deb building advice for my Squeeze machine, and found the only thing I needed to do was run update-rc.d zfs defaults after install to ensure that the pool is automounted on boot. They were just as painless for Wheezy, and needed no trickery to automount the zpool on boot.
Using the PPA on Ubuntu I had no such concerns.
The PPA pages detail all that is really needed to add the repository to Ubuntu, and installation is as simple as aptitude install ubuntu-zfs but there were a couple of different steps needed for the Debian Squeeze system. As Wheezy shouldn’t be too far off become the new stable, I won’t spend any time on those steps – Google is your friend.
Once it’s finished installing and the kernel modules are compiled you should be able to run zfs and zpool commands.
I won’t be using hot swap / external loading drives on either of these builds, at least not to begin with, so having a relatively painless way to identify which disk is which is relatively important to me. To that end, I decided to use the vdev_id.conf file to allow me to assign human-readable names to all the installed disks. As detailed on the project page, this technique is really more useful in much larger environments to ensure you can quickly and easily identify separate controllers for greater redundancy in your pools. In my case it’s more so I can quickly and easily identify which disk is broken or misbehaving when that time comes. A quick cross-reference of the disk serial numbers before I inserted them into the bays with the device assignments once booted helped me confirm the correct PCI addresses. The naming I decided on was:
~ cat /etc/zfs/vdev_id.conf
# Custom by-path mapping for large JBOD configurations
# Bay numbering is from left-to-right (internal bays)
#<ID> <by-path name>
alias Bay-0 pci-0000:00:11.0-scsi-0:0:0:0
alias Bay-1 pci-0000:00:11.0-scsi-1:0:0:0
alias Bay-2 pci-0000:00:11.0-scsi-2:0:0:0
alias Bay-3 pci-0000:00:11.0-scsi-3:0:0:0
After doing the above, running udevadm trigger will cause the file to be read and the symlinks to the correct devices will appear in /dev/disk/by-vdev/. The benefits should become clearer once the pool has been created.
For reference, Bay-0 is my root drive – I’ve not opted for ZFS on root, nor any real resilience due to space constraints (I may work around this in the future). For the rackmount build – as mentioned – I’ll be looking at small 2.5″ drives mirrored using mdadm for the root files.
At this point, all that’s left to do is create the pool. As mentioned, Bay-0 is my root disk, so the disks I’ll be looking at using are Bay-1, Bay-2, and Bay-3 – configured as raidz. To confirm things, I ran zpool create with the -n flag to verify what would be done. Once happy, the -n can be removed and the command run:
zpool create -f -o ashift=12 -O atime=off -m /pools/tank tank raidz Bay-1 Bay-2 Bay-3
A couple of notes on this:
- I used -f because I received the following error without it:
invalid vdev specification
use '-f' to override the following errors:
/dev/disk/zpool/Bay-1 does not contain an EFI label but it may contain partition
information in the MBR.
I’ve seen this error in my testing as well, and found a few threads on it, but nothing convincing. I confirmed with fdisk that the disks do not, in fact, contain any partition tables, and opted for -f to force past it.
- ashift=12 is per recommendation (drives with 4k blocksizes, per http://zfsonlinux.org/faq.html#HowDoesZFSonLinuxHandlesAdvacedFormatDrives
- atime=off as a default setting for the pool filesystems as I can’t see any real reason to enable it across them all.
- -m allows me to specify a mountpoint – whilst I’ll only have one pool, I figured specifying a specific pools directory would be a good habit to form if I ever want to import other pools in the future. The directory must exist before this command is run.
Once created, things can be confirmed with zpool status:
~ zpool status
scan: none requested
NAME STATE READ WRITE CKSUM
tank ONLINE 0 0 0
raidz1-0 ONLINE 0 0 0
Bay-1 ONLINE 0 0 0
Bay-2 ONLINE 0 0 0
Bay-3 ONLINE 0 0 0
At this point I’m ready to do pretty much what I want and can verify my pool is online and mounted with df.