# OLinuXIno Arch with btrfs root on external HDD

This post explains how to install Arch Linux on OLinuXIno LIME 2, with root on external btrfs HDD.

Here comes a long one… If you are not interested in why I’m doing what I’m doing, only in how, read only Installing the system section.

For a year I’ve been using rPI as my home server. It has performed adequately, but for various reasons I’ve decided to do a HW upgrade and to choose a little bit different approach for setup.

Price, power consumption, size and noise were (and still are) the key factors that need to be considered when choosing a new hardware.

My choise was A20-OLinuXIno-LIME2 . It is very cheap, has dual-core 1GHz armv7 CPU, 1GB RAM and onboard SATA. As a neat bonus, it is completely open hardware platform. Onboard SATA was the key feature that convinced me to use this board. On my previous rPI setup root has also been on the external HDD, connected using SATA <-> USB converter. This was a huge bottlenect, since Ethernet on rPI is also connected to the same USB hub.

The previous server setup was a single Arch system on which all my services were running. This was OK for most of them, but Gitlab… that thing broke every time there was a Ruby update. With no spare time on my hands I’ve simply stopped updating the server.

This time I’m going to address this issue. The base system will be still Arch Linux. But it will only be a minimal, bare system, host to several LXC containers. In those containerized isolated environments services themselves will run. This approach remedies the problem with problematic software dependent on a particular version of many things. And if I were to use btrfs, root of each Container can be a btrfs subvolume. This has some neat implications: Container cloning with Copy-On-Write support, snapshotting before messing with the container (so that changes can be reverted rapidly if something goes wrong)…

This is the reason I’m installing Arch Linux with btrfs root on external HDD. This post covers the installation of base system.

# Arch installation

## Choosing the right approach

There are several possible ways to achieve this. Arch on OLinuXIno uses the u-boot bootloader. It can load kernel image from serveral filesystems, but btrfs is not one of them. That automatically means a separate boot partition will be required.

How about root? Arch Linux ARM is installed from prepared .tar.gz file containing the whole root filesystem. But, attempting to extract it to the btrfs partition causes errors like

./usr/foo/bar bsdtar: Failed to set file flags


OK, so extract it to an ext4 partition and then convert it to the btrfs file system? That would work, but btrfs wiki warns about this: Warning: As of 4.0 kernels this feature is not much used or well tested anymore, and there have been some reports that the conversion doesn’t work reliably. Feel free to try it out, but make sure you have backups . Uh, that’s not something I would rely on when it comes to a root of a server.

I’ve chosen to install Arch Linux ARM from scratch, in the same way Arch Linux is installed on desktops. That means we will prepare an installation medium on a SD card, boot it and install arch to HDD from it, therefore avoid all those problems.

## Installing the system

This process is heavily based upon these instructions, combined and changed where necessary:

### Preparing the installation medium

Zero out the beginning of the SD card:

# dd if=/dev/zero of=/dev/sdX bs=1M count=8


Now we will partition the card in an interesting way. I suppose you know how to use fdisk, if not, read the manual and some tutorials. We will create two partitions, one for the whole installation system, and one for the new /boot partition. Start fdisk and create primary partition number 1 starting on sector 104448. Accept the default end. Now create primary partition number 2 starting on sector 2048 and with end on 104447. This will create a 50M partition.

Why have we done this? U-boot looks at the first partition and looks for the (among other things) /boot/zImage. For now, we want our installation partition to be the first, so that our installation medium will boot. Later, when the system is installed, we will swap those partitions and our new system will boot.

After fdisk-ing the card, it should look like this:

Device         Boot  Start      End  Sectors  Size Id Type
/dev/sdX1           104448 15353855 15249408  7.3G 83 Linux
/dev/sdX2             2048   104447   102400   50M 83 Linux


Format and mount the installation partition:

# mkfs.ext4 /dev/sdX1
# mount /dev/sdX1 /mnt/tmp


# wget http://archlinuxarm.org/os/ArchLinuxARM-armv7-latest.tar.gz
# bsdtar -xpf ArchLinuxARM-armv7-latest.tar.gz -C /mnt/tmp
# sync


# wget http://archlinuxarm.org/os/sunxi/boot/a20-olinuxino-lime2/u-boot-sunxi-with-spl.bin
# dd if=u-boot-sunxi-with-spl.bin of=/dev/sdX bs=1024 seek=8
# wget http://archlinuxarm.org/os/sunxi/boot/a20-olinuxino-lime2/boot.scr -O /mnt/tmp/boot/boot.scr
# umount /mnt/tmp
# sync


Now insert the card to your OLinuXIno (with connected HDD you will install Arch on) and boot. When the the boot finishes log in to the installation system (root pw is root (change it!)). Install those packages:

# pacman -Syu arch-install-scripts btrfs-progs


OK, now we are ready to use this image to install Arch Linux ARM from scratch.

### Installing (for real now, I promise…)

The routine stuff. Partition the HDD using fdisk:

Device     Boot   Start       End   Sectors   Size Id Type
/dev/sda1          2048   4196351   4194304     2G 82 Linux swap / Solaris
/dev/sda2       4196352 976773167 972576816 463.8G 83 Linux


Create boot, root and swap filesystems:

# mkfs.ext2 /dev/mmcblk0p2
# mkfs.btrfs /dev/sda2
# mkswap /dev/sda1


Mount it:

# mount /dev/sda2 /mnt
# mkdir /mnt/boot
# mount /dev/mmcblk0p2 /mnt/boot
# swapon /dev/sda1


Install base system packages:

# pacstrap /mnt base btrfs-progs


Note: The above command won’t install kernel itself (and I was wondering why it would’t boot…). We will install it later.

Now for the basic configuration. Create fstab:

# genfstab -p /mnt >> /mnt/etc/fstab


Now, our /boot partition is mmcblk0p2. We will change it later to mmcblk0p1, so let’s tell fstab that. Change line containing mmcblk0p2 to:

/dev/mmcblk0p1          /boot       ext2        rw,relatime 0 2


Chroot to the newly created system for even more configuration :) :

# arch-chroot /mnt


Set the hostname and timezone:

# echo 'your_hostname' > /etc/hostname
# ln -sf /usr/share/zoneinfo/<Zone>/<Subzone> /etc/localtime


Uncomment your locale in /etc/locale.gen and generate locales, set your language:

# locale-gen
# echo LANG=en_US.UTF-8 > /etc/locale.conf


Install the kernel:

# pacman -S linux-armv7


Set the root password using passwd. This should be enough, but don’t reboot your system yet!

I’m connecting to the OLinuXIno using ssh, therefore network must be up and sshd must be running in the new installation.

# pacman -S openssh
# systemctl enable sshd


My device has static IP, so I’ve written systemd-networkd config file:

[Match]
Name=eth0

[Network]
DNS=your_preferred_dns (maybe 8.8.8.8)

[Route]
Gateway=gateway


and followed Arch systemd-networkd wiki page, all this is explained there.

# systemctl enable systemd-networkd
# systemctl enable systemd-resolved
# rm /etc/resolv.conf
# ln -s /run/systemd/resolve/resolv.conf /etc/resolv.conf


Also, according to the wiki: replace dns with resolve in /etc/nsswitch.conf:

hosts: files resolve myhostname


### Making the system bootable

While still in the Arch chroot install a bootloader:

# pacman -S uboot-a20-olinuxino-lime2


It asks you whether you want to flash a new bootloader. Let it do so.

Also install

# pacman -S uboot-tools


we will need them for the final bootloader configuration.

By default u-boot installed by the above package looks for the kernel and everything else in /boot. Since we have a separate boot partition to u-boot it appears as if the kernel was in /. So let’s modify boot.txt to make it look for it there. Add those lines right after

setenv bootargs console=${console} root=/dev/sda2 rw rootwait  It’s basically a copy of existing script, but with /boot removed. if load${devtype} ${devnum}:${bootpart} ${kernel_addr_r} /zImage; then if load${devtype} ${devnum}:${bootpart} ${fdt_addr_r} /dtbs/${fdtfile}; then
if load ${devtype}${devnum}:${bootpart}${ramdisk_addr_r} /initramfs-linux.img; then
bootz ${kernel_addr_r}${ramdisk_addr_r}:${filesize}${fdt_addr_r};
else
bootz ${kernel_addr_r} -${fdt_addr_r};
fi;
fi;
fi

if load ${devtype}${devnum}:${bootpart} 0x48000000 /uImage; then if load${devtype} ${devnum}:${bootpart} 0x43000000 /script.bin; then
setenv bootm_boot_mode sec;
bootm 0x48000000;
fi;
fi


Next, run mkscr to make a boot script from this file.

# cd /boot
# ./mkscr


Get out of the chroot now (Ctrl + D) and unmount the new system

# umount -R /mnt


Last thing there is to do is to swap those two partitions on the SD card. Use fdisk, print existing partitions, note where they start and end, then erase them. Create new two partitions, but the first will start and end where the second did before.

Before:

Device         Boot  Start      End  Sectors  Size Id Type
/dev/mmcblk0p1      104448 15353855 15249408  7.3G 83 Linux
/dev/mmcblk0p2        2048   104447   102400   50M 83 Linux


After:

Device         Boot  Start      End  Sectors  Size Id Type
/dev/mmcblk0p1        2048   104447   102400   50M 83 Linux
/dev/mmcblk0p2      104448 15353855 15249408  7.3G 83 Linux


Write the new partition table to the card. Note: fdisk will complain

Calling ioctl() to re-read partition table.
Re-reading the partition table failed.: Device or resource busy


That’s OK, since one partition is still mounted as a root of our current system.

That’s it. sync and go ahead and reboot. Enjoy your new system :).

### Last-minute thoughts

A message appeared:

[  408.655073] systemd-journald[128]: Creating journal file /var/log/journal/bd00c0bbc8e94c7197c9d945a65fe627/user-1000.journal on a btrfs file system, and copy-on-write is enabled. This is likely to slow down journal access substantially, please consider turning off the copy-on-write file attribute on the journal directory, using chattr +C.


So let’s disable CoW for for /var/log/journal and all existing files (this must be done from other system, files mustn’t be used during this process):

# mv /var/log/journal /var/log/journal_old
# mkdir /var/log/journal
# chattr +C /var/log/journal
# cp -a /var/log/journal_old/* /var/log/journal
# rm -rf /var/log/journal_old


Optionally: create yourself a user, upload your public key and disable ssh password and root login.

And since we have btrfs, let’s make a snapshot of our fresh new install:

# btrfs subvolume snapshot / root_snapshot_2015-07-10_postinstall


In the next series of posts I will show you how I’ve installed various services into Linux Containers.