Moving a live encrypted system from one hard disk to another

This a short memento based on earlier articles Moving a live system from one hard disk to another and http://Single passphrase to boot Devuan GNU/Linux with multiple encrypted partitions. This is useful when you start to move over your systems partitions from HDD to SSD, that nowadays are clearly worth their cheap price.

This article is made for Devuan GNU/Linux but should not be distro specific – you just might want to replace devuan string in later command by something else.

We start by setting some variables depending on the relevant drives. Any doubt about which drive is what, running lsblk should help.

# new NVMe disk 
NDISK=/dev/nvme0n1
NPART_PREFIX=p

# or alternatively for a SATA new disk:
# NDISK=/dev/sdb
# NPART_PREFIX=

LUKS_PREFIX=250g21

(AHCI SATA SSD are faster than HDD, but AHCI itself will be the bottleneck, so I’d suggest to install a NVMe SSD if your mainboard allows).

# key necessary to mount all partitions with a singlepassphrase

key=/boot/klucz
if [ ! -e $key ]; then dd if=/dev/urandom of=$key bs=1024 count=4 && chmod 400 $key ; fi

Next step is to replicate the disk structure. While this article is BIOS-boot based, it should go along UEFI:

parted $NDISK

(parted shell)
mklabel gpt
mkpart biosreserved ext2 1049kB  50,3MB
toggle 1 bios_grub
mkpart efi fat32 50,3MB  500MB
toggle 2 msftdata
mkpart swap linux-swap 500MB   16,0GB
toggle 3 swap
mkpart root ext4 16,0GB 250GB
print

Model: KINGSTON SA2000M8250G (nvme)
Disk /dev/nvme1n1: 250GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags: 

Number  Start   End     Size    File system     Name          Flags
 1      1049kB  50,3MB  49,3MB                  biosreserved  bios_grub
 2      50,3MB  500MB   450MB                   efi           msftdata
 3      500MB   16,0GB  15,5GB  linux-swap(v1)  swap          swap
 4      16,0GB  250GB   234GB   ext4            root

quit

We build the new system partition (luks1 is still mandatory with grub, but that won’t be true forever we can suppose):

cryptsetup luksFormat --type luks1 "$NDISK$NPART_PREFIX"4
cryptsetup luksOpen "$NDISK$NPART_PREFIX"4 "$LUKS_PREFIX"devuan64
cryptsetup  luksAddKey "$NDISK$NPART_PREFIX"4 $key
mkfs.ext4 /dev/mapper/"$LUKS_PREFIX"devuan64 -L "$LUKS_PREFIX"devuan64

mkdir /tmp/"$LUKS_PREFIX"devuan64
mount /dev/mapper/"$LUKS_PREFIX"devuan64 /tmp/"$LUKS_PREFIX"devuan64

ignore="backups home dev lost+found media proc run sys tmp"
for dir in $ignore; do touch /$dir.ignore ; done && for dir in /*; do if [ -d $dir ]; then if [ ! -e $dir.ignore ]; then /usr/bin/rsync --archive --one-file-system --delete $dir /tmp/"$LUKS_PREFIX"devuan64/ ; else if [ ! -e /tmp/"$LUKS_PREFIX"devuan64/$dir ]; then mkdir /tmp/"$LUKS_PREFIX"devuan64/$dir; fi ; rm $dir.ignore ; fi ; fi ; done

We update required system files:

echo " " >> /tmp/"$LUKS_PREFIX"devuan64/etc/crypttab
echo "# "`date` >> /tmp/"$LUKS_PREFIX"devuan64/etc/crypttab
echo "$LUKS_PREFIX"devuan64 UUID=`blkid -s UUID -o value "$NDISK$NPART_PREFIX"4` $key luks,tries=3,discard >> /tmp/"$LUKS_PREFIX"devuan64/etc/crypttab
echo "$LUKS_PREFIX"swap `find -L /dev/disk -samefile "$NDISK$NPART_PREFIX"3 | grep by-id | tail -1` /dev/urandom cipher=aes-xts-plain64,size=256,swap,discard >> /tmp/"$LUKS_PREFIX"devuan64/etc/crypttab

# comment out old lines
nano /tmp/"$LUKS_PREFIX"devuan64/etc/crypttab


echo " " >> /tmp/"$LUKS_PREFIX"devuan64/etc/fstab
echo "# "`date` >> /tmp/"$LUKS_PREFIX"devuan64/etc/fstab
echo "/dev/mapper/"$LUKS_PREFIX"devuan64	/		ext4	errors=remount-ro		0 1" >> /tmp/"$LUKS_PREFIX"devuan64/etc/fstab
echo "/dev/mapper/"$LUKS_PREFIX"swap	none		swap	sw		0 0" >> /tmp/"$LUKS_PREFIX"devuan64/etc/fstab

# comment out old lines
nano /tmp/"$LUKS_PREFIX"devuan64/etc/fstab



echo "Make sure this is in grub config:"
echo
echo GRUB_CMDLINE_LINUX=\"rd.luks.key=$key:UUID=`blkid "$NDISK$NPART_PREFIX"4 -s UUID -o value`\"
echo GRUB_ENABLE_CRYPTODISK=\"y\"
echo GRUB_PRELOAD_MODULES=\"luks cryptodisk lvm\"

# update grub config
nano /tmp/"$LUKS_PREFIX"devuan64/etc/default/grub

Last step is to install the boot loader on the new disk:

mount --bind /dev /tmp/"$LUKS_PREFIX"devuan64/dev
mount --bind /sys /tmp/"$LUKS_PREFIX"devuan64/sys
mount -t proc /proc /tmp/"$LUKS_PREFIX"devuan64/proc
chroot /tmp/"$LUKS_PREFIX"devuan64

grub-mkdevicemap
update-initramfs -u

# need to be retyped since it not in chroot environment
NDISK=/dev/nvme0n1

grub-install $NDISK
grub-mkconfig > /boot/grub/grub.cfg

That’s all.

Using RAM for transient data

When a system have lots of I/O, trouble may arise. If an optical hard drive is über-solicited, quite easily you may get many kinds of failures, high CPU load, just because of I/O errors. In such case, using RAM as disk, aka RAM disk, may be a good option, as it allows way more I/O than an optical hard drive. Solid State Drive (SSD) addresses partly this issue, but it seems to, still, have way higher access time and latency than RAM. RAM disk, on the other hand,  is non persistent (unlike SSD, though), quite an annoying drawback so even if you write some scripts to save data, you will loose some in case of power failure.

RAM disk is, actually, especially appropriate for temporary data, like /var/run, /var/lock or /tmp. Linux >= 2.4  supports tmpfs, some kind of RAM disk, that (as far I understand) does not reserve blocks of memory (meaning: it does not matter if you have a big tmpfs, unused memory in the tmpfs will still be available to the whole system).

Most of my computers have more than 1 Gb RAM. And, most of the time, they never use the Swap space. For instance (relevant lines are si and so, as swap in, swap out):

bender:$ vmstat
 procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa
 0  0      0 4146984 674704 1309432    0    0     6     9    3   34  2  1 97  0

nibbler:$ vmstat
procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa
 0  0      0 862044  23944  84088    0    0    10     0   42   22  0  0 99  0

moe:$ vmstat
procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa
 1  0      0 280552 166884 1297376    0    0     7    58   73   12  8  2 90  1

So they are good candidates to use tmpfs whenever possible. Do so with Debian GNU/Linux is fast-forward. Just edit /etc/default/rcS as follows (for /var/run & /var/lock):

RAMRUN=yes
RAMLOCK=yes

and add, in /etc/fstab (for /tmp):

tmpfs             /tmp     tmpfs     noexec    0    0

Next time you boot, diskfree should provide you with something like:

  $ df
Filesystem           1K-blocks      Used Available Use% Mounted on
tmpfs                  1033292         0   1033292   0% /lib/init/rw
varrun                 1033292       648   1032644   1% /var/run
varlock                1033292         0   1033292   0% /var/lock
tmpfs                  1033292         4   1033288   1% /dev/shm
tmpfs                  1033292         0   1033292   0% /tmp

Update, questionable:

In Iceweasel/Firefox, to use tmpfs for caching data, on the page about:config, I added a new entry as follows:

browser.cache.disk.parent_directory | user set | string | /tmp/iceweasel-myusername

Somehow breaking LSB, on one box, I gave a try setting /var/tmp as tmpfs. Normally, /var/tmp content should not be erased on reboot. I considered writing a short script to save its data over reboot, however I assumed that copying stuff from and to /var/tmp on shutdown and boot could actually be slower than letting /var/tmp being reconstructed. I’ll need more input on this to estimate whether this is actually pertinent – so far, I can already tell KDE startup seems slower. Making so is just a matter of adding to /etc/fstab:

tmpfs             /var/tmp     tmpfs     noexec    0    0

Update’s Update:

Having /var/tmp on tmpfs is a no go. Debian GNU/Linux use it properly, having its content rebuilt at each reboot is too slow and having its content in tmpfs is likely to fill it. I dropped this bad idea.