Booting two Devuan GNU/Linux installed on encrypted partitions on a single disk

Followup of previous article (Single passphrase to boot Devuan GNU/Linux with multiple encrypted partitions), I found out that if you have two clone system, both on encrypted partitions, on the same hard disk, grub/os-prober as of today fails to automatically configure boot for the clone.

It the concept of having such clone system odd? Not really if you think of laptop that you use for two completely unrelated activities (work and out of the work?), that you do not want to mix at all.

I spent quite a time trying to understand why the clone system was ignored by os-prober and all, even though the partition it was on was mounted.

In the end, I decided it was easier to actually clone the config built for the running system, adjusting the UUID of partitions than to look further.

Here is my /etc/grub.d/11_linux_cryptoclone wrapper for /etc/grub.d/10_linux:

#! /bin/bash
set -e

# require GRUB_LINUX_CLONE_MAPPER_NAME to be set
# for instance to XY if the relevant fs is /dev/mapper/XY
# along with relevant grub parameters
#GRUB_ENABLE_CRYPTODISK=y
#GRUB_PRELOAD_MODULES="luks cryptodisk lvm"
#GRUB_LINUX_CLONE_MAPPER_NAME=XY

. /etc/default/grub
[ x"$GRUB_LINUX_CLONE_MAPPER_NAME" == x ] && exit

# setup
CLONE=$GRUB_LINUX_CLONE_MAPPER_NAME # only necessary to edit
CLONEUUID=`blkid /dev/mapper/$CLONE -o value -s UUID`
CLONENAME="Devuan GNU/Linux (on $CLONE)"
CLONECRYPTOUUID=`grep $CLONE /etc/crypttab | awk '{print $2}' | cut -f 2 -d =`
CLONECRYPTOUUIDGRF=`echo $CLONECRYPTOUUID | tr -d -`

ORIG=`df / --output=source | tail -1 | tr -d /dev/mapper`
ORIGUUID=`blkid /dev/mapper/$ORIG -o value -s UUID`
ORIGNAME="Devuan GNU/Linux"
ORIGCRYPTOUUID=`grep $ORIG /etc/crypttab | awk '{print $2}' | cut -f 2 -d =`
ORIGCRYPTOUUIDGRF=`echo $ORIGCRYPTOUUID | tr -d -`

# produce arranged conffile
>&2 echo "$ORIG -> $CLONE:"
`dirname "$0"`/10_linux | sed s/$ORIGCRYPTOUUID/$CLONECRYPTOUUID/ig | sed s/$ORIGCRYPTOUUIDGRF/$CLONECRYPTOUUIDGRF/ig | sed s/$ORIGUUID/$CLONEUUID/ig | sed s@"$ORIGNAME"@"$CLONENAME"@ig
>&2 echo " $ORIGCRYPTOUUID -> $CLONECRYPTOUUID"
>&2 echo " $ORIGUUID -> $CLONEUUID"

# make sure there is proper kernel and initrd installed
MCLONE=0
CLONEDIR=`grep /dev/mapper/$CLONE /etc/mtab | awk '{print $2}'`
if [ "x$CLONEDIR" == "x" ]; then
 MCLONE=1
 mount /dev/mapper/$CLONE
 CLONEDIR=`grep /dev/mapper/$CLONE /etc/mtab | awk '{print $2}'`
fi
for file in /boot/config-* /boot/init* /boot/vmlinuz-* /boot/System.map-*; do
 [ ! -e "$CLONEDIR/$file" ] && >&2 echo " $CLONE $file missing!"
done
>&2 echo " (remember $CLONE needs properly built initramfs)" 
if [ $MCLONE == 1 ]; then umount /dev/mapper/$CLONE ; fi

# EOF

As said it requires you to add GRUB_LINUX_CLONE_MAPPER_NAME=XY in /etc/default/grub, XY  being the /dev/mapper/XY of the clone system.

It expect the clone system to be similarly set up: it needs to have proper initramfs for the same kernel.

It also expect this clone system to be accessible and set in /etc/crypttab et /etc/fstab, since it needs to be able to find clone UUIDs which should not come as a surprise because if it would have to be if os-prober was to find it anyway.

Once done, you can simply run

update-grub
Advertisements

Single passphrase to boot Devuan GNU/Linux with multiple encrypted partitions

These days, considering the amount of data are stored on an average computer and how easy is it to get access to it once you get physical access, running such computer without any form of encryption seem unsound. Especially since it is reasonably easy to set up a en encrypted system and does not seems to imply much overhead.

When you have an old setup you are fine with, using numerous partitions or systems, it can be inconvenient, though. For instance  if you have to type a long specific passphrase 5 times when booting your computer.

There are a few things I found useful to make my life easier. Obviously, any shortcut security wise means less security. It is help to you to decide whether the risk is worth it or not depending on what kind of data you carry, what kind of attackers you expect, etc. This is part 1.

Single passphrase to boot GNU/Linux with multiple encrypted partitions

One obviously approach to type a single passphrase to boot a system is to have the boot loader files on a regular partition and the rest on a single encrypted partition. In GNU/Linux case, you would have /boot on a specific non-encrypted partition. Fact is anyone with access to your computer can easily replace your kernel or initramfs with a malicious one and you would not notice.

So I think non-encrypted /boot is as much of the table as would be a non-encrypted swap partition.

So for the boot manager grub to load, /boot need to be readable: the passphrase will be required here. The idea is that from this moment on, a keyfile will be used instead of passphrase to load any other partition.

I guess there is not much point to describe in detail the crypt setup itself (I followed the many guides out there). For each partition you want:

# 1. you create the partition with parted/fdisk

# 2. you format it as encrypted
cryptsetup luksFormat /dev/sdX1

# 2b. you record it in crypttab
# <target name> <source device> <key file> <options>
echo "Name1 UUID=`blkid -s UUID -o value /dev/sdX1` /boot/k/ka luks,tries=3" >> /etc/crypttab

# 3. you open the encrypt-formatted partition
cryptsetup luksOpen /dev/sdX1 Name1

# 4. you format the resulting /dev/mapper... to a regular filesystem
mkfs.ext4 /dev/mapper/Name1 -L Name1

# 4b. you record it in fstab (adjusting the mount point!)
# <file system> <mount point> <type> <options> <dump> <pass> 
echo "/dev/mapper/Name1 / ext4 errors=remount-ro 0 1" >> /etc/fstab

# you are set, you can mount the partition, 
#  and install the system/copy the system there

The swap  require specific treatment. Provided you know one with partition is the current  unencrypted swap is (here sdX?), this is enough:

# update crypttab
echo "SW `find -L /dev/disk -samefile /dev/sdX? | grep by-id | tail -1` /dev/urandom swap" >> /etc/crypttab

# update fstab
echo "/dev/mapper/SW  none   swap sw 0 0" >> /etc/fstab

Noticed the /boot/k/ka? That’s the unlocking key. You can use whatever other filename, just be consistent.

# generate some
dd if=/dev/urandom of=/boot/k/ka bs=1024 count=4
chmod 400 /boot/k/ka

# and obviously, add it to any luks formatted partition:
for part in `blkid | grep crypto_LUKS | awk {'print $1 '} | tr -d :$`; do cryptsetup -v luksAddKey $part /boot/k/ka; done

Then, you need a proper initramfs:

# dracut works almost out of the box
apt install dracut

# set up a few things
echo 'omit_dracutmodules+="systemd systemd-initrd dracut-system"' > /etc/dracut.conf.d/00-systemd.conf
echo 'add_dracutmodules+="crypt lvm"
install_items+="/sbin/e2fsck /sbin/cryptsetup /boot/k/ka"' > /etc/dracut.conf.d/99-luks.conf

# (re)build ramfs
dracut --force

# make sure there are no old initrd leftovers, that would confuse grub
rm /boot/initrd* -f

 

Then, you need to edit grub (version 2!) config:

# first obtain the UUID of crypted partition (not the /dev/mapper/... one) 
# that hold the /boot partition. 
# (it was Name1 earlier but obviously it depends to the real name you gave)
grep Name1 /etc/crypttab

# now edit /etc/default, with XXXXXXXXXXXX being the UUID value
# you just found.
GRUB_CMDLINE_LINUX="rd.luks.key=/boot/k/ka:UUID=XXXXXXXXXXXX"
GRUB_ENABLE_CRYPTODISK=y
GRUB_PRELOAD_MODULES="luks cryptodisk lvm"

# and now update-grub (install grub if not done yet)
update-grub

It took me a while to find the proper rd.luks.key value, no docs I read were clear about it. Many give the impression that putting rd.luks.key=/keyfile or rd.luks.key=/keyfile:/ would be enough since the key is actually on the same partition as grub.cfg. But no.

That is all. Rebooting now, you should be asked for the passphrase before getting the grub menu. And then boot process should be uninterrupted.

Getting nginx’s wildcard-based server names to pass Exim HELO syntax checks

Many PHP-based apps, like webmails, when using SMTP functions, depends on nginx server_name value to set up the HELO sent.

But if your server_name value is wildcard-based, you’ll get “syntactically invalid argument(s)” from the SMTP server. Example with ownCloud.

Assuming that the SMTP running on the same host as your webmail is not accepting mail but from the webmail itself, you can easily work around this. You can addd

helo_allow_chars=^~

in, for example, /etc/exim4/conf.d/main/00_webmail, if your server name is something like ~^mx.

 

Isn’t SRS breaking SPF itself, at least regarding spam?

Earlier on this blog, I proposed ways to implement SPF (Sender Policy Framework). I recently noticed mails forwarded by one of my servers being tagged as spam by gmail.com due to SPF checks. It means that while SPF works for my domains with near to 0 user base, no real business of forwarding, it is a nuisance for forwarding in general. So you are advised to use SRS (Sender Rewriting Scheme). Strangely enough it is not fully integrated on main servers and some implementation (Exim in Debian) are based on unmaintained library (SRS C library).

Unmaintained?

Fact is SRS is far from being nice. It makes so your own forwarding server is vouching for fowarded mails. But why would you want that?

SPF test will fail because your forwarding server is not a registered valid source for (forwarded) mails sent from domain X. SRS proposal is that your server will alter header so to forward the mail from X domain X to appear as sent from an address of your own domain for you server is a registered valid source.

I guess the logic is to make forwarders somehow responsible of filtering, not bad in principle.

But it also means that for each spam forwarders fail to identify, they’ll be tagged as spam originator. It is particulary annoying when forwarding is made on public addresses bound to attract spam. So it seems better to get a failed SPF test on every forwarded messages including valid ones than a valid SPF test on every forwarded messages including spam.

SPF without SRS breaks forwarding. But SPF with SRS, the workaround, breaks SPF itself regarding spam and will give you (your IPs, your domains) bad rep, with will make your legit mail at risk of being blacklisted, unless you apply an overly harsh policy on forwarded mails.

Annoying. I am thinking removing SPF completely, instead.  For now, I am updating my SPF records to remove any Fail statement, since there is no way for me to know whether one of my mail can legitimately be forwarded through several servers.  Funny enough, google that promotes SPF usage recommends using SoftFail over Fail. But I might even reset to Neutral.

Interesting link on topic : Mail server setup wih SRS ; Why not SPF?

Alternative: I implemented DKIM on my servers. Seems much smarter to have a server signature.

Cloning installed packages list over LXC containers with apt-clone

apt-clone is quite convenient to run LXC containers with the same set of installed packages.

here’s a short bash function to do run apt-clone on a list of containers to synchronize them all:

function lxc-clone {
    MKTEMP=`mktemp --dry-run` 
    
    guests=($(lxc-ls --active))

    # first get clones for each
    for guest in "${guests[@]}"; do
	echo -e "[${shell_datecolor}$(date +%H:%M:%S)${shell_clear} ${shell_containercolor}$guest:${shell_clear} ${shell_promptcolor}#${shell_clear} ${shell_invert}apt-clone clone $@${shell_clear}]"
	lxc-attach -n "$guest" -- apt-clone clone "$MKTEMP.$guest"
	cp -v `lxc-config lxc.lxcpath`/"$guest"/rootfs"$MKTEMP.$guest".apt-clone.tar.gz "$MKTEMP.$guest".apt-clone.tar.gz
    done

    # then do a restore of all in each
    for guest in "${guests[@]}"; do
	echo -e "[${shell_datecolor}$(date +%H:%M:%S)${shell_clear} ${shell_containercolor}$guest:${shell_clear} ${shell_promptcolor}#${shell_clear} ${shell_invert}apt-clone restore $@${shell_clear}]"
	for guestwithin in "${guests[@]}"; do
	    echo "=> ...$guestwithin"
	    cp -v "$MKTEMP.$guestwithin".apt-clone.tar.gz `lxc-config lxc.lxcpath`/"$guest"/rootfs"$MKTEMP.$guestwithin".apt-clone.tar.gz	    
	    lxc-attach -n "$guest" -- apt-clone restore "$MKTEMP.$guestwithin".apt-clone.tar.gz
	    rm -fv `lxc-config lxc.lxcpath`/"$guest"/rootfs"$MKTEMP.$guestwithin".apt-clone.tar.gz
	done
	
    done    

    rm -f "$MKTEMP".*.apt-clone.tar.gz
}

The variable $guest sets which LXC containers to work on. Here, it works on all active containers.

(the color variables are set in stalag13-00-shell.sh but arent required)

Setting up LXC containers with mapped GID/UID

Result of ps aux on a LXC host is quite messy! But that can be improved, with the benefit of having each LXC container using a specific namespace: for instance « having a process is unprivileged for operations outside the user namespace but with root privileges inside the namespace ». Easier to check on and likely to be more secure.

A reply to the question « what is an unpriviledged LXC container » provides a working howto.  The following is a proposal to implement it even more easily.

For each LXC container, you need to pick a UID/GID range. For instance, for container test1, let’s pick 100000 65536. It means that root in test1, will actually be 100000 on the main host. User 1001 in test1 will be 101001 on the main host and so on.

So you must add the map on the main host:

 usermod --add-subuids 100000-165535 root
 usermod --add-subgids 100000-165535 root

Then you must configure the relevant LXC container configuration file whose location varies according to you lxc.lxcpath.

# require userns.conf associated to the distribution used
lxc.include = /usr/share/lxc/config/debian.userns.conf

# specific user map
lxc.id_map = u 0 100000 65536
lxc.id_map = g 0 100000 65536

Then you need to update files ownership according to the new mapping. Original poster proposed a few shell commands but that would only be enough to start the container. Files within the container would not get inappropriate ownership: most likely, files that belongs to root/0 on the host would show up as owned by nobody/65534. For proper ownership to root/0 within the LXC container, they need to belong to 100000 on the the host.

Here comes my increase-uid-gid.pl script: it’ll take as argument your LXC container name (or alternatively a path, useful for mounts that are residing outside of it) and value to increment. In the first case, it’ll be 100000:

# shutting down the container before touching it
lxc-stop --name test1 

# obtain the script
cd
wget https://gitlab.com/yeupou/stalag13/raw/master/usr/local/bin/increase-uid-gid.pl
chmod +x ~/increase-uid-gid.pl

# chown files +100000
~/increase-uid-gid.pl --lxc=test1 --increment=100000

# start the container
lxc-start --name test1

That’s all. Obviously, you should check that every daemon is still functionning properly. If not, either it means a file owership changed was missed (happened once to a container with transmission-daemon) or maybe its mode was not properly set beforehand (happened once to a container with exim4 that was not setuid – it led to failure with procmail_pipe).

Next container test2? Edit `lxc-config lxc.lxcpath`/test2/config:

# require userns.conf associated to the distribution used
lxc.include = /usr/share/lxc/config/debian.userns.conf

# specific user map
lxc.id_map = u 0 200000 65536
lxc.id_map = g 0 200000 65536

Then run:

lxc-stop --name test2
usermod --add-subuids 200000-165535 root
usermod --add-subgids 200000-165535 root
~/increase-uid-gid.pl --lxc=test2 --increment=200000
lxc-start  --name test2

I tested the script on 16 LXC containers with no problem so far.

If you need to deal with extra mounted directories (lxc.mount.entry=…), use –path option.

If you need to deal with a container that was already mapped (for instance already 100000 65536 but you would like it to be 300000 65536), you’ll need to raise the –limit that is by default equal to increment value: that would be –increment=200000 –limit=300000. This limit exists so you can re-run the script on the same container with no risk of having files getting out of range.

For the record:

For the record, follows the script as it is today (but it always best to get latest version from gitlab – because I wont update any bugfixes/improvements on this page) :

#!/usr/bin/perl

use strict;
use File::Find;
use Getopt::Long;

### options
my ($getopt, $help, $path, $lxc, $increase, $limit);
eval {
    $getopt = GetOptions("help" => \$help,
			 "lxc=s" => \$lxc,
	                 "path=s" => \$path,
	                 "increase=i" => \$increase,
	                 "limit=i" => \$limit);
};

if ($help or
    !$increase or
    (!$path and !$lxc)) {
    # increase is mandatory
    # either path or lxc also
    # print help if missing
        print STDERR "
  Usage: $0 [OPTIONS] --lxc=name --increase=100000
             or
         $0 [OPTIONS] --path=/directory/ --increase=100000

Will increase all files UID/GID by the value set.

      --lxc=name    LXC container name, will be used to determine path
      --path=/dir   No LXC assumption, just work on a given path
    
      --increase=n  How much to increment
      --limit=n     Increase limit, by default equal to increase

Useful for instance when you add to a LXC container such config:
  lxc.id_map = u 0 100000 65536
  lxc.id_map = g 0 100000 65536

And the host system having the relevant range set: 
  usermod --add-subuids 100000-165535 root
  usermod --add-subgids 100000-165535 root

It would update UID/GID within rootfs to match the proper range. Note that
additional configured mount must also be updated accordingly, using --path 
for instance.

By default, limit is set to increase value so you can run it several time on 
the same container, the increase will be effective only once. You can set the
limit to something else, for instance if you want to increase by 100000 a 
container already within the 100000-165536 range, you would have to 
use --increase=100000 --limit=200000.

This script is primitive: it should work in most case, but if some service fail
to work after the LXC container restart, it is probably because one or several 
files were missed.

Author: yeupou\@gnu.org
       https://yeupou.wordpress.com/
";
	exit;
}

# limit set to increase by default
$limit = $increase unless $limit;

# if lxc set, use it to define path
if ($lxc) {
    my $lxcpath = `lxc-config lxc.lxcpath`;
    chomp($lxcpath);
    $path = "$lxcpath/$lxc/rootfs";
}

# in any case, path must be given and found
die "path $path: not found, exit" unless -e $path;
print "path: $path\n";


### run
find(\&wanted, $path);

# if lxc, check main container config
if ($lxc) {
    my $lxcpath = `lxc-config lxc.lxcpath`;
    chomp($lxcpath);
    
    # https://unix.stackexchange.com/questions/177030/what-is-an-unprivileged-lxc-container
    # directory for the container
    chown(0,0, "$lxcpath/$lxc");
    chmod(0775, "$lxcpath/$lxc");
    # container config
    chown(0,0, "$lxcpath/$lxc/config");
    chmod(0644, "$lxcpath/$lxc/config");
    # container rootfs - chown will be done during the wanted()
    chmod(0775, "$lxcpath/$lxc/rootfs");
}


exit;

sub wanted {
    print $File::Find::name;
    
    # find out current UID/GID
    my $originaluid = (lstat $File::Find::name)[4];
    my $newuid = $originaluid;
    my $originalgid = (lstat $File::Find::name)[5];
    my $newgid = $originalgid;
    
    # increment but only if we are below the new range
    $newuid += $increase if ($originaluid < $increase);
    $newgid += $increase if ($originalgid < $increase);

    # update if there is at least one change
    if ($originaluid ne $newuid or
	$originalgid ne $newgid) {
	chown($newuid, $newgid, $File::Find::name);
	print " set to UID:$newuid GID:$newgid\n";
    } else {
	print " kept to UID:$originaluid GID:$originalgid\n";
    }
      
}

# EOF