Downgrading Nextcloud 18.0.3.0 to 17.0.5

I upgraded Nextcloud to the latest save without realizing the gallery app has been replaced by an half-baked “photos” app, completely useless to share pictures in any relevant way to me, in addition to a bug with ublock origin making the whole “sharing” interface disappearing.

Rolling back is not as easy at it seems: the “occ” php app check with your config version if it matches the software version, and if not matching just plainly refuses to work with:

An unhandled exception has been thrown:
OC\HintException: [0]: Downgrading is not supported and is likely to cause unpredictable issues (from 18.0.3.0 to 17.0.5.0) ()

So you need to update this config/config.php. Then, playing with occ app:list, occ app:remove and occ app:install only you can get back to a working install.

SPF-aware greylisting with Exim and memcache

This is a followup of my 2011’s article avoiding Spams with SPF and greylisting within Exim. What changed since then? I actually am not more harrassed by spam that I was earlier on. It works. I am spam free since a decade now. No, but, however, several importants mail providers have a tendancy to send mail through multiples SMTPs, so many it took a while for any of them to do at least two attempt. So some mails takes ages to pass the greylist.

Contemplating the idea to use opensmtpd, I incidentally found an interesting proposal to mix greylisting of IP with SPF-validated domains.

The idea is that you greylist either an SMTP IP or a domain including any SMTP IP approved by SPF.

I updated the memcached-exim.pl script previously used and described. It was simplified because I dont think useful to actually make greylist per sender and recipient, only per IP or domain. Now it either only greylist IP, if not validated by SPF, or the domain and IP on success (to save a few SPF further test).

I dont think it should have any noticeable impact on the server behavior. SPF is anyway checked, so it is meaningless since there is local caching DNS on my mail servers.

The earlier /etc/exim4/memcached.conf is actually no longer required (defaults are enough). You still need exim configuration counterparts:  /etc/exim4/conf.d/main/00_stalag13-config_0greylist and /etc/exim4/conf.d/acl/26_stalag13-config_check_rcpt.

Delisting an Exim4 server from Office365 ban list

Ever tried to get delisted from Office365 ban list, for whatever reason you might try to get (new IP for a server that was abused in the past or else, you won’t know since they wont tell – and it even looks like they probably dont even really know)?

It is a funny process, because it involves receive a mail from their servers, a mail that will probably be flagged as spam, with clues so big that it might be blocked at SMTP time.

With Exim4, you’ll probably get in the log something like:

2019-08-20 22:18:09 1i0Aa1-0004Hu-8h H=mail-eopbgr740042.outbound.protection.outlook.com (NAM01-BN3-obe.outbound.protection.outlook.com) [40.107.74.42] X=TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256 CV=no F=<no-reply@microsoft.com> rejected after DATA: maximum allowed line length is 998 octets, got 3172

Long story short (this length test is not welcomed by all users), add /etc/exim4/conf.d/main/00_localoptions add

IGNORE_SMTP_LINE_LENGTH_LIMIT=1

and then restart the server.

Try delisting and check your spam folder. You should get now the relevant mail. Whatever we think about the lenght limit test of Exim4 (based on RCF, isn’t it?), you still end up with a mail sent by Office365 like this:

X-Spam-Flag: YES
X-Spam-Level: *****
X-Spam-Status: Yes, score=5.2 required=3.4 tests=BASE64_LENGTH_79_INF,
	HTML_IMAGE_ONLY_08,HTML_MESSAGE,MIME_HTML_ONLY,MIME_HTML_ONLY_MULTI,
	MPART_ALT_DIFF,SPF_HELO_PASS,SPF_PASS autolearn=no autolearn_force=no
	version=3.4.2
X-Spam-Report: 
	* -0.0 SPF_PASS SPF: sender matches SPF record
	* -0.0 SPF_HELO_PASS SPF: HELO matches SPF record
	*  0.7 MIME_HTML_ONLY BODY: Message only has text/html MIME parts
	*  0.7 MPART_ALT_DIFF BODY: HTML and text parts are different
	*  0.0 HTML_MESSAGE BODY: HTML included in message
	*  1.8 HTML_IMAGE_ONLY_08 BODY: HTML: images with 400-800 bytes of
	*      words
	*  2.0 BASE64_LENGTH_79_INF BODY: base64 encoded email part uses line
	*      length greater than 79 characters
	*  0.0 MIME_HTML_ONLY_MULTI Multipart message only has text/html MIME
	*      parts

Considering the context, it screams incompetence.

Booting two Devuan GNU/Linux installed on encrypted partitions on a single disk

Followup of previous article (Single passphrase to boot Devuan GNU/Linux with multiple encrypted partitions), I found out that if you have two clone system, both on encrypted partitions, on the same hard disk, grub/os-prober as of today fails to automatically configure boot for the clone.

It the concept of having such clone system odd? Not really if you think of laptop that you use for two completely unrelated activities (work and out of the work?), that you do not want to mix at all.

I spent quite a time trying to understand why the clone system was ignored by os-prober and all, even though the partition it was on was mounted.

In the end, I decided it was easier to actually clone the config built for the running system, adjusting the UUID of partitions than to look further.

Here is my /etc/grub.d/11_linux_cryptoclone wrapper for /etc/grub.d/10_linux:

#! /bin/bash
set -e

# require GRUB_LINUX_CLONE_MAPPER_NAME to be set
# for instance to XY if the relevant fs is /dev/mapper/XY
# along with relevant grub parameters
#GRUB_ENABLE_CRYPTODISK=y
#GRUB_PRELOAD_MODULES="luks cryptodisk lvm"
#GRUB_LINUX_CLONE_MAPPER_NAME=XY

. /etc/default/grub
[ x"$GRUB_LINUX_CLONE_MAPPER_NAME" == x ] && exit

# setup
CLONE=$GRUB_LINUX_CLONE_MAPPER_NAME # only necessary to edit
CLONEUUID=`blkid /dev/mapper/$CLONE -o value -s UUID`
CLONENAME="Devuan GNU/Linux (on $CLONE)"
CLONECRYPTOUUID=`grep $CLONE /etc/crypttab | awk '{print $2}' | cut -f 2 -d =`
CLONECRYPTOUUIDGRF=`echo $CLONECRYPTOUUID | tr -d -`

ORIG=`df / --output=source | tail -1 | tr -d /dev/mapper`
ORIGUUID=`blkid /dev/mapper/$ORIG -o value -s UUID`
ORIGNAME="Devuan GNU/Linux"
ORIGCRYPTOUUID=`grep $ORIG /etc/crypttab | awk '{print $2}' | cut -f 2 -d =`
ORIGCRYPTOUUIDGRF=`echo $ORIGCRYPTOUUID | tr -d -`

# produce arranged conffile
>&2 echo "$ORIG -> $CLONE:"
`dirname "$0"`/10_linux | sed s/$ORIGCRYPTOUUID/$CLONECRYPTOUUID/ig | sed s/$ORIGCRYPTOUUIDGRF/$CLONECRYPTOUUIDGRF/ig | sed s/$ORIGUUID/$CLONEUUID/ig | sed s@"$ORIGNAME"@"$CLONENAME"@ig
>&2 echo " $ORIGCRYPTOUUID -> $CLONECRYPTOUUID"
>&2 echo " $ORIGUUID -> $CLONEUUID"

# make sure there is proper kernel and initrd installed
MCLONE=0
CLONEDIR=`grep /dev/mapper/$CLONE /etc/mtab | awk '{print $2}'`
if [ "x$CLONEDIR" == "x" ]; then
 MCLONE=1
 mount /dev/mapper/$CLONE
 CLONEDIR=`grep /dev/mapper/$CLONE /etc/mtab | awk '{print $2}'`
fi
for file in /boot/config-* /boot/init* /boot/vmlinuz-* /boot/System.map-*; do
 [ ! -e "$CLONEDIR/$file" ] && >&2 echo " $CLONE $file missing!"
done
>&2 echo " (remember $CLONE needs properly built initramfs)" 
if [ $MCLONE == 1 ]; then umount /dev/mapper/$CLONE ; fi

# EOF

As said it requires you to add GRUB_LINUX_CLONE_MAPPER_NAME=XY in /etc/default/grub, XY  being the /dev/mapper/XY of the clone system.

It expect the clone system to be similarly set up: it needs to have proper initramfs for the same kernel.

It also expect this clone system to be accessible and set in /etc/crypttab et /etc/fstab, since it needs to be able to find clone UUIDs which should not come as a surprise because if it would have to be if os-prober was to find it anyway.

Once done, you can simply run

update-grub

Single passphrase to boot Devuan GNU/Linux with multiple encrypted partitions

These days, considering the amount of data are stored on an average computer and how easy is it to get access to it once you get physical access, running such computer without any form of encryption seem unsound. Especially since it is reasonably easy to set up a en encrypted system and does not seems to imply much overhead.

When you have an old setup you are fine with, using numerous partitions or systems, it can be inconvenient, though. For instance  if you have to type a long specific passphrase 5 times when booting your computer.

There are a few things I found useful to make my life easier. Obviously, any shortcut security wise means less security. It is help to you to decide whether the risk is worth it or not depending on what kind of data you carry, what kind of attackers you expect, etc. This is part 1.

Single passphrase to boot GNU/Linux with multiple encrypted partitions

One obviously approach to type a single passphrase to boot a system is to have the boot loader files on a regular partition and the rest on a single encrypted partition. In GNU/Linux case, you would have /boot on a specific non-encrypted partition. Fact is anyone with access to your computer can easily replace your kernel or initramfs with a malicious one and you would not notice.

So I think non-encrypted /boot is as much of the table as would be a non-encrypted swap partition.

So for the boot manager grub to load, /boot need to be readable: the passphrase will be required here. The idea is that from this moment on, a keyfile will be used instead of passphrase to load any other partition.

I guess there is not much point to describe in detail the crypt setup itself (I followed the many guides out there). For each partition you want:

# 1. you create the partition with parted/fdisk

# 2. you format it as encrypted
cryptsetup luksFormat /dev/sdX1

# 2b. you record it in crypttab
# <target name> <source device> <key file> <options>
echo "Name1 UUID=`blkid -s UUID -o value /dev/sdX1` /boot/k/ka luks,tries=3" >> /etc/crypttab

# 3. you open the encrypt-formatted partition
cryptsetup luksOpen /dev/sdX1 Name1

# 4. you format the resulting /dev/mapper... to a regular filesystem
mkfs.ext4 /dev/mapper/Name1 -L Name1

# 4b. you record it in fstab (adjusting the mount point!)
# <file system> <mount point> <type> <options> <dump> <pass> 
echo "/dev/mapper/Name1 / ext4 errors=remount-ro 0 1" >> /etc/fstab

# you are set, you can mount the partition, 
#  and install the system/copy the system there

The swap  require specific treatment. Provided you know one with partition is the current  unencrypted swap is (here sdX?), this is enough:

# update crypttab
echo "SW `find -L /dev/disk -samefile /dev/sdX? | grep by-id | tail -1` /dev/urandom swap" >> /etc/crypttab

# update fstab
echo "/dev/mapper/SW  none   swap sw 0 0" >> /etc/fstab

Noticed the /boot/k/ka? That’s the unlocking key. You can use whatever other filename, just be consistent.

# generate some
dd if=/dev/urandom of=/boot/k/ka bs=1024 count=4
chmod 400 /boot/k/ka

# and obviously, add it to any luks formatted partition:
for part in `blkid | grep crypto_LUKS | awk {'print $1 '} | tr -d :$`; do cryptsetup -v luksAddKey $part /boot/k/ka; done

Then, you need a proper initramfs:

# dracut works almost out of the box
apt install dracut

# set up a few things
echo 'omit_dracutmodules+="systemd systemd-initrd dracut-system"' > /etc/dracut.conf.d/00-systemd.conf
echo 'add_dracutmodules+="crypt lvm"
install_items+="/sbin/e2fsck /sbin/cryptsetup /boot/k/ka"' > /etc/dracut.conf.d/99-luks.conf

# (re)build ramfs
dracut --force

# make sure there are no old initrd leftovers, that would confuse grub
rm /boot/initrd* -f

 

Then, you need to edit grub (version 2!) config:

# first obtain the UUID of crypted partition (not the /dev/mapper/... one) 
# that hold the /boot partition. 
# (it was Name1 earlier but obviously it depends to the real name you gave)
grep Name1 /etc/crypttab

# now edit /etc/default, with XXXXXXXXXXXX being the UUID value
# you just found.
GRUB_CMDLINE_LINUX="rd.luks.key=/boot/k/ka:UUID=XXXXXXXXXXXX"
GRUB_ENABLE_CRYPTODISK=y
GRUB_PRELOAD_MODULES="luks cryptodisk lvm"

# and now update-grub (install grub if not done yet)
update-grub

It took me a while to find the proper rd.luks.key value, no docs I read were clear about it. Many give the impression that putting rd.luks.key=/keyfile or rd.luks.key=/keyfile:/ would be enough since the key is actually on the same partition as grub.cfg. But no.

That is all. Rebooting now, you should be asked for the passphrase before getting the grub menu. And then boot process should be uninterrupted.

Getting nginx’s wildcard-based server names to pass Exim HELO syntax checks

Many PHP-based apps, like webmails, when using SMTP functions, depends on nginx server_name value to set up the HELO sent.

But if your server_name value is wildcard-based, you’ll get “syntactically invalid argument(s)” from the SMTP server. Example with ownCloud.

Assuming that the SMTP running on the same host as your webmail is not accepting mail but from the webmail itself, you can easily work around this. You can addd

helo_allow_chars=^~

in, for example, /etc/exim4/conf.d/main/00_webmail, if your server name is something like ~^mx.