Receiving and parsing DMARC reports on main and secondary mail servers

DMARC is made so aggregated reports will be sent by others servers to yours in order for you to find out if some spam source are trying to impersonate your domains. DMARC working on top of SPF and DKIM, so you need at least one of these. I already mentioned SPF here (even though my setup changed since then, since I use “v=spf1 mx a -all” as SPF DNS record now). If you have neither SPF or DKIM, I suggest you take a look at mail-tester.com first.

The destination of these reports os set in your DMARC DNS record , something like:

v=DMARC1; p=quarantine; rua=mailto:dmarc@thisdomain; sp=quarantine; ri=604800

Unfortunately, these reports are XML and frequent. There are not made to be read by human. And not so many solutions to parse and aggregate these are available; not so surprisingly since lot of customer services based on this are provided by companies.

I do not require anything overly fancy, here’s my basic setup satisfying my modest needs: I only need this to work on two twin mail servers (mxA and mxB here), no matter if one is down.¬† Since reports are sent by mail, so to the first server that’ll accept them, they need to be replicated from/to mxA to/from mxB.

On each server, create dmarc user. Then create home subdirectories:

adduser dmarc
su dmarc
cd 
mkdir mxA mxB

As suggested earlier, add dmarc user as recipient for the reports in the DNS record:

_dmarc 10800 IN TXT "v=DMARC1; p=quarantine; rua=mailto:dmarc@thisdomain; sp=quarantine; ri=604800"

Install the parser, filling database

On each servers, install dmarcts-report-parser.

# included in my -utils-exim deb package but you can simply clone it:
git clone https://github.com/techsneeze/dmarcts-report-parser.git dmarcts-report-parser
cp dmarcts-report-parser/dmarcts-report-parser.pl /usr/local/bin
chmod +x /usr/local/bin/dmarcts-report-parser.pl

# copy conffile (assuming we're on mxA)
cp dmarcts-report-parser/dmarcts-report-parser.conf.sample /home/dmarc/mxA/dmarcts-report-parser.conf

# requires some libraries
apt-get install libmail-imapclient-perl libmime-tools-perl libxml-simple-perl libclass-dbi-mysql-perl libio-socket-inet6-perl libio-socket-ip-perl libperlio-gzip-perl libmail-mbox-messageparser-perl unzip

The conffile needs to be modified, most notably:

$delete_reports=1
$delete_failed = 1;

You also need a MySQL server with  dmarc database:

apt-get install mariadb-server
mysql -e "CREATE DATABASE dmarc"

From this moment, the command cd /home/dmarc/mxA/ && dmarcts-report-parser.pl -m *.mbox should already be effective. But you need to fill these mbox files. We’ll do so with a simple ~/.procmailrc as follows:

HERE=mxA
NOTHERE=mxB

DATE=`date +%Y%m%d`
DEFAULT=$HOME/$HERE/$DATE-$HERE.mbox
CLONE=$HOME/$NOTHERE/$DATE-$HERE.mbox

LOGFILE=$HOME/received-reports.log
:0c:
$CLONE

:0:
$DEFAULT

For log rotation, we create a specific ~/.logrotaterc as follows:

/home/dmarc/received-reports.log {
 weekly
 rotate 4
 compress
 missingok
}

Finally, it is just a matter of adding a cronjob so the script looks for .mbox on daily basis (and to rotate logs), crontab -e :

# feed database
15 4 * * * cd ~/mxA/ && dmarcts-report-parser.pl -m *.mbox 2>/dev/null >/dev/null
# rotate logs
0 5 * * * /usr/sbin/logrotate /home/dmarc/.logrotaterc --state /home/dmarc/.logrotate.state

Replicating data

Now, we to set up some ssh access. Only on mxA:

ssh-keygen -t rsa
cat .ssh/id_rsa.pub

The output of the cat command just have to be copied on mxB in ~/.ssh/authorized_keys

Again on mxA, we set up the cronjob doing the actual copy (and removal after copy) with crontab -e

0 3 * * * rsync --quiet --ignore-missing-args --include '*.mbox' --exclude '*' -e "ssh -p 22" mxB.thisdomain:~/mxA/* ~/mxA/ && rsync --quiet --ignore-missing-args --include '*.mbox' --exclude '*' -e "ssh -p 22" ~/mxB/ mxB.thisdomain:~/mxB/* && rm -f ~/mxB/*.mbox && ssh -p 22 mxB.thisdomain 'rm -f ~/mxA/*.mbox'

 

Viewing reports

On and http server configure to run PHP files, install dmarcts-report-parser.

cd /srv
git clone https://github.com/techsneeze/dmarcts-report-viewer.git dmarc
cd dmarc
ln -s dmarcts-report-viewer.php index.php

Eye-candy wise, it is perfectible – but the data is there.