Using networked filesystems hosted by LXC containers with Samba

For more than a decade, I used NFS on my home server to share files. I did not consider using Samba for anything but to provide Windows access to shares. NFSv3 then NFSv4 suited me, allowing per host/IP write access policy. The only main drawback was very crude handling of NFS server downtime: X sessions would be half-frozen, requiring restart to be usable once again.

However, I moved recently my servers to LXC (which I’ll probably document a bit later) and NFS server on Debian, as you can guess from nfs-kernel-server package’s name, is kernel-based: not only it apparently defeats the purpose of LXC containers to actually have a server within a container tied to the kernel, but it does not seems to really work reliably. I managed to get it running, but it had to be run on both the master host and within the container. Even then, depending which started first could make the shares unavailable to hosts.

I checked a few articles over the web (https://superuser.com/questions/515080/alternative-to-nfs-or-better-configuration-instable-network-simple-to-set-up, http://serverfault.com/questions/372151/nas-performance-nfs-vs-samba-vs-glusterfs etc) and it looked that, as of today, you can expect decent performances from Samba, as much as of NFS. That could possibly be proven wrong if I was using massively NFS, writing a lot through networked file systems, opening a big number of files simultaneously, moving big files around a lot, but I have really simple requirements: no latency when browsing directories, no latency when playing 720p/1080p videos and that’s about it.

I had already a restricted write access directory per user, via Samba, but I use it only on lame systems as temporary area: on proper systems, I use SSH/scp/rsync/git to manipulate/save files.

Dropping NFS, I have now quite a simple setup, here are relevant parts of my /etc/samba/smb.conf:

[global]

## Browsing/Identification ###

# What naming service and in what order should we use to resolve host names
# to IP addresses
 name resolve order = lmhosts host wins bcast


#### Networking ####

# The specific set of interfaces / networks to bind to
# This can be either the interface name or an IP address/netmask;
# interface names are normally preferred
 interfaces = eth0

# Only bind to the named interfaces and/or networks; you must use the
# 'interfaces' option above to use this.
# It is recommended that you enable this feature if your Samba machine is
# not protected by a firewall or is a firewall itself. However, this
# option cannot handle dynamic or non-broadcast interfaces correctly.
 bind interfaces only = true


#### File names ####

# remove characters forbidden on Windows
mangled names = no

# charsets
dos charset = iso8859-15
unix charset = UTF8


####### Authentication #######

# "security = user" is always a good idea. This will require a Unix account
# in this server for every user accessing the server. See
# /usr/share/doc/samba-doc/htmldocs/Samba3-HOWTO/ServerType.html
# in the samba-doc package for details.
 security = user

# Private network
 hosts allow = 192.168.1.


# You may wish to use password encryption. See the section on
# 'encrypt passwords' in the smb.conf(5) manpage before enabling.
 encrypt passwords = true

# If you are using encrypted passwords, Samba will need to know what
# password database type you are using. 
 passdb backend = tdbsam

obey pam restrictions = yes

guest account = nobody
 invalid users = root bin daemon adm sync shutdown halt mail news uucp operator www-data sshd Debian-exim debian-transmission
 map to guest = bad user

# This boolean parameter controls whether Samba attempts to sync the Unix
# password with the SMB password when the encrypted SMB password in the
# passdb is changed.
 unix password sync = yes


#======================= Share Definitions =======================


realm = ...


[commun]
comment = Commun
path = /srv/common
browseable = yes
writable = yes
public = yes
guest ok = yes
valid users = @smbusers
force group = smbusers
create mode = 0660
directory mode = 0770
force create mode = 0660
force directory mode = 0770

[tmpthisuser]
comment = Données protégées
path = /srv/users/thisuser
browseable = yes
writable = yes
public = yes
valid users = thisuser
create mode = 0600
directory mode = 0700
force create mode = 0600
force directory mode = 0700
guest ok = no

 

I installed package libpam-smbpass and edited /etc/pam.d/samba as follow:

@include common-auth
@include common-account
@include common-session-noninteractive
@include common-password

For this setup to work, you need every user allowed to connect:

  • to be member of group smbusers – including nobody (or whatever the guest account is) ;
  • to have a unix password set ;
  • to be known to samba (smbpasswd -e thisuser or option -a).

If you are not interested in per user access restricted area, only nobody account will need to be taken care of.

And, obviously, files and directories ownership and modes must be set accordingly:

cd /srv/common
# (0770/drwxrwx---) GID : (nnnnn/smbusers)
find . -type d -print0 | xargs -0 chmod 770 -v
find . -type f -print0 | xargs -0 chmod 660 -v
cd /srv/users
# (0700/drwx------) UID : ( nnnn/ thisuser) GID : ( nnnn/ thisuser)
find . -type d -print0 | xargs -0 chmod 700 -v
find . -type f -print0 | xargs -0 chmod 600 -v
# main directories, in addition, need sticky bit some future directory get proper modes
chmod 2770 /srv/common/*
chmod 2700 /srv/users/*

To access this transparently over GNU/Linux systems, just add in /etc/fstab:

//servername/commun /mountpoint cifs guest,uid=nobody,gid=users,iocharset=utf8 0 0

This assumes that any users entitled to access files belongs to users group. If not, update accordingly.

With this setup, there is no longer any IP based specific write access set but, over years, I found out it was quite useless for my setup.

The only issue I have is with files with colon within  (“:”). Due to MS Windows limitations, CIFS list these files but access is made impossible. The easier fix I found was to actually rename these files (not a problem due to the nature of the files served) through a cronjob /etc/cron.hourly/uncolon :

#!/bin/bash
# a permanent cifs based fix would be welcomed
find "/srv" -name '*:*' -exec rename 's/://g' {} +

but I’d be interested in better options.

 

 

Switching from NFSv3 to NFSv4

Today, I switched over NFSv4. I guess there published it for some reason and people claim it could increase file transfert rate by 20%.

In my case, to get it working properly, I…

Modified /etc/default/nfs-kernel-server on server side to have

NEED_SVCGSSD=no

Modified /etc/default/nfs-common on both clients and server side to have

NEED_STATD=no
NEED_IDMAPD=yes
NEED_GSSD=no

Modified /etc/exports on server side to have something starting by


/server 192.168.1.1/24(ro,fsid=0,no_subtree_check,async)

/server/temp 192.168.1.1/24(rw,nohide,no_subtree_check,async,all_squash,anonuid=65534,anongid=65534)

[...]

It forces you to set a root for the NFS server, in my case /server (which I had already in my NFSv3 scenario, so…), aka fsid=0.
You also need to specify nohide for any exports.

Modified /etc/fstab on clients side to set mount type to nfs4 and to remove the /server part from the paths, no longer necessary as path are relatives to fsid=0 which is /server. It gives entries like:

[...]
gate:/temp /stockage/temp nfs4 nolock 0 0
[...]

I had an export which was a symlink to somewhere in /home. NFSv4 is stricter than NFSv3 and there is no way to export something outside from fsid=0. So I made a bind, adding to /etc/fstab on server side:

[...]
/home/torrent/watch /server/torrent/watch none bind 0 0
[...]

After restarting nfs-kernel-server on the server side and nfs-common on both sides, umount NFS partitions and doing a mount -a on the client side, everything seems fine.

Using partitions labels

Recent linux versions (yes, I’m talking kernel here – linux is not an operating system) introduce new IDE drivers. It implies a device naming convention change. Instead of hda, hdb, etc, you get sda, sdb, etc, just like SCSI drives.

I have three hard disks on my main workstation – plenty of partitions. So in my case, it makes sense to use a unique identifier for each partition so nothing breaks up whenever I add/remove a drive or boot on an older kernel with the previous IDE drivers.

There are already uniques ids for each partition available using the command blkid. It returns unbearables and meaningless, but very uniques, ids like af8485cf-de97-4daa-b3d9-d23aff685638.

So it is best, for me at least, to label partitions properly according to their content and physical disposition, which makes for uniques id too in the end.

For ext3 partitions, I just did:

e2label /dev/sda2 sg250debian64
e2label /dev/sda3 sg250home

For the swap, e2label cannot help, so we set the label with mkswap, recreating it:

swapoff /dev/sda1
mkswap -L sg250swap /dev/sda1
swapon -L sg250swap

For ntfs partitions, I did:

apt-get install ntfsprogs
ntfslabel /dev/sdb1 hi150suxor
ntfslabel /dev/sdb2 hi150suxor2

Then, /etc/fstab must be edited as:


LABEL=sg250swap none swap sw 0 0

LABEL=sg250debian64 / ext3 errors=remount-ro 0 1
LABEL=sg250home /home ext3 defaults 1 2

LABEL=hi150suxor /mnt/suxor ntfs-3g defaults,user,noauto 0 0
LABEL=hi150suxor2 /mnt/suxor2 ntfs-3g defaults,user,noauto 0 0

Finally, grub (or any other boot loader) config should be updated to reflect that. However, unless I’m mistaken, with grub2 as shipped by debian, everything is generated usings scripts that does not seem to handle labels 😦