FreeBSD Jails
This document describes the process I use when setting up a FreeBSD server that hosts more than a few services. For security and ease of maintenance, I install each type of "service" in its own jail. The cost of compartmentalizing services into jails is that common dependencies are installed in multiple jails (things like perl, sudo, gettext, etc). The advantage is that each virtual machine has only the required software installed. When you maintain the VM, it is much easier to manage the dependency relationships. I have installed dozens of FreeBSD servers in jails and this document embodies much of the experience I have gained.
My latest jail server is a Dell 2950 with 8GB of RAM and 2 quad-core Xeon CPUs. To get the best possible disk I/O, I have a pair of mirrored 150GB 3.5" 15k RPM SAS disks. I have a third 150GB disk installed for a near-line backup.
OS Installation
I started by installing FreeBSD amd64. Since my server is in a data center 1800 miles away, I used DRAC to virtually mount the ISO install image and remotely install the OS. I configured my disks as follows:
| partition | size | used |
|---|---|---|
| / | 1G | 45% |
| /usr | 8G | 32% |
| /var | 4G | 3% |
| /home | 115G |
The % numbers shown are based on a typical jail setup. In addition to a typical install, there is a 2nd backup kernel on /, and /usr has the ports and source trees, as well as a current world built in /usr/obj. The latter is required by ezjail for installing and upgrading jails.
Configuration
cat /etc/sysctl.conf
security.jail.allow_raw_sockets=1 # not having this will cause the system to be very sluggish during file I/O as well # as hanging during the nightly cron jobs vfs.ufs.dirhash_maxmem=16777216 kern.ipc.maxsockbuf=4262144 # http://www.freebsd.org/doc/en/books/handbook/configtuning-disk.html vfs.write_behind=0 # don't dump core files unless we specifically ask for them! kern.coredump=0
cat /boot/loader.conf
# raise some of the VFS buffer sizes and prevents system from hanging on disk I/O # there probably a better way to resolve this. kern.maxusers="512" autoboot_delay="3" beastie_disable="NO" # console redirection for DRAC & ILO SOL boot_multicons="YES" console="comconsole,vidconsole" comconsole_speed="57600" boot_serial="YES"
Applications
I installed the few applications that are needed in the host environment: openntpd, rsync, and sudo.
ntpd
Every machine should have run some type of clock synchronization tool. Whether it be ntpdate as a cron job, or the built-in ntpd, or openntpd. The built in ntpd works well but also insists on binding to every IP on the machine. In reality, I'd prefer that it bound to zero IPs and only acted as a network client. Preventing ntpd from binding to all IPs has been a requested feature of ntpd since 2006 and nothing has been accomplished. So I install openntpd instead. It does what I want by default (binds to zero ports). OpenNTPd also uses half the RAM (4.7 vs 10M) of NTPd.
ezjail
Installing ezjail is as simple as typing: make install clean in /usr/ports/sysutil/ezjail. Once installed, I save myself later effort by customizing the default flavour. Ezjail flavours are templates that customize the way new jails are created. Files in the template directory are copied into each new jail when it is created. My 'default' template has the following custom files:
jails:/home/jails/flavours/default # find .
. ./etc ./etc/make.conf ./etc/periodic.conf ./etc/resolv.conf ./etc/rc.conf ./etc/COPYRIGHT ./etc/CST6CDT ./usr ./usr/local ./usr/local/etc ./usr/local/etc/sudoers ./usr/home ./usr/home/matt ./usr/home/matt/.cshrc ./usr/home/matt/.ssh ./usr/home/matt/.ssh/authorized_keys ./usr/ports ./ezjail.flavour ./pkg ./root ./root/.cshrc
A quick browse through ezjail.flavour will help understand the process more fully.
Creating Jails
ezjail-admin create -f default mysql 127.0.0.2 ezjail-admin create -f default dns 127.0.0.3 ezjail-admin create -f default monitor 127.0.0.4 ezjail-admin create -f default nictool 127.0.0.5