3.1. General Concerns

Before grabbing a floppy or your BSD distribution on CD, there are a few decisions to be made that apply to both FreeBSD and OpenBSD installations.

3.1.1. What Are You Building?

The first and most important question you have to ask yourself before starting to build a system is, "What am I building?" A clear idea of the role this system will play in your environment is vital. The following system classes help define the role of your system.

3.1.1.1 Workstation

The BSD workstation is a daily use system usually with a list of installed packages at least two or three pages long. These systems almost always have the X Window System installed. They might not be upgraded or maintained other than by the end user and at her whim. At businesses, these systems are almost always found behind a firewall. These systems are generally out of your control after (or even before) system build, so user education is your best bet to keeping them secure. You may also want to consider an organization-wide "image" of FreeBSD or OpenBSD that has been preconfigured (locked down) and tested by security-minded system administrators.

3.1.1.2 Workgroup server

By definition, a server provides some kind of service to users. Servers can be shared development machines, exporters of filesystems, intranet development systems, etc. Workgroup servers generally interact with a variety of users and are not treated as carefully as infrastructure servers, though they do tend to be administered and upgraded by a system administrator. These are the familiar multiuser Unix systems. Rolling these systems into your patching and upgrade process for infrastructure servers is usually a good idea. These systems could exist on the same protected network as workstations, or better yet, on a separate network accessible to internal users.

3.1.1.3 Infrastructure server

As the title implies, these systems provide some kind of critical infrastructure service: name service (DNS), network time (NTP), dynamic address assignment (DHCP), email, external web presence, and so on. Infrastructure servers are often protected from both users on a local area network and users on the Internet by one or more firewalls. They are critical to the proper functioning of the network and have the bare minimum software installed. System administrators who are responsible for the specific services provided by a given server are often the only users granted access, and they perform patching, upgrades, and software installs.

In some cases, service operators may request access to the system for publishing content (i.e., in the case of an external web presence), or administering the particular service for which they're responsible. If possible, provide restricted access via one of the following means:

Automated synchronization

It may be possible to set up an internal repository for content or configuration and periodically synchronize this data with the external server. This may be achieved using tools like rsync/rdist, or by building an internal CVS repository and performing regular checkouts from the infrastructure server. For more information about secure file distribution techniques, see Chapter 4.

FTP

Running a chrooted FTP daemon on an interface only accessible from the internal network will provide an acceptable means for file transfer. Of course, FTP is a clear-text protocol and other options should be pursued first.

SCP only (SSH denied)

The scponly product is available from http://www.sublimation.org/scponly/ and will restrict access so that operators are only able to transfer files and not gain a shell.

Service-specific remote maintenance

Some services offer innate remote maintenance capabilities. BIND, a popular DNS server, may provide the requisite remote administration capability via the rndc utility. You could also create a stealth master nameserver to which configuration changes are made but that is not directly queried for data.

System jails

Ensure operators log into a jailed environment on the system that provides access to the files and directories.

Bear in mind that one of the main reasons service operators desire access to production infrastructure systems is to ensure that their changes do not produce any undesirable results. This end may also be achieved by providing service operators systems on the internal network or on a test network that mimic the behavior and support the identical configuration as a production counterpart.

3.1.1.4 Multipurpose system

When you must get the job done, but do not have the money to build 20 specialized servers, you need to combine functionality. Behold the multipurpose machine! An unfortunate product of budgetary constraints, this class of system can incorporate any combination of the aforementioned classes of systems. When a multipurpose machine fails, is subjected to an attack, or merely needs to be rebooted, you lose several services all at the same time. These machines can be made more secure with careful use of chrooting, jails, good monitoring, constant auditing, and patching. However the effort and painstaking care involved in maintaining a multipurpose machine is immense. These days, reasonable workstations capable of providing key services can be purchased for very little. Plan a migration process away from multipurpose systems by removing services one by one until the original system can be decommissioned or reassigned to a single task.

Be aware, however, that there are dangers in segregating all services onto separate machines. There are several criticisms you may face when presenting this option to your supervisors or peers, and we will address the most common concerns here.

More systems require more administration

Unless you can develop a standard system configuration and upgrade process, you may experience an unnecessarily high total cost of ownership (TCO) as a result of maintaining many specialized servers. Standardizing and automating will go a long way to making system maintenance easy and reducing TCO. Combining services onto one system can make sense as long as you retain the system class concept. Finally, remember to take into account the cost of failure of several critical services at once. As always, there is a tradeoff between security and convenience. Pick a solution that meets your security and usability requirements.

Cheap or used systems tend to have faulty hardware

Although it may sound trite, avoid buying cheap or used systems, or using them over the long term. The migration away from a multipurpose server may be a time-consuming process for budgetary reasons. If you feel you can find cheap or used hardware now that will last at least a year (presumably you will have additional funds at that time), it may be worth the risk. If desirable, build two machines to serve the same purpose instead of one either as a hot-failover system or a cold spare. The services you plan on deploying on the server and the state of your current multipurpose machine will help you decide when to take the plunge.

Names will have to change everywhere

It can be quite problematic to place a particular service (such as email, internal web, DNS, etc.) on a server with a new name. This is a valid complaint, but is also a problem that needs resolution anyway. Prepare for migrating services by creating alternate and more general namespaces for your services. Use canonical names (CNAME records) in DNS instead of hostnames in client and server configuration files. After the system configurations have been updated across the organization, service migration may begin.

This is by no means a comprehensive list of attacks against the multi-server model, but hopefully should provide you enough ammunition to convince management and your peers that the pain is worth the effort.

3.1.2. Media and Network

Once you have a good idea of the kind of system you are building, it is time to look at your media options for installation. Both operating systems offer a variety of media options, the most common being bootable floppy or CD and network installs. With all installs, there are two major security concerns: data integrity and host vulnerability.

In order to ensure the highest integrity of data, a CD install is safest. If the CD has been acquired from a reliable source (authorized vendor) you can be reasonably confident that the contents do not pose a security risk. In most cases you will want to use the latest release of the operating system available. When working with critical pieces of infrastructure, you would not be overly cautious to fall back a few releases.

With FreeBSD, significant changes occur at the turn of the major version number of the operating system (i.e., between 4.x and 5.x). In this case you might want to stick with the latest 4.x release until the new version has sufficiently matured. The tradeoff here is that you will need to plan a major release upgrade sooner, rather than later.

OpenBSD, on the other hand, releases on a strict calendar schedule that is somewhat independent of feature changes and technology. Thus, version numbers do not tell you what major changes you face when upgrading. For example, OpenBSD's i386 architecture switched its binary format from a.out to elf when it released Version 3.4. Upgrading by building from source was not possible from Version 3.3 to Version 3.4. Peruse the "What's New" section of the release announcement to see if there have been major introductions that you would rather avoid on vital systems.

3.1.2.1 To be networked or not to be networked

If you choose to have your network interface up during the install process, whether for the purposes of installing the operating system or merely because the interface happens to be plugged in, you may be exposing yourself to needless risk. If your system uses its network interface during the install process, you're placing implicit trust in the network to which your system is attached.

When your system needs to contact another on your network, it issues an arp(8) (address resolution protocol) request. A response is received with the hardware address of the target system. But what if a malicious system were to answer instead and provide you the wrong hardware address? Your system may then send subsequent requests to the malicious system instead. This kind attack is known as arp poisoning and facilitates a man-in-the-middle attack (MITM).

This may sound implausible, but a careless roaming user who brought in his wireless access point and attached it to the network may have unwittingly created a back door and facilitated this attack.

A MITM attack may also be possible if someone has compromised a DNS server for an organization you wish to contact, or your local DNS server. Looking up the IP address of ftpx.freebsd.org may return the wrong IP and you may start downloading binaries that seem legitimate but are actually riddled with Trojans.

Admittedly, there may be perfectly legitimate cases where a network install is both appropriate and desirable. If you have the infrastructure in place that provides a reasonably secure copy of the operating system distribution, you may want to use this local repository as your FTP source. If your network is isolated or you are well protected behind a firewall, the risks of remote host compromise are minimal. After all, as soon as your system is built, you're likely to perform an upgrade you'll need to place trust in your network at that point anyway.

Having a firewall does not mean you are safe. A firewall controls what traffic is allowed into and out of your organization. Firewalls can be poorly configured, or may allow legitimate traffic to a service that can be compromised with specially crafted (but valid) input. Other means of entry and exit are possible and a source of great concern to security administrators. Blindly trusting the security of your network is never sound planning.

One final thought: if the system you're building will eventually live outside of your innermost firewall, perhaps on your DMZ, or completely beyond your security perimeter, consider performing the installation and hardening on a more protected network. When the system has been locked down to your satisfaction, move it out to the perimeter network. A few tweaks to /etc/resolv.conf, /etc/hosts, and the rc.conf or rc.conf.local startup configuration file should be all you need to configure the system on the new network.

The Dangers of DHCP

If you do choose to build a server on a local network, you are given the option to automatically configure your network interface using DHCP. Although this may be convenient, it poses certain security risks and may interfere with subsequent configuration.

DHCP is without authentication or a sanity check. A network may contain any number of DHCP servers, and nothing says they have to be working in tandem. Even in a trusted networking environment, a user could bring in a wireless device with DHCP enabled providing a means for intruders to enter your protected network and for your system to be inadvertently placed on this insecure network during system build!

If you use DHCP during the install process and you are 100% confident that you have acquired your network configuration from the correct server, you may still run into problems later when you rebuild your kernel. We go into more detail when we cover kernel reconfiguration for FreeBSD later in this chapter.

For these reasons, we recommend you choose a static address when building servers. Workstations generally remain configured with DHCP and on the network on which they were built.

3.1.2.2 Media verification

Ensure your media was acquired from a reliable source. Retail, shrink-wrapped software is generally safest, and moderately stale local copies of FTP repositories are the next best bet. It is not often you hear about an FTP distribution server being compromised, but it is not unheard of. Consider using tools like md5(1) against your downloaded CD images to verify that the hashes match those provided as reference on trusted FTP distribution servers, as shown in Example 3-1.

Example 3-1. Verifying the md5 checksum of a FreeBSD .iso

% fetch http://ftp2.freebsd.org/pub/FreeBSD/ISO-IMAGES-i386/5.3/CHECKSUM.MD5 CHECKSUM.MD5                                  100% of  278  B  366 kBps % fetch http://ftp2.freebsd.org/pub/FreeBSD/ISO-IMAGES-i386/5.3/5.3-RELEASE-i386-disc1.iso 5.3-RELEASE-i386-disc1.iso                    100% of  644 MB  315 kBps % cat CHECKSUM.MD5 MD5 (5.3-RELEASE-i386-bootonly.iso) = e370ae39bb34f0789c638b6ad50038a2 MD5 (5.3-RELEASE-i386-disc1.iso) = fbcbfdff31f27de396f257e0a37a78b8 MD5 (5.3-RELEASE-i386-disc2.iso) = 21874a5663022768336e4cc73d1dd30d MD5 (5.3-RELEASE-i386-miniinst.iso) = 96124b2608ba481693e04d364d485e3c % md5 5.3-RELEASE-i386-disc1.iso MD5 (5.3-RELEASE-i386-disc1.iso) = fbcbfdff31f27de396f257e0a37a78b8

3.1.3. Preexisting Vulnerabilities

Choosing your installation media is a decision that should never be taken lightly. Even before installing FreeBSD or OpenBSD on a system, you should check out the appropriate web page to determine whether you're about to install software with known vulnerabilities. FreeBSD lists security advisories chronologically on their security page at http://www.freebsd.org/security/, and you'll find analogous information for OpenBSD on the Errata page at http://www.openbsd.org/errata.html.

FreeBSD and OpenBSD release versions of the operating system on a semiregular basis. Just because you received a CD of OpenBSD or FreeBSD in the mail a week ago does not mean that you are about to install the "latest version" of the operating system. Any bugs or security issues discovered shortly after release have likely been announced on OpenBSD security-announce or FreeBSD security-notifications and been addressed in the source tree.

If you are installing an even older distribution, you can almost guarantee that upon installation there will be vulnerabilities (possibly remotely exploitable) in the system. Avoid installing particularly old software if you can. Be very conscious of your network connectivity and potential exposure if you must install a dated distribution.

The Whats and Whys of CVSup

CVS (Concurrent Versions System), is a software source code version control system. Developers use CVS and other version control systems to track changes as they work with source code. The entire FreeBSD and OpenBSD source trees are housed in CVS repositories so that when developers check out a file, modify it, and check it back in, changes to the file are recorded. This later facilitates looking at change history or performing rollbacks to previous versions of a given file.

Getting the latest sources for your FreeBSD operating system is most often achieved by using CVSup. This is a highly optimized network distribution package for CVS repositories. If you already have an "old checkout" of a CVS repository, running cvsup(1) only transfers data if a file has changed, and then only the changes to the file, not the entire file.

cvs(1) is used by OpenBSD to check out sources just as a developer would need to in order to modify code. Direct access to this facility allows changes made locally to be merged in with changes made to the code base by the OpenBSD development team. Likewise, the command cvs update -dPA performs a file update similar to cvsup.

After installing a FreeBSD or OpenBSD system, you will need to update it. Both installation procedures below describe the steps involved in performing a system upgrade. Bear in mind that the steps involved (cvsup, make, installation) are all fairly time consuming.

The cvsup or cvs update procedure will download all the source code for your operating system. If you installed from CD, it's beneficial to install the CD's copy of the sources so that you only have to download updates. Either way, the sources should end up in /usr/src.

After you have the source, you will need to run some form of make, which prepares new binaries for your system based on the updated code base. When this is done, you may begin the install step, which does exactly what you think it does. If you have not been through a system upgrade before, set aside plenty of time. On a fast (1+ Ghz) system, you may need an hour or more. On a slower system (300Mhz), the make step can easily consume 6 hours.

For more information about cvsup(1) and cvs(1), read the manpages for each.

One of the first steps you must take after successfully installing your system is to retrieve/update your operating system's source tree (usually using cvsup(1) or cvs(1)) and upgrade or patch the operating system until all known issues have been addressed. Immediately following the installation, you won't know if the running services have known security issues, unless you've consulted the appropriate resources.

Knowing about the existence (or absence) of security issues will help you decide whether you should bring your network interface up during the install process. You may discover that you need to wait until after the install and after you have turned off some or all of your listening services before connecting the system to your network.

Performing a CVS update procedure may open the door to MITM attacks and Trojaned code. While software systems have been built to automatically download file signatures or hashes with files with the intention of guaranteeing data integrity, a good cracker will replace both these server-provided hashes and corresponding files making these automatic solutions ineffective. The only means to realistically increase confidence that the file you download is legitimate is to perform out-of-band tests. These might include:

• Visiting the vendor's primary web site and comparing the hash there to the hashes on all the mirror sites, especially the site from which you obtained the file. They should all be the same.

• Confirming with other administrators or software providers on mailing lists that the hash is a valid one.

• Downloading a file to a local repository once and waiting a month or so to see if anyone has identified rogue data within that timeframe before performing an upgrade.

Of course, few administrators have the time necessary to follow any of these steps. In this case, functionality requirements may outweigh security concerns. Be cautious.

3.1.4. Slicing Up Your Filesystem

It may seem out of place to be reading about filesystem slicing in a chapter about a secure installation and hardening. However, availability is inexorably tied to security, and proper filesystem portioning can affect data and service availability, so we must address this issue.

Your first step will be determining whether your system will contain anything besides BSD during the fdisk(8) table setup. Servers from vendors such as Dell and Compaq ship with maintenance or utility partitions that provide disk management tools and hardware test suites. You will probably want to keep these handy in the event of a system malfunction. For an infrastructure server, dual-booting with another operating system should be avoided.

If you are not already familiar with what kinds of data are stored in which directories, spend a moment reading the hier(7) manpage. You will find a complete description of the filesystem hierarchy and the kind of files you are liable to find in any given directory.

After fdisk, define your partitions. Laying out your filesystems properly from the start will save you grief down the road. It's almost always a good idea to have separate filesystems for /, /tmp, /var, /usr, and perhaps /home, as shown in Figure 3-1. You may also want others specific to your situation.

Figure 3-1. A possible disk layout of distinct filesystems

There are four major motivations for not simply creating a large root volume.

Integrity

A system that experiences a power loss runs the risk of losing or corrupting data. This may limit itself to a few files, a directory, or the entire filesystem. The branches of the file hierarchy store different kinds of data as described in the manpage for hier(7). The /home, /usr, /tmp, and /var filesystems, for instance, are fairly dynamic to varying degrees and experience many frequent changes. The root filesystem (excluding the others) tends to be fairly static. The chance of corruption in the event of a power or system failure will relate directly to the filesystem's volatility.

Availability

You can't divorce availability from security. Running out of space in the root filesystem will interrupt service. When done maliciously, it is a form of denial of service attack. The likelihood of overextending available space on the root filesystem is significantly reduced when frequently written-to filesystems (such as /tmp and /var) are separated from the root. A rogue application or user will only be able to fill filesystems on which write access is granted.

Security

Mount options such as disabling the interpretation of character or block special devices, disabling set-user-identifier (a.k.a. set-user-id and setuid) or set-group-identifier (a.k.a. set-group-id and setgid) bits, disallowing execution of any binaries altogether, and setting a filesystem as read-only are all possible when separate filesystems are defined. With a monolithic filesystem, none of these options could be applied. On the other hand, slicing out a partition such as /var/log on a centralized logging host would enable you to apply all these options to the filesystem.

Performance

On a system with a great deal of disk activity, it can be useful to isolate frequently written-to filesystems near the perimeter of the drive platters to isolate them from more frequently read filesystems. Incorporating these directories as part of a larger filesystem places no constraints on the physical location of data stored within in this case, fragmentation may cause a marked decrease in performance.

Arguments against filesystem slicing assert that a system will not behave as expected when /tmp and /var are filled and therefore there is little value in slicing out those filesystems. While this is true, your obligation is to mitigate these risks. Consider putting subdirectories of these filesystems on different partitions (such as /var/mail or /var/spool for instance) so that an attack that disrupts a service by filling a filesystem will not make the server itself nonfunctional. Bear in mind that even if users are able to "fill" these filesystems, the BSD operating systems reserve space for the root user, thus it is not uncommon to see "full" filesystems over 100% capacity. To configure how much space is reserved, look at the -m command-line argument to tunefs(8). Avoid changing this value unless you really know what you are doing.

Take special care with the /tmp and /var filesystems. These are frequently written to by non-root processes and are more liable to fill. The multipurpose /var directory is especially important as it not only stores critical system logs, name server configuration, and system database files, but also your print spool and mail queue. These data may constantly vie for space.

FreeBSD 4.4 and OpenBSD 3.4 introduced growfs(8). With this utility and the availability of cheap hard drives, running out of space on a filesystem is not an issue. It is now common for system administrators to leave the majority of a disk unallocated so that space is available when a filesystem needs to be grown.

In the event that you have multiple disks at your disposal and wish to use them in a non-RAID configuration, the install processes of both operating systems allow you to mount your partitions from both drives. In this case, you may also want to create swap partitions on both drives. The following excerpt from the FreeBSD Handbook provides the rationale.

On larger systems with multiple SCSI disks (or multiple IDE disks operating on different controllers), it is recommend that a swap is configured on each drive (up to four drives). The swap partitions should be approximately the same size. The kernel can handle arbitrary sizes but internal data structures scale to 4 times the largest swap partition. Keeping the swap partitions near the same size will allow the kernel to optimally stripe swap space across disks. Large swap sizes are fine, even if swap is not used much. It might be easier to recover from a runaway program before being forced to reboot.

When you have finished deciding how to slice up your disk, you may end up with a layout similar to the one in Figure 3-1. In this case, a utility system existed on the first slice of the disk, which was maintained. The rest of the disk was devoted to the operating system. Within this second slice separate partitions were made for each of the filesystems mentioned previously.

3.1.5. XFree86

The X Window System is rarely found installed on servers that house critical services. X is an extremely large and complex collection of binaries and libraries that introduces a huge code base from which vulnerabilities emerge from time to time. XFree86 is an open source implementation of the X Window System that is distributed with a variety of contemporary open source operating systems including Linux and the BSDs.

The X Window System is based upon the X protocol that takes the client-server model up to the application level. The goal of this protocol is to provide a graphical user interface within and between systems transparently. While this design improves look and feel, portability, and performance, all the security issues of the client-server model add to the already long list of application vulnerabilities.

That said, the XFree86 development team has done a phenomenal job over the years improving the security architecture of their implementation of the X Window System and it is today far safer to run X on servers than it ever was before. However, the goal of X is and always has been to provide a convenient graphical user interface.

Infrastructure servers are typically remotely accessed machines and provide only very specific services over clearly defined protocols. The only "use" of the server is by the administrator, and administration is usually console based.

You might think of workgroup servers as systems that need X. Sure enough, they may need to support the execution of X applications. What is referred to as the "X Server," however, allows locally executed applications to appear on the server's display. This means local memory allocated to the video card adapter and the PCI bus needs to be accessed. When direct memory access is being provided to an application, a vector for attack is introduced. The bottom line: the X Server rarely needs to be installed on workgroup servers.

Given the security ramifications of installing the entire X distribution, we recommend against doing so for workgroup and infrastructure servers. As needs arise for users to run X applications, these applications may be installed as packages or through ports. In the latter case, dependant packages will be automatically retrieved and installed. This assures you that only the software required to make a given application function is installed, but no more.

3.1.6. Users and Passwords

At the end of the installation, you're prompted for a password for the root user. In the case of FreeBSD, you are also prompted to create an additional user. Do it. This will be your user account, and could be added to the "wheel" group if you will need to su(1) to root. For OpenBSD, create an additional user account after installing the OS.

It's very important that you create an account for yourself instead of running around the system as the root user. Operating as a nonprivileged user protects you from you. Poorly thought out commands or even typographical errors as root can send you scrambling for backup tapes. Even seasoned administrators can get a little spacebar happy and turn the command rm -rf /my/dir/prefix* to rm -rf /my/dir/prefix *. If your current directory happens to be /usr, by the time you press Ctrl-C, all manner of vital system files may be gone.

The two typical ways to gain super-user privileges are su and sudo(8). The former is available in the base installation of both operating systems, and the latter is included in the base install of OpenBSD and as a port/package in FreeBSD. When a user is a member of the wheel group, that user can use su to become root. However, doing so introduces an enticement: now that the user is root, why spend extra keystrokes to drop privileges again? The sudo command solves this problem by running only a single command at a time as root and logging the command as it runs it. You are immediately returned to your nonprivileged shell after the command has executed.

For more information about the differences between su and sudo, and some of the motivations for and pitfalls of each, see Chapter 4.

When creating a user account, take the time to develop a strong password both for root and any users you create as described in Chapter 1.

3.1.7. Summary

At this point you should know what you are building and where it is going in your network. You are armed with the installation media (and method) you have chosen and have thought about the risks associated with those decisions. Without further ado, let's dig into the not-so-glorious installation processes for FreeBSD and OpenBSD.

If you will be installing FreeBSD, keep reading. If you want to follow the weapon laden puffer fish, skip ahead to the section entitled Section 3.4 just ahead!