Configuring Solaris as a Secure Router

To differentiate between a host and router, let's first define the functions of each. A host is typically a system with any number of interfaces that may or may not be connected on the same network. A host does not allow traffic to enter in one interface and out another. A typical server in a high-availability configuration has multiple interfaces, with each interface connected to a different network segment to prevent a single point of failure.

A router is a system with a minimum of two interfaces connected to at least two segments of different networks. The router allows traffic to reach its destination by entering one interface and passing out through another. An interface is loosely defined as a physical connection that allows other systems to communicate with the system via Ethernet, serial port, point-to-point link, or some other method. We will not get into a discussion of how the decision is made for the traffic to reach its destination; that issue is outside the scope of this chapter. A good reference on traffic routing and TCP/IP is TCP/IP Illustrated, Volume 1: The Protocols, by W. Richard Stevens.

Reasoning and Rationale

Let's attempt to answer the inevitable question, "Why use Solaris?" There are numerous platforms and designs available to use as a low-cost router, all of which are viable solutions. Some key factors in selecting one over others are the availability of hardware, the amount of time allotted to deploy the solution, and business needs. Solaris is a good choice as a router because it is stable, easily secured, and easily deployed.

Solaris is a stable operating system, capable of months and even years of continuous uptime without the need for rebooting. Solaris is scalable, able to run in both small and large environments on systems from workstation size to enterprise class. Additionally, Solaris is an easily secured operating system. The ability to shut down all services on the system, make configuration changes to a running kernel, and create multiple layers and access control on the system without bouncing the system make Solaris the perfect choice for a network with a 110-percent uptime requirement.

Finally, Solaris is easily deployed as a router or gateway. The stock install of Solaris provides connectivity between the divided portions of the same network or even different networks altogether by simply turning up the system with the interfaces configured to communicate with connected networks. This feature of simplicity is a double-edged sword. It can lead to easily deploying a system to perform the important job of routing network traffic. It can also result in an unintended and unrestricted path between two networks that could be exploited to leverage an attack.

Routing Conditions

In a default installation, Solaris routes traffic if a specific set of conditions are met:

• The system has at least two interfaces.

• The /etc directory contains at least two hostname.interface files that configure the interfaces when the system is booted.

• The stock /etc/r2.d/S69inet file exists.

The first condition refers to any two interfaces and is not limited by a difference in types of card, such as one 10Base card and one 100Base card, or types, such as Ethernet and point-to-point. The second condition refers to the hostname.interface file. For example, in the case of the previously mentioned cards, one hostname.le0 file and one hostname.hme0 file would represent the 10Mbit, or le0 interface, and the 100Mbit, or hme0 card.

The third condition refers to the init file executed when the system enters run-level 2. This is important because it's common for a systems administrator to make alterations to the S69inet script in an effort to secure the system. If a system is to be used as a router, it is recommended that an initial install be performed on the system to ensure the integrity of the machine. If such an install is not possible, it is at least recommended that these two scripts be replaced with known unaltered copies or those verified against the Solaris Fingerprint Database. We discuss using new installs and existing installs as routers later in this chapter.

When the system bootstraps, it first executes the script /etc/rcS.d/S30network.sh. This script is responsible for configuring the loopback and other interfaces. It brings up any interfaces on the system that have a hostname.interface file in /etc. When the system enters run-level 2, it executes the S69inet script, which handles all the necessary functions of bringing up network connectivity. This script is designed to bring up a system with multiple network configuration options. However, here we review only the details pertinent to our discussion.

The S30network.sh Script

To get a better understanding of how network connectivity is initialized on a system, let's first look at the S30network.sh script internals. The S30network.sh is an inode-level link to the master script network, located in /etc/init.d. This script is the preliminary phase of network initialization. It is executed early in the bootstrap process to service the needs of any other systems that might be relying on the host to provide some service. Diskless clients are an example of such hosts.

Let's look at some of the snippets of code from this script. Typically, a host that is going to act as a router does not rely on things such as DHCP or NIS, because these are added risks and could result in a higher probability of vulnerability exposure and potential compromise. This could allow an attacker to compromise the Achilles' heel of a network. Therefore, we examine the sections of code that are pertinent to our discussion. Suffice it to say that a router should not rely on other systems for anything.

In our first sample from the script, we have the following code on line 22:

/sbin/ifconfig lo0 plumb 127.0.0.1 up 2>&1 >/dev/null

This command is responsible for bringing up the loopback interface when the system bootstraps. This is the first initialization of an interface by the system. Shortly thereafter, we have this piece of code beginning on line 71:

interface_names="'echo /etc/hostname.*[0-9] 2>/dev/null'" if [ "$interface_names" != "/etc/hostname.*[0-9]" ]; then

This section of code begins the parsing of hostname.interface files, putting them into the $interface_names environment variable. The interface name is placed into environment variable $1 later, and on line 106 the interface is plumbed:

/sbin/ifconfig $1 plumb

Finally, when line 123 of the script is reached, the interfaces are configured by the following code:

/sbin/ifconfig $1 inet $ifcmds \     2>&1 >/dev/null

When this script completes, all network interfaces with a hostname.interface configuration file in the /etc directory are configured and ready for communication. The system completes its execution of the init scripts in the /etc/rcS.d directory and, under normal circumstances, continues to multiuser mode. The init program moves on to /etc/rc2.d and eventually executes S69inet, which begins the second phase of network initialization.

The S69inet Script

S69inet is executed by init when the system reaches run-level 2, or multiuser mode. S69inet is an inode link to master configuration file /etc/init.d/inetinit. At this stage, the necessary routing functions are configured, and if a machine is configured with two or more interfaces, the system begins routing traffic between interfaces. If this is the intended configuration, this can be a good thing. However, if the system's intention is to function as a multihomed host in a high-availability configuration, this configuration can have unexpected results.

To get a better understanding of why Solaris automatically routes traffic when two interfaces are present, let's look at some of the code in the S69inet script. We'll look only at the code pertinent to our discussion. On line 93, we have the following block:

if [ "$_INIT_NET_STRATEGY" = "dhcp" ] && [ -n "'/sbin/dhcpinfo Router'" ]; then         defrouters=`/sbin/dhcpinfo Router` elif [ -f /etc/defaultrouter ]; then         defrouters='/usr/bin/grep -v \^\# /etc/defaultrouter | \             /usr/bin/awk '{print $1}''         if [ -n "$defrouters" ]; then

This code first checks DHCP for routing information. If the system does not return routing information from the program dhcpinfo, it next checks for the existence of the file /etc/defaultrouter, which is used for static default route entries. The last line in the block checks the variable $defrouters for a nonzero value. If the variable length is greater than zero, some further checking of routing information is performed. If the check on the last line of the block yields a nonzero value, the system sets the default routes contained in /etc/defaultrouter on line 124. Otherwise, it flushes the routing table. If neither of the first two tests is true, the script sets the $defrouters variable to a null value.

The decision of whether to run the system as an IPv4 router is made on line 186. The script first checks for the existence of the /etc/notrouter file. Following this check, the script checks the configured interfaces to count the number that were configured via DHCP. The script then checks for a number of interfaces greater than two (loopback plus one interface) or if any point-to-point interfaces are configured. Finally, the script checks to see if the /etc/gateways file exists. If:

• The /etc/notrouter file does not exist, the number of interfaces configured by DHCP is equal to zero, and the number of interfaces configured, including the loopback device, is greater than two

• There are one or more point-to-point connections, or

• The /etc/gateways file exists

the script executes ndd to manipulate the IP kernel module and sets the ip_forwarding variable to 1. The script then launches in.routed and forces in.routed to supply routing information. The in.rdisc daemon is started next, launched in router mode. Otherwise, ip_forwarding is set to 0, in.rdisc is launched in solicitation mode to discover routers on the network, and in.routed is launched in quiet mode.

Configuring for Routing

A default installation of Solaris with more than two interfaces (including the loopback interface) that aren't configured by DHCP will route traffic by default. This process, of course, depends on the system having not been altered by administrative staff. In some situations, however, it might be impossible to reinstall an operating system on a machine that will be routing traffic. In this situation, we need to be able to configure the system to route traffic manually.

Let's walk through a check of an already configured and functioning system to ensure that it's ready to route traffic. First, we make a list of items to check and, if necessary, alter. We'll do this in step-by-step fashion, in order to pay due attention to detail and ensure that we don't miss a step that could result in failure of our objective. Following the step-by-step account, we briefly discuss each step and any possible caveats.

A Seven-Point Checklist

Here's our checklist:

1. Check for interfaces configured via DHCP.

2. Ensure that each interface to be configured has a corresponding hostname.interface file in /etc and that the contents of the files are valid.

3. Check the /etc/rcS.d/S30network.sh file (inode link to /etc/init.d/network) for signs of alteration.

4. Check the /etc/rc2.d/S69inet file (inode link to /etc/init.d/inetinit).

5. Check for the /etc/notrouter file, and if it exists, remove it.

6. After the system has booted, poll /dev/ip for the status of the ip_forwarding variable.

7. Test the system in an isolated environment to ensure traffic routing.

Each step is covered in more detail in the following sections.

Step 1: Check for Interfaces Configured via DHCP

In the first step, we verify that all the interfaces are being configured with static information. As previously mentioned, a system using interfaces configured by DHCP will not be configured as a router. The easiest way to check for this configuration is by using the ifconfig command on a running system and then examining the output. Using the all flag with ifconfig typically displays the pertinent information, as we see in Figure 7.1.

Figure 7.1: A hme0 Interface That Has Been Configured with DHCP

Step 2: Ensure Each Interface Has Corresponding File

Ensure that each interface to be configured has a corresponding hostname.interface file in /etc and that the contents of the files are valid. It is necessary to have a hostname.interface file for each interface to be configured when the system is bootstrapped. A nonexistent hostname.interface file will result in a nonexistent interface. Similarly, an incorrectly formatted hostname.interface file will result in an incorrectly configured interface.

For each interface to be configured by the system, create a hostname.interface file. For example, if there were two 100Mbit interfaces on a system, there would have to be a hostname.hme0 and hostname.hme1 file in the /etc directory. The device names can be discovered by reviewing the output of command, as we see in Figure 7.2.

Figure 7.2: Browsing dmesg for Interfaces Detected during Bootstrap

Ensure that the hostname.interface files contain one of two things: an IP address or a host name with an entry in the /etc/hosts file. In a standard configuration, the host name is placed in the hostname.interface file with an entry for the host name in the /etc/hosts file. When the system boots, it resolves this host name against the /etc/hosts file and configures the interface with the corresponding IP address. Although it's possible to place an IP address directly in the hostname.interface file, it is recommended that, for consistency, you follow the standard procedure. The file must contain either an IP address or a host name; it can't contain both.

Steps 3 and 4: Check the /etc/rcS.d/S30network.sh and /etc/rc2.d/S69inet Files

Check the /etc/rcS.d/S30network.sh file (inode link to /etc/init.d/network) for signs of alteration. In addition, check the /etc/rc2.d/S69inet file (inode link to /etc/init.d/inetinit). It is common practice for a systems administrator to alter boot scripts in order to create a more secure system. This practice can lead to problems for those who inherit such a system, however, because problems can occur that are not immediately traceable. Two scripts commonly modified are the /etc/rcS.d/S30network.sh and /etc/rc2.d/S69inet scripts.

Often, documents that discuss the hardening of systems instruct administrators to alter these files and change or comment out sections of code to create a more secure configuration. Some automated system-hardening tools alter these scripts as well. These scenarios can result in unpredictable behavior and abundant frustration when a system's mission and configuration change.

In the third and fourth steps, we verify the integrity of these two files. We can do this via one of three methods. The first method, and the most unreliable one, is to visually inspect the file for signs of alteration by using an editor and examining the change time of the file. The second and more reliable method is to compare the file against a known unaltered copy of the file. The third and most secure method is to compare the file md5 sum of the file against the known sum in the Sun Fingerprints Database. When in doubt, restore from the CD-ROM.

Step 5: Check for the /etc/notrouter File and, If It Exists, Remove It

Check for the /etc/notrouter file; if it exists, remove it. The /etc/notrouter file is used to keep the system from being configured as a router. A typical system on which this file will exist is a correctly configured multihomed host. This file is not created by default, nor is there a configuration option in the install process to create it. Therefore, a freshly installed system will not have this file and so this situation won't be a concern. However, you should manually check previously installed hosts. If this file exists, remove it.

Step 6: Poll /dev/ip for the Status of the ipforwarding Variable

After the system bootstrap, poll /dev/ip for the status of the ip_forwarding variable. The system will not route traffic if IP forwarding is not turned on. Therefore, after you've taken the previous configuration steps, reboot the system, and the ip_forwarding variable of the IP kernel module will be polled to ensure that the system is prepared to route traffic. The result is a Boolean. If the variable returns 1, the configuration was successful and the system is ready to route traffic. If the variable returns 0, there was an error somewhere in procedure and the system will not route traffic.

Step 7: Test the System

Test the system in an isolated environment to ensure traffic routing. In the final step, testing should be conducted to ensure that the system is functional. A private, isolated segment of network should be created to test the router's functionality and ensure proper configuration, reliability, and performance.

Security Optimization

A number of parameters associated with TCP/IP on a Solaris system can be modified to provide enhanced security. The configuration of ARP, IP, TCP, UDP, and ICMP in their default state might not provide the greatest level of security. For the sake of brevity, we don't delve deeply into this topic nor discuss it in brief. This would not do the topic justice. This topic has been covered comprehensively in a document, "Solaris Operating Environment Network Settings for Security," by Keith Watson and Alex Noordergraaf of Sun Microsystems Blueprints. Their documents are available from Sun Blueprints at www.sun.com/blueprints.

Security Implications

You don't have a hope of security or integrity for your network without first having a secure router. Therefore, the implementation of a system as a router must be secure by design. This consideration must be made at the very beginning of system design and observed diligently through deployment and afterward in maintenance. The intricacies of designing a secure router are covered in detail elsewhere in this book. Here we give some general guidelines to enhance security. From these guidelines, we'll repeat the minimalism mantra.

Minimal Installation

A secure router should include a minimal, functional installation of the operating system. However, this is more a management issue than a security issue. Simply put, smaller software installations make machines that are more easily managed and monitored for intrusion.

A system with a smaller installation is more easily managed because only the necessary pieces are in place. What constitutes necessary is the software to achieve your mission and business needs. A system with a minimal installation also removes a number of unnecessary services and makes it easier to monitor the system for intrusion.

There are two camps on the types of software that should be installed on a system. One side is against having a C compiler on the system; the other is for it. Neither side is right or wrong, but both have valid lines of reasoning to take into account.

The side against an accessible local C compiler fears a local user compiling exploits or other programs and using the system for unauthorized activities. Such violations could lead to a local user gaining elevated privileges or unauthorized network access. The other side of the argument believes that having a C compiler on the local system is a necessary utility. Without a C compiler, they believe, it's impossible to build programs from source.

We're happy to announce that we're proud members of both camps. We're against local users having unlimited free reign of a system through some goody built with a C compiler, but not against having the C compiler. This risk can be eliminated through proper permissions and access control such as RBAC or simple access control lists (ACLs).

Minimal Services

A router needs very little in terms of services. Since the system has one purpose, there isn't a necessity for things such as NFS, NIS, RPC, and sendmail. By eliminating these services, you enhance overall system performance.

Additionally, eliminating these services closes entry points for possible intruders. By limiting the channels that allow an intruder potential access to the system, we've mitigated the risk of opening a system to future compromise by a new vulnerability. Shutting down all services or using the system solely as a router isn't always possible. This is, however, the recommended practice.

Many of these services are started via the Internet daemon (inetd). Commenting out the services is a good practice. Commenting out the services and not starting inetd at all is the best methodology. The inetd is started in the /etc/rc2.d/S69inet script.

Another good practice is checking the rc directories in /etc for programs that might be started. For example, the rc3.d directory starts a number of services that, in addition to being unnecessary, also have a history of security risks. Services such as the NFS server and the DMI compatibility programs are started at run-level 3.

If you are interested we urge you to check out the articleBack to the Basics: Solaris and init available at www.securityfocus.com/infocus/1359. This document describes the services started on a stock install of Solaris and where they're started. Through the ps and netstat programs, it's possible to narrow down the majority of undesired services and disable them. If the use of these programs fails to yield the port number on which a particular service is running, the lsof utility can be a saving grace.

Minimal Users

A Solaris system is a multiuser system. However, a router should not be a multiuser system. Giving general users access to a system through which the traffic of the entire network flows is not only dangerous, it's reckless. Shell access to the router should be limited to administrative staff and strictly regulated. A router shouldn't bring unnecessary attention to itself by handling e-mail or other such services. It's unnecessary to state that the system is critical.

Minimal Dynamic Information

One feature that can turn into a problem on any network is dynamic information. Such information includes routing protocols, name services, and the like. These services are designed to make network management easier, but the design of such services often isn't the most secure.

A router should be limited in the amount of dynamic information on which it relies. Solaris routers typically start the in.routed and in.rdisc daemons when launching and gain routing information through UDP and ICMP. With any service that relies on dynamic data updates, it's possible to generate fictitious data and send it to the host, which could result in a denial-of-service (DoS) or other attack. Therefore, it is a best practice to eliminate all services on the router that rely on dynamic data, including in.rdisc and in.routed.

Minimal Cleartext Communication

One final note on minimalism: It is a best practice to communicate with this system using the minimal amount of cleartext possible. Although we can build the most armored host on Earth and surround it with armed guards, if we're communicating with the system via a channel that can be intercepted by a potential intruder, our efforts are in vain.

The best policy is to use one of the available implementations of the Secure Shell (SSH) protocol. If you want to add other means of communication and administration to the system, such as a Web-based configuration interface or perhaps a Web-based intrusion detection log analyzer, do so via a cryptographically secure channel. Any services that provide remote interactive communication are vulnerable to sniffing or connection hijacking. The only way to ensure communication integrity is via cryptography.

Unconfiguring Solaris Routing

We previously discussed the process of configuring Solaris as a router. We talked about some of the caveats involved with configuration and implementation. We also discussed the steps necessary to make Solaris function as a router from a default install as well as a previously implemented install.

In this section, we take a look at taking a Solaris router and returning it to host stage. As always, it's a best practice to do an initial install on a system before changing the system's purpose and mission. However, this isn't always an option. We discussed in a step-by-step scenario the process of changing an existing system to a router. In this section, we discuss in a step-by-step list of procedures the process of changing a system from a router to a multihomed host.

A Three-Point Checklist

Let's look at the steps necessary to ensure that the system isn't routing traffic. As we did previously, we create a step-by-step list of procedures to configure and check the system. We follow the list with a brief discussion of the steps:

1. Check for the /etc/notrouter file. If it does not exist, create it.

2. Check the value of ip_forwarding in the IP kernel module after the system has been rebooted.

3. Test the system by attempting to reach one interface of the system through the other.

Each step in this checklist is covered in further detail in the sections that follow.

Step 1: Check for the /etc/notrouter File

Check for the /etc/notrouter file. If it does not exist, create it. As previously mentioned, the system checks a number of things when booting and before making the determination that it will be a router. When /etc/rc2.d/S69inet executes, it tests for the existence of the /etc/notrouter file. If this file is not found, it acts as a router. However, if this file is found, it acts as a host. You can create this file by simply using the touch command.

Step 2: Check the Value of ipforwarding

Check the value of ip_forwarding in the IP kernel module after the system has been rebooted. After the /etc/notrouter file has been created and the system has been rebooted, check the ip_forwarding variable. As /etc/rc2.d/S69inet executes and discovers the notrouter file, the code that sets the ip_forwarding variable to 1 should not execute.

Step 3: Test the System

Test the system by attempting to reach one interface of the system through the other. The purpose of this test is to confirm that one interface on the system is not reachable via the other interface. In a typical multihomed host configuration, the system has at least two interfaces connected to different segments of network and incapable of communicating with one another without first sending traffic to a router. You can perform this test using one of any number of network debugging tools. One way to run the test is to use the source-routing functionality of the traceroute program.

In this example, we see that traceroute is executed on Solaris machine, and the traffic is directed at another Solaris machine with two interfaces. The –g flag specifies the IP to use as a gateway, which is the Solaris system with two interfaces. The end point is the other interface of the system. A successful configuration of a multihomed host results in the failure of this test.