Jailing UML Instances
| The centerpiece of any layered security design for a large UML server is the jail that the UML instances are confined to. Even though UML confines its users, it is prudent to assume that, at some point, someone will find a hole through which they can escape onto the host. No such holes are known, but it's impossible to prove that they don't exist and, if one did exist, that it couldn't be exploited. This jail will make use of the Linux chroot system call, which confines a process to a specific directory. You can see the effect of using the chroot command, which is a wrapper around the system call, to confine a shell to /sbin. host# chroot /sbin ./sash Stand-alone shell (version 3.7) > -pwd / > -ls . .. MAKEDEV Notice how the current directory is /, but its contents are those of /sbin. chroot arranges that the directory specified as the jail becomes the new process's root directory. The process can do anything it normally has permissions to do within that directory tree but can't access anything outside it. This fact forced the choice of sash as the shell to run within the chroot. Most other shells are dynamically loaded, so they need libraries from /lib and /usr/lib in order to run. When jailed, they can no longer access those librarieseven if the libraries are within the jail, they will be in the wrong location for the dynamic loader to find, even when the dynamic loader itself can be found. So, for demo purposes, the easiest way to show how chroot works is by running a statically linked shell within its own directory. More serious purposes require constructing a complete but minimal environment, which we will do now. This environment must contain everything that the jailed process will need, but nothing else. We will construct a jail that is surprisingly empty. This provides as few tools as possible to an attacker who somehow manages to break out of a UML instance. He or she will want to continue the attack in order to subvert the host. In order to do this, the attacker will need to break out of the chroot environment. If there is a vulnerability (and I am aware of no current holes in chroot), the attacker will need tools in order to exploit it. Making the chroot environment as empty as it can be will go some way toward denying him or her these tools. First we must decide what a UML instance needs in order to simply boot. Looking at the shared libraries that UML links against and a typical UML command line gives us a start: host% ldd linux linux-gate.so.1 => (0x003ca000) libutil.so.1 => /lib/libutil.so.1 (0x00c87000) libc.so.6 => /lib/libc.so.6 (0x0020a000) /lib/ld-linux.so.2 (0x001ec000) host% ./linux con0=fd:0,fd:1 con1=none con=pts ssl=pts \ umid=debian mem=450M ubda=../../debian30 devfs=nomount With this UML binary, those libraries would need to be present within. /lib and within the jail in order to even launch it. After launching, the command line makes a number of other requirements in order for UML to boot:
This would be far from being an empty directory, and it would contain files such as libraries and device nodes that an attacker might find useful. Fortunately, these requirements can be reduced in some fairly simple ways. First, to eliminate the requirement for libraries, we can make the UML executable statically, rather than dynamically, linked. If CONFIG_MODE_TT is enabled, UML is linked statically. However, for a serious server, it is highly recommended that the UML instances use either skas0, if the server is running an unmodified kernel, or skas3, if the skas3 patch can be applied to the host kernel. With CONFIG_MODE_TT disabled, UML will link dynamically. However desirable this is in general, it complicates setting up a jail. So, a configuration option for UML, CONFIG_STATIC_LINK, forces the UML build to produce a statically linked executable, even when CONFIG_MODE_TT is disabled. Enabling CONFIG_STATIC_LINK results in a larger UML binary, which is slightly less efficient for the host because the UML instances are no longer sharing library code that they would share if they were linked dynamically. This is unavoidableeven if you copied the necessary libraries into the jail, each UML instance would have its own copy of them, so there would still be no sharing. There is a clever way to have the libraries be present in each jail but still shared with each otheruse the mount --bind capability described earlier to mount the necessary libraries into the jails. However, this is too cleverit opens up a possible security hole. If an attacker were somehow able to break out of a UML instance, gain access to the jail contents, and modify the libraries, those libraries would be modified for the entire system. So, if the attacker could add code to libc, at some point that code would be executed by a root-owned process, and the host would be subverted. So, for security reasons, we need no shared code between the UML instance and anything else on the system. Once we have made that decision, there is no further cost to statically linking the UML binary. The next issue is the /dev/pts requirements imposed by the console and serial line arguments. These are easy to dispose of by changing those configurations to ones that require no files in the jail. We have a variety of possibilitiesnull, port, fd, and none all fill the bill. null and none effectively make the consoles and serial lines unusable. port and fd make them usable from the host outside the jail. For an fd configuration, you would have to open the necessary file descriptors and pass them to the UML instance on its command line. Finally, there is the umid directory. We can't eliminate it without losing the ability to control the instance, but we can specify that it be put someplace other than the user's home directory within the jail. By creating a ./tmp directory within the jail and using the uml_dir switch to UML, we can arrange for the pid file and mconsole socket to be put there. At this point, the jail contents look like this: host% ls -Rl .: total 1033664 -rw-rw-r-- 1 jdike jdike 1074790400 Aug 18 17:46 debian30 -rwxrwxr-x 1 jdike jdike 20860108 Aug 18 17:39 linux drwxrwxr-x 2 jdike jdike 4096 Aug 18 17:46 tmp ./tmp: total 0 As a quick test of whether UML can boot in this environment and of its new command-line switches, we can do the following as root in the jail directory: host# chroot. ./linux con0=fd:0,fd:1 con1=none con=port:9000 \ ssl=port:9000 umid=debian mem=450M ubda=debian30 \ devfs=nomount uml_dir=tmp It does boot, printing out a couple messages we haven't seen before: /proc/cpuinfo not available - skipping CPU capability checks No pseudo-terminals available - skipping pty SIGIO check Because /proc and /dev are not available inside the jail, UML couldn't perform some of its normal checks of the host's capabilities. These are harmless, as the /proc/cpuinfo checks come into play only on old processors, and the pseudo-terminal test is necessary only when attaching consoles to host pseudo-terminals, which we are not doing. Running UML in this way is useful as a test, but we ran UML as root, which is very much not recommended. Running UML as a normal, nonprivileged user is one of the layers of protection the host has, and running UML as root throws that away. Root privileges are needed in order to enter the chroot environment, so we need a way to drop them before running UML. It is tempting to try something like this: host# chroot jail su 1000 ./linux ... However, this won't work because the su binary must be present inside the jail, which is undesirable. So, we need something like the following small C program, which does the chroot, changes its uid to one we provide on the command line, and executes the remainder of its command line: #include <stdio.h> #include <errno.h> #include <stdlib.h> int main(int argc, char **argv) { int uid; char *dir, **command, *end; if(argc < 3){ fprintf(stderr, "Usage - do-chroot dir uid \ command-line...\n"); exit(1); } dir = argv[1]; uid = strtoul(argv[2], &end, 10); if(*end != '\0'){ fprintf(stderr, "the uid \"%s\" isn't a number\n", \ argv[2]); exit(1); } command = &argv[3]; if(chdir(dir) < 0){ perror("chroot"); exit(1); } if(chroot(".") < 0){ perror("chroot"); exit(1); } if(setuid(uid) < 0){ perror("setuid"); exit(1); } execv(command[0], command); perror("execv"); exit(1); }This is run as follows: host# do-chroot jail 1000 ./linux con0=fd:0,fd:1 con1=none \ con=port:9000 ssl=port:9000 umid=debian mem=450M \ ubda=debian30 devfs=nomount uml_dir=tmp Since I am specifying a nonexistent uid, everything in the jail should be owned by that user in order to prevent permission problems: host# chown -R 1000.1000 jail Now UML runs as we would like. It is owned by a nonexistent user, so it has even fewer permissions on the host than something run by a normal user. We saw the contents of the jail directory as we have it set up. With the UML instance running, there are a couple more things in it: host% ls -Rl .: total 1033664 -rw-rw-r-- 1 1000 1000 1074790400 Aug 18 19:12 debian30 -rwxrwxr-x 1 1000 1000 20860108 Aug 18 17:39 linux drwxrwxr-x 3 1000 1000 4096 Aug 18 19:12 tmp ./tmp: total 4 drwxr-xr-x 2 1000 root 4096 Aug 18 19:12 debian ./tmp/debian: total 4 srwxr-xr-x 1 1000 root 0 Aug 18 19:12 mconsole -rw-r--r-- 1 1000 root 5 Aug 18 19:12 pid We also have the mconsole socket and the pid file in the tmp directory. This is reasonably minimal, but we can do better. Some files are opened and never closed. In these cases, we can remove the files after we know that the UML instance has opened them. The instance will be able to access the file through the open file descriptor and won't need the file to actually exist within its jail. Chief among these are the UML binary and the filesystem. We can remove them after we are sure that UML is running and has opened its filesystem. It is tempting to remove them immediately after executing UML, but that is somewhat prone to failure because the removals might run before the UML instance has run or before it has opened its filesystem. To avoid this, we can use the MConsole notify mechanism we saw in Chapter 8. We'll use a slightly modified version of the Perl script used in that chapter to read notifications from a UML instance: use UML::MConsole; use Socket; use strict; @ARGV < 2 and die "Usage : running.pl notify-socket uid"; my $sock = $ARGV[0]; my $uid = $ARGV[1]; !defined(socket(SOCK, AF_UNIX, SOCK_DGRAM, 0)) and die "socket failed : $!\n"; !defined(bind(\*SOCK, sockaddr_un($sock))) and die "UML::new - bind failed : $!\n"; chown $uid, $uid, $sock || die "chown failed - $!"; my ($type, $data) = UML::MConsole->read_notify(\*SOCK, undef); $type ne "socket" and die "Expected socket notification, got \"$sock\" " . "notification with data \"$data\""; exit 0; Running this as root like this: host# perl running.pl tmp/notify 1000 and adding the following: mconsole=notify:tmp/notify to the UML command line will cause the running.pl script to exit when the instance announces that it has booted sufficiently to respond to MConsole requests. At this point, the UML instance is clearly running and has the root filesystem open, so the UML binary and the filesystem can be safely removed. Under tmp, there is the MConsole socket and pid file. The pid file is for management convenience, so it can be read and removed. The MConsole socket can be moved outside the jail, where it's inaccessible to anyone who somehow manages to break out of the UML instance, but where an MConsole client can access it. The only thing that can't be removed is the notify socket, which has to stay where it is so that the UML instance can send notifications to it. If that socket is removed, you lose an element of control since you can't find out if the instance has crashed. If this is OK, you can remove the socket, and the UML instance will run in a completely empty jail. One thing we haven't done here is to provide the UML instance with a swap device. Like the root filesystem, the swap device file needs to be in the jail. If it's removed, it can possibly be lost by the UML instance. If swapoff is run inside the instance, the block driver will close the swap device file. When this happens, the instance will lose the only handle it had to the file. If swapon is subsequently run, the block driver will attempt to open the file, and fail, since you removed it. This is not a problem for the root filesystem since, once mounted, it is never unmounted until the instance is shut down. One side effect of removing the UML binary is that reboot will stop working. Rebooting is implemented by exec-ing the binary to get a clean start for the new instance. If the binary has been removed, exec will fail. However, this is probably not a big problem since a reboot is no different from a shutdown followed by a restart. You need to be careful with the root filesystem. If you simply copy it into the jail, boot the UML instance, and remove the filesystem file, the instance will have access to the filesystem as long as it keeps the file open. When it shuts down, it will close the file, and it will be removed, along with whatever changes were made to it. To prevent this, you should keep the filesystem out of the jail and make a hard link to it from inside the jail. Now there will remain one reference to the filethe original name for itand it will not be removed when the instance closes it. |
