TUN/TAP with RoutingIn earlier chapters we briefly looked at how to put a UML on the network. Now we will go into this area in some depth. The most involved part is setting up the host when the UML will be given access to the physical network. The host is responsible for transmitting packets between the network and the UML, so correct setup is essential for a working UML network. There are two different methods for configuring the host to allow a UML access to the outside world: routing packets to and from the UML and bridging the host side of the virtual interface to the physical Ethernet device. First we will use the former method, which is more complicated. We will start with a completely unconfigured host, on which a UML will fail to get access to the network, and, step by step, we'll debug it until the UML is a fully functional network node. This will provide a good understanding of exactly what needs to be done to the host and will allow you to adapt it to your own needs. The step-by-step debugging will also show you how to debug connectivity problems that you encounter on your own. Later in this chapter, we will cover the second method, bridging. It is simpler but has disadvantages and pitfalls of its own. Configuring a TUN/TAP DeviceWe are going to use the same host mechanism, TUN/TAP, as before. Since we are doing this entirely by hand, we need to provide a TUN/TAP device for the UML to attach to. This is done with the tunctl utility: host% tunctl Failed to open '/dev/net/tun' : Permission denied This is the first of many roadblocks we will encounter and overcome. In this case, we can't manipulate TUN/TAP devices as a normal user because the permissions on the control device are too restrictive: host% ls -l /dev/net/tun crw------- 1 root root 10, 200 Jul 30 07:36 /dev/net/tun I am going to do something that's a bit risky from a security standpointchange the permissions to allow any user to create TUN/TAP devices: host# chmod 666 /dev/net/tun When the TUN/TAP control device is open like this, any user can create an interface and use it to inject arbitrary packets into the host networking system. This sounds nasty, but the actual practicality of an attack is doubtful. When you can construct packets and get the host to route them, you can do things like fake name server, DHCP, or Web responses to client requests. You could also take over an existing connection by faking packets from one of the parties. However, faking a server response requires knowing there was a request from a client and what its contents were. This is difficult because you have set yourself up to create packets, not receive them. Receiving packets still requires help from root. Faking a server response without knowing whether there was an appropriate request requires guessing and spraying responses out to the network, hoping that some host has just sent a matching request and will be faked out by the response. If successful, such an attack could persuade a DHCP client to use a name server of your choice. With a maliciously configured name server, this would allow the attacker to see essentially all of the client's subsequent network traffic since nearly all transactions start with a name lookup. Another possibility is to fake a name server response. If successful, this would allow the attacker to intercept the resulting connection, with the possibility of seeing sensitive data if the intercepted connection is to a bank Web site or something similar. However, opening up /dev/net/tun as I have just done would require that such an attack be done blind, without being able to see any incoming packets. So, attacks on clients must be done randomly, which would require very high amounts of traffic for even a remote chance of success. Attacks on existing connections must similarly be done blind, with the added complication that the attack must correctly guess a random TCP sequence number. So, normally, a successful attack would be remote. However, you should take this possibility seriously. The permissions on /dev/net/ tun are a layer of protection against this sort of attack, and removing it increases the possibility of being attacked using an unrelated vulnerability. For example, if there was an exploit that allowed an attacker to sniff the network, the arguments I just made about how unlikely a successful attack would be go right out the window. Attacks would no longer be blind, and the attacker could see DHCP and name requests and try to respond to them through a TUN/TAP device, with good chances of success. In this case, the /dev/net/tun permissions would have likely stopped the attacker. So, before opening up /dev/net/tun, consider whether you have untrusted, and possibly malicious, users on the host and whether you think there is any possibility of holes that would allow outsiders to gain shell access to the host. If that is remotely possible, you may consider a better option, which is used by Debiancreate a uml-users group and make /dev/net/tun accessible only to members of that group. This reduces the number of accounts that could possibly be used to attack your network. It doesn't eliminate the risk, as one of those users could be malicious, or an outsider could gain access to one of those accounts. However you have decided to set up /dev/net/tun, you should have read and write access to it, either as a normal user or as a member of a uml-users group. Once this is done, you can try the tunctl command again and it will succeed: host% tunctl Set 'tap0' persistent and owned by uid 500 This created a new TUN/TAP device and made it usable by the tunctl user. For scripting purposes, a -b option makes tunctl output only the new device name: host% tunctl -b tap1 This eliminates the need to parse the relatively verbose output from the first form of the command. There are also -u and -t options, which allow you to specify, respectively, which user the new TUN/TAP device will belong to and which TUN/TAP device that will be: host# tunctl -u jdike -t jeffs-uml Set 'jeffs-uml' persistent and owned by uid 500 This demonstrates a highly useful feature: the ability to give arbitrary names to the devices. Suitably chosen, these can serve as partial documentation of your UML network setup. We will use this jeffs-uml device from now on. For cleanliness, we should shut down all of the TUN/TAP devices created by our playing with tunctl with commands such as the following: host% tunctl -d tap0 Set 'tap0' nonpersistent ifconfig -a will show you all the network interfaces on the system, so you should probably shut down all of the TUN/TAP devices except for the last one you made and any others created for some other specific reason. The first thing to do is to enable the device: host# ifconfig jeffs-uml 192.168.0.254 up host# ifconfig jeffs-uml jeffs-uml Link encap:Ethernet HWaddr 2A:B1:37:41:72:D5 inet addr:192.168.0.254 Bcast:192.168.0.255 \ Mask:255.255.255.0 inet6 addr: fe80::28b1:37ff:fe41:72d5/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:5 overruns:0 carrier:0 collisions:0 txqueuelen:500 RX bytes:0 (0.0 b) TX bytes:0 (0.0 b) As usual, choose an IP address that's suitable for your network. If IP addresses are scarce, you can reuse one that's already in use by the host, such as the one assigned to its eth0. Basic ConnectivityLet's add a new interface to a UML instance. If you have an instance already running, you can plug a new network interface into it by using uml_mconsole: uml_mconsole debian config eth0=tuntap,jeffs-uml OK If you are booting a new UML instance, you can do the same thing on the command line by adding eth0=tuntap,jeffs-uml. This differs from the syntax we saw earlier. Before, we specified an IP address and no device name. Here, we specify the device name but not an IP address. When no device name is given, that signals the driver to invoke the uml_net helper to configure the host. When a name is given, the driver uses it and assumes that it has already been configured appropriately. Now that the instance has an Ethernet device, we can configure it and bring it up: UML# ifconfig eth0 192.168.0.253 up Let's try pinging the host: UML# ping 192.168.0.254 PING 192.168.0.254 (192.168.0.254): 56 data bytes --- 192.168.0.254 ping statistics --- 28 packets transmitted, 0 packets received, 100% packet loss Nothing but silence. The usual way to start debugging problems like this is to sniff the interface using tcpdump or a similar tool. With the ping running again, we see this: host# tcpdump -i jeffs-uml -l -n tcpdump: verbose output suppressed, use -v or -vv for full \ protocol decode listening on jeffs-uml, link-type EN10MB (Ethernet), capture \ size 96 bytes 18:12:34.115634 IP 192.168.0.253 > 192.168.0.254: icmp 64: echo \ request seq 0 18:12:35.132054 IP 192.168.0.253 > 192.168.0.254: icmp 64: echo \ request seq 256 Ping requests are coming out, but no replies are getting back to it. This is a routing problemwe have not yet set any routes to the TUN/TAP device, so the host doesn't know where to send the ping replies. This is easily fixed: host# route add -host 192.168.0.253 dev jeffs-uml Now, pinging from the UML instance works: UML# ping 192.168.0.254 PING 192.168.0.254 (192.168.0.254): 56 data bytes 64 bytes from 192.168.0.254: icmp_seq=0 ttl=64 time=0.7 ms 64 bytes from 192.168.0.254: icmp_seq=1 ttl=64 time=0.1 ms 64 bytes from 192.168.0.254: icmp_seq=2 ttl=64 time=0.1 ms --- 192.168.0.254 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.1/0.3/0.7 ms It's always a good idea to check connectivity in both directions, in case there is a problem in one direction but not the other. So, check whether the host can ping the instance: host% ping 192.168.0.253 PING 192.168.0.253 (192.168.0.253) 56(84) bytes of data. 64 bytes from 192.168.0.253: icmp_seq=0 ttl=64 time=0.169 ms 64 bytes from 192.168.0.253: icmp_seq=1 ttl=64 time=0.077 ms --- 192.168.0.253 ping statistics --- 2 packets transmitted, 2 received, 0% packet loss, time 1000ms rtt min/avg/max/mdev = 0.077/0.123/0.169/0.046 ms, pipe 2 So far, so good. The next step is to ping a host on the local network by its IP address: UML# ping 192.168.0.3 PING 192.168.0.3 (192.168.0.3): 56 data bytes --- 192.168.0.3 ping statistics --- 7 packets transmitted, 0 packets received, 100% packet loss No joy. Using tcpdump to check what's happening shows this: host# tcpdump -i jeffs-uml -l -n tcpdump: verbose output suppressed, use -v or -vv for full \ protocol decode listening on jeffs-uml, link-type EN10MB (Ethernet), capture \ size 96 bytes 18:20:29.522769 arp who-has 192.168.0.3 tell 192.168.0.253 18:20:30.524576 arp who-has 192.168.0.3 tell 192.168.0.253 18:20:31.522430 arp who-has 192.168.0.3 tell 192.168.0.253 The UML instance is trying to figure out the Ethernet MAC address of the target. To this end, it's broadcasting an arp request on its eth0 interface and hoping for a response. It's not getting one because the target machine can't hear the request. arp requests, like other Ethernet broadcast protocols, are limited to the Ethernet segment on which they originate, and the UML eth0 to host TUN/TAP connection is effectively an isolated Ethernet strand with only two hosts on it. So, the arp requests never reach the physical Ethernet where the other machine could hear it and respond. This can be fixed by using a mechanism known as proxy arp and enabling packet forwarding. First, turn on forwarding: host# echo 1 > /proc/sys/net/ipv4/ip_forward Then enable proxy arp on the host for the TUN/TAP device: host# echo 1 > /proc/sys/net/ipv4/conf/jeffs-uml/proxy_arp This will cause the host to arp to the UML instance on behalf of the rest of the network, making the host's arp database available to the instance. Retrying the ping and watching tcpdump shows this: host# tcpdump -i jeffs-uml -l -n tcpdump: verbose output \ suppressed, use -v or -vv for full protocol decode listening on jeffs-uml, link-type EN10MB (Ethernet), capture \ size 96 bytes 19:25:16.465574 arp who-has 192.168.0.3 tell 192.168.0.253 19:25:16.510440 arp reply 192.168.0.3 is-at ae:42:d1:20:37:e5 19:25:16.510648 IP 192.168.0.253 > 192.168.0.3: icmp 64: echo \ request seq 0 19:25:17.448664 IP 192.168.0.253 > 192.168.0.3: icmp 64: echo \ request seq 256 There is still no pinging, but the arp request did get a response. We can verify this by seeing what's in the UML arp cache. UML# arp Address HWtype HWaddress Flags Mask \ Iface 192.168.0.3 ether AE:42:D1:20:37:E5 C \ eth0 If you see nothing here, it's likely because too much time elapsed between running the ping and the arp, and the arp entry got flushed from the cache. In this case, rerun the ping, and run arp immediately afterward. Since the instance is now getting arp service for the rest of the network, and ping requests are making it out through the TUN/TAP device, we need to follow those packets to see what's going wrong. On my host, the outside network device is etH1, so I'll watch that. On other machines, the outside network will likely be eth0. It's also a good idea to select only packets involving the UML, to eliminate the noise from other network activity: host# tcpdump -i eth1 -l -n host 192.168.0.253 tcpdump: verbose output suppressed, use -v or -vv for full \ protocol decode listening on eth1, link-type EN10MB (Ethernet), capture size \ 96 bytes 19:36:14.459076 IP 192.168.0.253 > 192.168.0.3: icmp 64: echo \ request seq 0 19:36:14.461960 arp who-has 192.168.0.253 tell 192.168.0.3 19:36:15.460608 arp who-has 192.168.0.253 tell 192.168.0.3 Here we see a ping request going out, which is fine. We also see an arp request from the other host for the MAC address of the UML instance. This is going unanswered, so this is the next problem. We set up proxy arp in one direction, for the UML instance on behalf of the rest of the network. Now we need to set it up in the other direction, for the rest of the network on behalf of the instance, so that the host will respond to arp requests for the instance: host# arp -Ds 192.168.0.253 eth1 pub Retrying the ping gets some good results: UML# ping 192.168.0.3 PING 192.168.0.3 (192.168.0.3): 56 data bytes 64 bytes from 192.168.0.3: icmp_seq=0 ttl=63 time=133.1 ms 64 bytes from 192.168.0.3: icmp_seq=1 ttl=63 time=4.0 ms 64 bytes from 192.168.0.3: icmp_seq=2 ttl=63 time=4.9 ms --- 192.168.0.3 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 4.0/47.3/133.1 ms To be thorough, let's make sure we have connectivity in the other direction and ping the UML instance from the other host: 192.168.0.3% ping 192.168.0.254 PING 192.168.0.254 (192.168.0.254) from 192.168.0.3 : 56(84) \ bytes of data. 64 bytes from 192.168.0.254: icmp_seq=1 ttl=64 time=6.48 ms 64 bytes from 192.168.0.254: icmp_seq=2 ttl=64 time=2.76 ms 64 bytes from 192.168.0.254: icmp_seq=3 ttl=64 time=2.75 ms --- 192.168.0.254 ping statistics --- 3 packets transmitted, 3 received, 0% loss, time 2003ms rtt min/avg/max/mdev = 2.758/4.000/6.483/1.756 ms We now have basic network connectivity between the UML instance and the rest of the local network. Here's a summary of the steps we took.
Thoughts on SecurityAt this point, the machinations of the uml_net helper should make sense. To recap, let's add another interface to the instance and let uml_net set it up for us: host% uml_mconsole debian config eth1=tuntap,,,192.168.0.252 OK Configuring the new device in the instance shows us this: UML# ifconfig eth1 192.168.0.251 up * modprobe tun * ifconfig tap0 192.168.0.252 netmask 255.255.255.255 up * bash -c echo 1 > /proc/sys/net/ipv4/ip_forward * route add -host 192.168.0.251 dev tap0 * bash -c echo 1 > /proc/sys/net/ipv4/conf/tap0/proxy_arp * arp -Ds 192.168.0.251 jeffs-uml pub * arp -Ds 192.168.0.251 eth1 pub Here we can see the helper doing just about everything we just finished doing by hand. The one thing that's missing is actually creating the TUN/TAP device. uml_net does that itself, without invoking an outside utility, so that doesn't show up in the list of commands it runs on our behalf. Aside from knowing how to configure the host in order to support a networked UML instance, this is also important for understanding the security implications of what we have done and for customizing this setup for a particular environment. What uml_net does is not secure against a nasty root user inside the instance. Consider what would happen if the UML user decided to configure the UML eth0 with the same IP address as your local name server. uml_net would set up proxy arp to direct name requests to the UML instance. The real name server would still be there getting requests, but some requests would be redirected to the UML instance. With a name server in the UML instance providing bogus responses, this could easily be a real security problem. For this reason, uml_net should not be used in a serious UML establishment. Its purpose is to make UML networking easy to set up for the casual UML user. For any more serious uses of UML, the host should be configured according to the local needs, security and otherwise. What we just did by hand isn't that bad because we set the route to the instance and proxy arp according to the IP address we expected it to use. If root inside our UML instance decides to use a different IP address, such as that of our local name server, it will see no traffic. The host will only arp on behalf of the IP we expect it to use, and the route is only good for that IP. All other traffic will go elsewhere. A nasty root user can still send out packets purporting to be from other hosts, but since it can't receive any responses to them, it would have to make blind attacks. As I discussed earlier, this is unlikely to enable any successful attacks on its own, but it does remove a layer of protection that might prove useful if another exploit on the host allows the attacker to see the local network traffic. So, it is probably advisable to filter out any unexpected network traffic at the iptables level. First, let's see that the UML instance can send out packets that pretend to be from some other host. As usual for this discussion, these will be pings, but they could just as easily be any other protocol. UML# ifconfig eth0 192.168.0.100 up UML# ping 192.168.0.3 PING 192.168.0.3 (192.168.0.3): 56 data bytes --- 192.168.0.3 ping statistics --- 4 packets transmitted, 0 packets received, 100% packet loss Here I am pretending to be 192.168.0.100, which we will consider to be an important host on the local network. Watching the jeffs-uml device on the host shows this: host# tcpdump -i jeffs-uml -l -n tcpdump: verbose output suppressed, use -v or -vv for full \ protocol decode listening on jeffs-uml, link-type EN10MB (Ethernet), capture \ size 96 bytes 20:20:34.978090 arp who-has 192.168.0.3 tell 192.168.0.100 20:20:35.506878 arp reply 192.168.0.3 is-at ae:42:d1:20:37:e5 20:20:35.508062 IP 192.168.0.100 > 192.168.0.3: icmp 64: echo \ request seq 0 We can see those faked packets reaching the host. Looking at the host's interface to the rest of the network, we can see they are reaching the local network: tcpdump -i eth1 -l -n tcpdump: verbose output suppressed, use -v or -vv for full \ protocol decode listening on eth1, link-type EN10MB (Ethernet), capture size \ 96 bytes 20:23:30.741482 IP 192.168.0.100 > 192.168.0.3: icmp 64: echo \ request seq 0 20:23:30.744305 arp who-has 192.168.0.100 tell 192.168.0.3 Notice that arp request. It will be answered correctly, so the ping responses will go to the actual host that legitimately owns 192.168.0.100, which is not expecting them. That host will discard them, so they will cause no harm except for some wasted network bandwidth and CPU cycles. However, it would be preferable for those packets not to reach the network or the host in the first place. This can be done as follows: host# iptables -A FORWARD -i jeffs-uml -s \! 192.168.0.253 -j \ DROP Warning: wierd character in interface `jeffs-uml' (No aliases, \ :, ! or *). iptables is apparently complaining about the dash in the interface name, but it does create the rule, as we can see here: host# iptables -L Chain FORWARD (policy ACCEPT) target prot opt source destination DROP all -- !192.168.0.253 anywhere Chain INPUT (policy ACCEPT) target prot opt source destination Chain OUTPUT (policy ACCEPT) target prot opt source destination So, we have just told iptables to discard any packet it sees that:
After creating this firewall rule, you should be able to rerun the previous ping and tcpdump will show that those packets are not reaching the outside network. At this point, we have a reasonably secure setup. As originally configured, the UML instance couldn't see any traffic not intended for it. With the new firewall rule, the rest of the network will see only traffic from the instance that originates from the IP address assigned to it. A possible enhancement to this is to log any attempts to use an unauthorized IP address so that the host administrator is aware of any such attempts and can take any necessary action. You could also block any packets from coming in to the UML instance with an incorrect IP address. This shouldn't happen because the proxy arp we have set up shouldn't attract any packets for IP addresses that don't belong somehow to the host, and any such packets that do reach the host won't be routed to the UML instance. However, an explicit rule to prevent this might be a good addition to a layered security model. In the event of a malfunction or compromise of this configuration, such a rule could end up being the one thing standing in the way of a UML instance seeing traffic that it shouldn't. This rule would look like this: host# iptables -A FORWARD -o jeffs-uml -d \! 192.168.0.253 -j \ DROP Warning: wierd character in interface `jeffs-uml' (No aliases, \ :, ! or *). Access to the Outside NetworkWe still have a bit of work to do, as we have demonstrated access only to the local network, using IP addresses rather than more convenient host names. So, we need to provide the UML instance with a name service. For a single instance, the easiest thing to do is copy it from the host: host# cat > /etc/resolv.conf ; generated by /sbin/dhclient-script search user-mode-linux.org nameserver 192.168.0.3 I cut the contents of the host's /etc/resolv.conf and pasted them into the UML. You should do the same on your own machine, as my resolv.conf will almost certainly not work for you. We also need a default route, which hasn't been necessary for the limited testing we've done so far but is needed for almost anything else: UML# route add default gw 192.168.0.254 I normally use the IP address of the host end of the TUN/TAP device as the default gateway. If you still have the unauthorized IP address assigned to your instance's eth0, reassign the original address: ifconfig eth0 192.168.0.253 Now we should have name service: UML# host 192.168.0.3 Name: laptop.user-mode-linux.org Address: 192.168.0.3 That's a local namelet's check for a remote one: UML# host www.user-mode-linux.org www.user-mode-linux.org A 66.59.111.166 Now let's try pinging it, to see if we have network access to the outside world: UML# ping www.user-mode-linux.org PING www.user-mode-linux.org (66.59.111.166): 56 data bytes 64 bytes from 66.59.111.166: icmp_seq=0 ttl=52 time=487.2 ms 64 bytes from 66.59.111.166: icmp_seq=1 ttl=52 time=37.8 ms 64 bytes from 66.59.111.166: icmp_seq=2 ttl=52 time=36.0 ms 64 bytes from 66.59.111.166: icmp_seq=3 ttl=52 time=73.0 ms --- www.user-mode-linux.org ping statistics --- 4 packets transmitted, 4 packets received, 0% packet loss round-trip min/avg/max = 36.0/158.5/487.2 ms Copying /etc/resolv.conf from the host and setting the default route by hand works but is not the right thing to do. The real way to do these is with DHCP. The reason this won't work here is the same reason that ARP didn't workthe UML is on a different Ethernet strand than the rest of the network, and DHCP, being an Ethernet broadcast protocol, doesn't cross Ethernet broadcast domain boundaries. DHCP through a TUN/TAP DeviceSome tools work around the DHCP problem by forwarding DHCP requests from one Ethernet domain to another and relaying whatever replies come back. One such tool is dhcp-fwd. It needs to be installed on the host and configured. It has a fairly scary-looking default config file. You need to specify the interface from which client requests will come and the interface from which server responses will come. In the default config file, these are etH2 and eth1, respectively. On my machine, the client interface is jeffs-uml and the server interface is eth1. So, a global replace of eth2 with jeffs-uml, and leaving eth1 alone, is sufficient to get a working dhcp-fwd. Let's get a clean start by unplugging the UML eth0 and plugging it back in. First we need to bring the interface down: UML# ifconfig eth0 down Then, on the host, remove the device: host% uml_mconsole debian remove eth0 OK Now, let's plug it back in: host% uml_mconsole debian config eth0=tuntap,,fe:fd:c0:a8:00:fd,\ 192.168.0.254 OK Notice that we have a new parameter to this command. We are specifying a hardware MAC address for the interface. We never did this before because the UML network driver automatically generates one when it is assigned an IP address for the first time. It is important that these be unique. Physical Ethernet cards have a unique MAC burned into their hardware or firmware. It's tougher for a virtual interface to get a unique identity. It's also important for its IP address to be unique, and I have taken advantage of this in order to generate a unique MAC address for a UML's Ethernet device. When the administrator provides an IP address, which is very likely to be unique on the local network, to a UML Ethernet device, the driver uses that as part of the MAC address it assigns to the device. The first two bytes of the MAC will be 0xFE and 0xFD, which is a private Ethernet range. The next four bytes are the IP address. If the IP address is unique on the network, the MAC will be, too. When configuring the interface with DHCP, the MAC is needed before the DHCP server can assign the IP. Thus, we need to assign the MAC on the command line or when plugging the device into a running UML instance. There is another case where you may need to supply a MAC on the UML command line, which I will discuss in greater detail later in this chapter. That is when the distribution you are using brings the interface up before giving it an IP address. In this case, the driver can't supply the MAC after the fact, when the interface is already up, so it must be provided ahead of time, on the command line. Now, assuming the dhcp-fwd service has been started on the host, dhclient will work inside UML: UML# dhclient eth0 Internet Software Consortium DHCP Client 2.0pl5 Copyright 1995, 1996, 1997, 1998, 1999 The Internet Software \ Consortium. All rights reserved. Please contribute if you find this software useful. For info, please visit http://www.isc.org/dhcp-contrib.html Listening on LPF/eth0/fe:fd:c0:a8:00:fd Sending on LPF/eth0/fe:fd:c0:a8:00:fd Sending on Socket/fallback/fallback-net DHCPREQUEST on eth0 to 255.255.255.255 port 67 DHCPACK from 192.168.0.254 bound to 192.168.0.9 -- renewal in 21600 seconds. Final TestingAt this point, we have full access to the outside network. There is still one thing that could go wrong. Ping packets are relatively small; in some situations small packets will be unmolested but large packets, contained in full-size Ethernet frames, will be lost. To check this, we can copy in a large file: UML# wget http://www.kernel.org/pub/linux/kernel/v2.6/\ linux-2.6.12.3.tar.bz2 --01:35:56-- http://www.kernel.org/pub/linux/kernel/v2.6/\ linux-2.6.12.3.tar.bz2 => `linux-2.6.12.3.tar.bz2' Resolving www.kernel.org... 204.152.191.37, 204.152.191.5 Connecting to www.kernel.org[204.152.191.37]:80... connected. HTTP request sent, awaiting response... 200 OK Length: 37,500,159 [application/x-bzip2] 100%[====================================>] 37,500,159 \ 87.25K/s ETA 00:00 01:43:04 (85.92 KB/s) - `linux-2.6.12.3.tar.bz2' saved \ [37500159/37500159] Copying in a full Linux kernel tarball is a pretty good test, and in this case, it's fine. If this does nothing for you, it's likely that there's a problem with large packets. If so, you need to lower the Maximal Transfer Unit (MTU) of the UML's eth0: UML# ifconfig eth0 mtu 1400 You can determine the exact value by experiment. Lower it until large transfers start working. The cases where I've seen this involved a PPPoE connection to the outside world. PPPoE usually means a DSL connection, and I've seen UML connectivity problems when the host was my DSL gateway. Lowering the MTU to 1400 made the network fully functional. In fact, the MTU for a PPPoE connection is 1492, so lowering it to 1400 was overkill. BridgingAs mentioned at the start of this chapter, there are two ways to configure a host to give a UML access to the outside world. We just explored one of them. The alternative, bridging, doesn't require the host to route packets to and from the UML, and so doesn't require new routes to be created or proxy arp to be configured. With bridging, the TUN/TAP device used by the UML instance is combined with the host's physical Ethernet device into a sort of virtual switch. The bridge interface forwards Ethernet frames from one interface to another based on their destination MAC addresses. This effectively merges the broadcast domains associated with the bridged interfaces. Since this caused DHCP and arp to not work when we were doing IP forwarding, bridging provides a neat solution to these problems. If you currently have an active UML network, you should shut it down before continuing: UML# ifconfig eth0 down Then, on the host, remove the device: host% uml_mconsole debian remove eth0 OK Bring down and remove the TUN/TAP interface, which will delete the route and one side of the proxy arp, and delete the other side of the proxy arp: host# ifconfig jeffs-uml down host% tunctl -d jeffs-uml Set 'jeffs-uml' nonpersistent host# arp -i jeffs-uml -d 192.168.0.253 pub Now, with everything cleaned up, we can start from scratch: host% tunctl -u jdike -t jeffs-uml Let's start setting up bridging. The idea is that a new interface will provide the host with network access to the outside world. The two interfaces we are currently using, eth0 and jeffs-uml, will be added to this new interface. The bridge device will forward frames from one interface to the other as needed, so that both eth0 and jeffs-uml will see traffic that's intended for them (or that needs to be sent to the local network, in the case of eth0). The first step is to create the device using the brctl utility, which is in the bridge-utilities package of your favorite distribution: host# brctl addbr uml-bridge In the spirit of giving interfaces meaningful names, I've called this one uml-bridge. Now we want to add the two existing interfaces to it. For the physical interface, choose a wired Ethernetfor some reason, wireless interfaces don't seem to work in bridges. The virtual interface will be the jeffs-uml TUN/TAP interface. We need to do some configuration to make it usable: host# ifconfig jeffs-uml 0.0.0.0 up These interfaces can't have their own IP addresses, so we have to clear the one on eth0. This is a step you want to think about carefully. If you are logged in to the host remotely, this will likely kill your session and any network access you have to it. If the host has two network interfaces, and you know that your session and all other network activity you care about is traveling over the other, then it should be safe to remove the IP address from this one: host# ifconfig eth0 0.0.0.0 We can now add the two interfaces to the bridge: host# brctl addif uml-bridge jeffs-uml host# brctl addif uml-bridge eth0 And then we can look at our work: host# brctl show bridge name bridge id STP enabled \ interfaces uml-bridge 8000.0012f04be1fa no \ eth0 \ jeffs-uml At this point, the bridge configuration is done and we need to bring it up as a new network interface: host# dhclient uml-bridge Internet Systems Consortium DHCP Client V3.0.2 Copyright 2004 Internet Systems Consortium. All rights reserved. For info, please visit http://www.isc.org/products/DHCP /sbin/dhclient-script: configuration for uml-bridge not found. \ Continuing with defaults. Listening on LPF/uml-bridge/00:12:f0:4b:e1:fa Sending on LPF/uml-bridge/00:12:f0:4b:e1:fa Sending on Socket/fallback DHCPDISCOVER on uml-bridge to 255.255.255.255 port 67 interval 4 DHCPOFFER from 192.168.0.10 DHCPREQUEST on uml-bridge to 255.255.255.255 port 67 DHCPACK from 192.168.0.10 /sbin/dhclient-script: configuration for uml-bridge not found. \ Continuing with defaults. bound to 192.168.0.2 -- renewal in 20237 seconds. The bridge is functioning, but for any local connectivity to the UML instance, we'll need to set a route to it: host# route add -host 192.168.0.253 dev uml-bridge Now we can plug the interface into the UML instance and configure it there: host% uml_mconsole debian config eth0=tuntap,jeffs-uml,\ fe:fd:c0:a8:00:fd OK UML# ifconfig eth0 192.168.0.253 up Note that we plugged the jeffs-uml TUN/TAP interface into the UML instance. The bridge is merely a container for the other two interfaces, which can actually send and receive frames. Also note that we assigned the MAC address ourselves rather than letting the UML driver do it. A MAC is necessary in order to make a DHCP request for an IP address, while the driver requires the IP address before it can construct the MAC. In order to break this circular requirement, we need to assign the MAC that the interface will get. Now we can see some benefit from the extra setup that the bridge requires. DHCP within the UML instance now works: UML# dhclient eth0 Internet Systems Consortium DHCP Client V3.0.2-RedHat Copyright 2004 Internet Systems Consortium. All rights reserved. For info, please visit http://www.isc.org/products/DHCP Listening on LPF/eth0/fe:fd:c0:a8:00:fd Sending on LPF/eth0/fe:fd:c0:a8:00:fd Sending on Socket/fallback DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 5 DHCPOFFER from 192.168.0.10 DHCPREQUEST on eth0 to 255.255.255.255 port 67 DHCPACK from 192.168.0.10 bound to 192.168.0.253 -- renewal in 16392 seconds. This requires no messing around with arp or dhcp-fwd. Binding the TUN/TAP interface and the host's Ethernet interface makes each see broadcast frames from the other. So, DHCP and arp requests sent from the TUN/TAP device are also sent through the eth0 device. Similarly, arp requests from the local network are forwarded to the TUN/ TAP interface (and thus the UML instance's eth0 interface), which can respond on behalf of the UML instance. The bridge also forwards nonbroadcast frames, based on their MAC addresses. So, DHCP and arp replies will be forwarded as necessary between the two interfaces and thus between the UML instance and the local network. This makes the DHCP forwarding and the proxy arp that we did earlier completely unnecessary. The main downside to bridging is the need to remove the IP address from the physical Ethernet interface before adding it to the bridge. This is a rather pucker-inducing step when the host is accessible only remotely over that one interface. Many people will use IP forwarding and proxy arp instead of bridging rather than risk taking their remote server off the net. Others have written scripts that set up the bridge, taking the server's Ethernet interface offline and bringing the bridge interface online. Bridging and SecurityBridging provides access to the outside network in a different way than we got with routing and proxy arp. However, the security concerns are the samewe need to prevent a malicious root user from making the UML instance pretend to be an important server. Before, we filtered traffic going through the TUN/TAP device with iptables. This was appropriate for a situation that involved IP-level routing and forwarding, but it won't work here because the forwarding is done at the Ethernet level. There is an analogous framework for doing Ethernet filtering and an analogous tool for configuring it: ebtables, with the "eb" standing for "Ethernet Bridging." First, in order to demonstrate that we can do nasty things to our network, let's change our Ethernet MAC to one we will presume belongs to our name server or DHCP server. Then let's verify that we still have network access: UML# ifconfig eth0 hw ether fe:fd:ba:ad:ba:ad # ping -c 2 192.168.0.10 PING 192.168.0.10 (192.168.0.10) 56(84) bytes of data. 64 bytes from 192.168.0.10: icmp_seq=0 ttl=64 time=3.75 ms 64 bytes from 192.168.0.10: icmp_seq=1 ttl=64 time=1.85 ms --- 192.168.0.10 ping statistics --- 2 packets transmitted, 2 received, 0% packet loss, time 1018ms rtt min/avg/max/mdev = 1.850/2.803/3.756/0.953 ms, pipe 2 We do, so we need to fix things so that the UML instance has network access only with the MAC we assigned to it. Precisely, we want any Ethernet frame leaving the jeffs-uml interface on its way to the bridge that doesn't have a source MAC of fe:fd:c0:a8:00:fd to be dropped. Similarly, we want any frame being forwarded from the bridge to the jeffs-uml interface without that destination MAC to be dropped. The ebtables syntax is very similar to iptables, and the following commands do what we want: host# ebtables -A INPUT --in-interface jeffs-uml \ --source \! FE:FD:C0:A8:00:FD -j DROP host# ebtables -A OUTPUT --out-interface jeffs-uml \ --destination \! FE:FD:C0:A8:00:FD -j DROP host# ebtables -A FORWARD --out-interface jeffs-uml \ --destination \! FE:FD:C0:A8:00:FD -j DROP host# ebtables -A FORWARD --in-interface jeffs-uml \ --source \! FE:FD:C0:A8:00:FD -j DROP There is a slight subtlety heremy first reading of the ebtables man page suggested that using the FORWARD chain would be sufficient since that covers frames being forwarded by the bridge from one interface to another. This works for external traffic but not for traffic to the host itself. These frames aren't forwarded, so we could spoof our identity to the host if the ebtables configuration used only the FORWARD chain. To close this hole, I also use the INPUT and OUTPUT chains to drop packets intended for the host as well as those that are forwarded. At this point the ebtables configuration should look like this: ebtables -L Bridge table: filter Bridge chain: INPUT, entries: 1, policy: ACCEPT -s ! fe:fd:c0:a8:0:fd -i jeffs-uml -j DROP Bridge chain: FORWARD, entries: 2, policy: ACCEPT -d ! fe:fd:c0:a8:0:fd -o jeffs-uml -j DROP -s ! fe:fd:c0:a8:0:fd -i jeffs-uml -j DROP Bridge chain: OUTPUT, entries: 1, policy: ACCEPT -d ! fe:fd:c0:a8:0:fd -o jeffs-uml -j DROP We can check our work by trying to ping an outside host again: host# ping -c 2 192.168.0.10 PING 192.168.0.10 (192.168.0.10) 56(84) bytes of data. From 192.168.0.253 icmp_seq=0 Destination Host Unreachable From 192.168.0.253 icmp_seq=1 Destination Host Unreachable --- 192.168.0.10 ping statistics --- 2 packets transmitted, 0 received, +2 errors, 100% packet \ loss, time 1018ms, pipe 3 We should also check that we haven't made things too secure by accidentally dropping all packets. Let's reset our MAC to the approved value and see that we have connectivity: UML# ifconfig eth0 hw ether FE:FD:C0:A8:00:FD UML# ping -c 2 192.168.0.10 PING 192.168.0.10 (192.168.0.10) 56(84) bytes of data. 64 bytes from 192.168.0.10: icmp_seq=0 ttl=64 time=40.4 ms 64 bytes from 192.168.0.10: icmp_seq=1 ttl=64 time=3.93 ms --- 192.168.0.10 ping statistics --- 2 packets transmitted, 2 received, 0% packet loss, time 1036ms rtt min/avg/max/mdev = 3.931/22.190/40.449/18.259 ms, pipe 2 At this point, the UML instance can communicate with other hosts using only the MAC that we assigned to it. We should also be concerned with whether it can do harm by spoofing its IP. UML# ifconfig eth0 192.168.0.100 UML# ifconfig eth0 hw ether FE:FD:C0:A8:00:FD UML# ping -c 2 192.168.0.10 PING 192.168.0.10 (192.168.0.10) 56(84) bytes of data. 64 bytes from 192.168.0.10: icmp_seq=0 ttl=64 time=3.57 ms 64 bytes from 192.168.0.10: icmp_seq=1 ttl=64 time=1.73 ms --- 192.168.0.10 ping statistics --- 2 packets transmitted, 2 received, 0% packet loss, time 1017ms rtt min/avg/max/mdev = 1.735/2.655/3.576/0.921 ms, pipe 2 It can, so we need to apply some IP filtering. Because the jeffsuml interface is part of a bridge, we need to use the physdev module of iptables: host# iptables -A FORWARD -m physdev --physdev-in jeffs-uml \ -s \! 192.168.0.253 -j DROP Warning: wierd character in interface `jeffs-uml' (No aliases, \ :, ! or *). host# iptables -A FORWARD -m physdev --physdev-out jeffs-uml \ -d \! 192.168.0.253 -j DROP Warning: wierd character in interface `jeffs-uml' (No aliases, \ :, ! or *). host# iptables -A INPUT -m physdev --physdev-in jeffs-uml \ -s \! 192.168.0.253 -j DROP Warning: wierd character in interface `jeffs-uml' (No aliases, \ :, ! or *). host# iptables -A OUTPUT -m physdev --physdev-out jeffs-uml \ -d \! 192.168.0.253 -j DROP Warning: wierd character in interface `jeffs-uml' (No aliases, \ :, ! or *). These take care of packets intended for both this host and other systems. Earlier, I didn't include a rule to prevent packets with incorrect destination IP addresses from reaching a UML instance because the proxy arp and routing provided pretty good protection against that. I'm including the equivalent rule here because we don't have the same protection the bridging exposes the UML instances much more to the local network. |