The Management Console
| The UML Management Console (MConsole) support comes in two distinct piecesa protocol and the clients that support the protocol. All we've seen so far is the default MConsole client, uml_mconsole. The protocol determines how uml_mconsole (and other clients) send requests to the MConsole driver in the UML kernel and get responses back. We will talk more about the MConsole protocol later in this chapter. For now, it suffices to say that the protocol is dead simple, and it takes much less than a day of work to implement a basic client for it in any reasonable language such as C or a scripting language such as Perl. MConsole clients can, and do, implement some functionality that has no counterpart in the MConsole protocol. These things are implemented locally, within the client, and will differ from client to client. The upcoming discussion refers to the uml_mconsole client. Later, we will talk about some other MConsole clients. uml_mconsole can be used to perform a number of types of queries and control operations on a UML instance, such as:
MConsole QueriesVersionThe most basic query is the version command, which returns the version of the kernel that the UML instance is running. The syntax is simple: host% uml_mconsole debian version OK Linux usermode 2.6.13-rc5 #29 Fri Aug 5 19:12:02 EDT 2005 \ i686 This returns nearly the same output as uname -a would return inside the UML instance: # uname -a Linux usermode 2.6.13-rc5 #29 Fri Aug 5 19:12:02 EDT 2005 \ i686 GNU/Linux The output is composed of the following pieces:
You don't generally care about the version that a particular UML is running since, if you are a careful UML administrator, you should know that already. The real value of this query is that it serves as a sort of ping to the UML to check that it is alive, at least enough to respond to interrupts. Hardware ConfigurationWe've seen this use of uml_mconsole already, when figuring out which host devices our UML consoles and serial lines had been attached to and when hot-plugging block and network devices. Even in those examples, we've seen only some of the available functionality. All of the drivers that have MConsole support, which is all of the commonly used ones, support the following operations:
The syntax for hot-plugging a device is: config device=configuration The syntax for hot-unplugging a device is: remove deviceThe syntax for requesting the configuration of a device is: config deviceUnplugging a device will fail if the device is busy in some way that makes it hard or impossible to remove. Table 8.1 summarizes the device names, syntax of the configuration data, and busyness criteria.
Halting and Rebooting a UML InstanceA UML instance can be halted or rebooted from the host using the halt or reboot commands, respectively. The kernel will run its shutdown procedure, which involves flushing unwritten data out to stable storage, shutting down some subsystems, and freeing host resources. However, this is a forced shutdownthe distribution's shutdown procedure will not run. So, services that are running inside the UML will not be shut down cleanly, and this may cause some problems with them on the next reboot. For example, pid files won't be removed, and these may prevent the initialization scripts from starting services by faking them into believing they are already running. For a mechanism to shut down the guest more cleanly, use the MConsole cad command. The halt and reboot commands are useful when the UML instance's userspace isn't responding reasonably and can't shut itself down. If the kernel is still responding to interrupts, these commands can ensure a clean kernel shutdown with the filesystems unmounted and clean. Invoking the Ctrl-Alt-Del HandlerThe distribution's Ctrl-Alt-Del handler can be invoked using the cad command. Unlike the halt and reboot commands, cad can be used to cleanly shut down the guest, including running the distribution's full shutdown procedure. This will cause all the services to be cleanly turned off, so there will be no problems as a result on the next boot. The exact action taken by the UML instance in response to this command depends on the distribution. The init process is in charge of handling this, as the kernel passes the event on to it. An entry in /etc/ inittab controls what init does. The most common action is to reboot, as shown in this entry: # What to do when CTRL-ALT-DEL is pressed. ca:12345:ctrlaltdel:/sbin/shutdown -t1 -a -r now Before booting a UML instance on a filesystem image, it's best to decide on the preferred action for Ctrl-Alt-Del. If you want to halt the UML instance rather than reboot it, remove the -r from the inittab entry above. Note that actually pressing the Ctrl, Alt, and Del keys on your keyboard into a UML session will not have the desired effect. If that has any effect at all, it will reboot the host since the keyboard belongs to it rather than the UML instance. Since UML doesn't have anything like a keyboard that can be made to treat a particular key combination specially, it uses this rather more abstract method in order to obtain the same results. Invoking the SysRq HandlerThe SysRq key is another way to get the kernel to perform some action on your behalf. Generally, this is intended to debug a sick system or to shut it down somewhat cleanly when nothing else will work. Like Ctrl Alt-Del, access to this is provided through the MConsole protocol, using the sysrq command to the uml_console client. Use of this command requires that the UML kernel have CONFIG_MAGIC_SYSRQ enabled. Failure to do this will result in an error such as the following: host% uml_mconsole debian sysrq p ERR Sysrq not compiled in The facility also must be turned on during boot. This is controlled by the /proc/sys/kernel/sysrq file (if it contains 1, SysRq is enabled; 0 means that it is disabled) and by the kernel.sysrq sysctl parameter. Some distributions disable SysRq by default during boot. For example, Fedora Core 4 disables it with these lines in /etc/sysctl.conf: # Controls the System Request debugging functionality of the kernel kernel.sysrq = 0 You would need to change that 0 to 1in order for the instance to support sysrq requests. Any output from a sysrq command is returned to the MConsole client and sent to the UML kernel log and, depending on the distribution, the main console. For example, invoking the sysrq m command, to dump the kernel's memory statistics, looks like this: host% uml_mconsole debian sysrq m OK SysRq : Show Memory Mem-info: DMA per-cpu: cpu 0 hot: low 62, high 186, batch 31 used:174 cpu 0 cold: low 0, high 62, batch 31 used:19 Normal per-cpu: empty HighMem per-cpu: empty Free pages: 433128kB (0kB HighMem) Active:1995 inactive:1157 dirty:2 writeback:0 unstable:0 \ free:108282 slab:917 mapped:1399 pagetables:510 DMA free:433128kB min:2724kB low:3404kB high:4084kB \ active:7980kB inactive:4628kB present:463768kB pages_scanned:0 \ all_unreclaimable? no lowmem_reserve[]: 0 0 0 Normal free:0kB min:0kB low:0kB high:0kB active:0kB inactive:0kB \ present:0kB pages_scanned:0 all_unreclaimable? no lowmem_reserve[]: 0 0 0 HighMem free:0kB min:128kB low:160kB high:192kB active:0kB \ inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no lowmem_reserve[]: 0 0 0 DMA: 2*4kB 2*8kB 5*16kB 2*32kB 1*64kB 0*128kB 1*256kB 1*512kB \ 0*1024kB 1*2048kB 105*4096kB = 433128kB Normal: empty HighMem: empty Swap cache: add 0, delete 0, find 0/0, race 0+0 Free swap = 0kB Total swap = 0kB Free swap: 0kB 115942 pages of RAM 0 pages of HIGHMEM 3201 reserved pages 5206 pages shared 0 pages swap cached The output will also appear in the kernel log. This and the output of many of the other commands are dumps of internal kernel state and aren't meant to be analyzed by normal users. This information is useful when a UML instance is in sufficiently bad shape as to require internal kernel information for a diagnosis. Table 8.2 summarizes the sysrq commands and what they do.
Stopping and Restarting a UML InstanceMConsole provides the ability to stop and continue a UML instance. When it is stopped in this manner, it is doing nothing but interpreting MConsole commands. Nothing else is happening. Processes aren't running and nothing else in the kernel is running, including device interrupts. This state will persist until the MConsole driver receives the command to continue. The main use of this functionality is to perform an online backup by stopping the instance, having it write out all unwritten file data to disk, copying the now clean filesystem to someplace safe, and continuing the UML instance. The full procedure looks like this: host% uml_mconsole debian stop OK host% uml_mconsole debian sysrq s OK SysRq : Emergency Sync host% cp --sparse=always cow save-cow host% uml_mconsole debian go OK The sysrq s command performs the synchronization of unwritten data to disk, resulting in this output to the kernel log: SysRq : Emergency Sync Emergency Sync complete In this example, I just copied the UML instance's COW file to a file in the same directory. Obviously, a rather more organized backup system would be advisable on a serious UML host. Such a system would keep track of what UML instances had been backed up, when they were last backed up, and the location of the backups. I used the --sparse=always switch to cp in order to preserve the sparseness of the COW file. This is important for speeding up the copy and for conserving disk space. Without it, all unoccupied blocks in the COW file will be filled with zeros on disk in the copy. This will result in those zero-filled blocks occupying host page cache for a while and will require that they all be written out to disk at some point. Keeping the copy sparse ensures that unoccupied blocks don't become instantiated, so they don't occupy memory before being written to disk, I/O bandwidth while being written, and disk space afterward. The copy took just under three seconds on my laptop, making this a very quick way to get a backup of the UML instance's data, causing almost no downtime. This works particularly well with COW files since backing up a full filesystem image would take noticeably longer and consume more bandwidth while writing the copy out to disk. Logging to the UML Instance's Kernel LogThe log command enables arbitrary text to be inserted into the UML instance's kernel log. This was written in order to allow administrators of UML honeypots to overwrite the preexisting, UML-specific kernel log with a log that looks like it came from a physical machine. Since the purpose of a virtual honeypot is to pretend to be a physical machine, it is important that there be no easy ways for an intruder to discern that it is a virtual machine. Examining the kernel log is a fairly easy way to tell what sort of machine you're on because it contains a great deal of information about the hardware. Since the kernel log has a fixed size, logging enough data will cause any previous data to be lost, and the kernel log will contain only what you logged with MConsole. There are probably limited uses of this ability outside of honeypots, but it could be useful in a situation where events inside a UML instance need to be coordinated with events outside. If the kernel log of the UML instance is the official record of the procedure, the log MConsole command can be used to inject outside messages so that the kernel log contains all relevant information in chronological order. The uml_mconsole client has a log -f <file> command that will log the contents of the given file to the UML instance's kernel log. Examining the UML Instance's /procYou can use the MConsole proc command to examine the contents of any file within the UML's /proc. This is useful for debugging a sick UML instance, as well as for gathering performance data from the host. This output gets returned to the MConsole client, as seen here: host% uml_mconsole debian proc meminfo OK MemTotal: 450684 kB MemFree: 434608 kB Buffers: 724 kB Cached: 8180 kB SwapCached: 0 kB Active: 7440 kB Inactive: 3724 kB HighTotal: 0 kB HighFree: 0 kB LowTotal: 450684 kB LowFree: 434608 kB SwapTotal: 0 kB SwapFree: 0 kB Dirty: 0 kB Writeback: 0 kB Mapped: 5632 kB Slab: 3648 kB CommitLimit: 225340 kB Committed_AS: 10820 kB PageTables: 2016 kB VmallocTotal: 2526192 kB VmallocUsed: 24 kB VmallocChunk: 2526168 kB This sort of thing would be useful in monitoring the memory consumption of the UML instances running on a host. Its intended purpose is to allow a daemon on the host to monitor memory pressure inside the UML instances and on the host, and to use memory hot-plug to shift memory between instances in order to optimize use of the host's physical memory. At this writing, this is a work in progress, as support in the host kernel is needed in order to make this work. A prototype of the host functionality has recently been implemented. However, it is unclear whether this interface will survive or when this ability will appear in the mainline kernel. Currently, this command can be used only for /proc files that you know exist. In other words, it doesn't work on directories, meaning you can't use it to discover what processes exist inside the UML and get their statistics. Forcing a Thread into ContextThe MConsole stack command is a bit of a misnomer. While it does do what it suggests, its real purpose is somewhat different. Sending this command to a UML instance will cause it to dump the stack of the given process: host% uml_mconsole debian stack 1 OK EIP: 0073:[<400ecdb2>] CPU: 0 Not tainted ESP: 007b:bf903da0 \ EFLAGS: 00000246 Not tainted EAX: ffffffda EBX: 0000000b ECX: bf903de0 EDX: 00000000 ESI: 00000000 EDI: bf903dd8 EBP: bf903dd8 DS: 007b ES: 007b 15b07a20: [<080721bd>] show__regs+0xd1/0xd8 15b07a40: [<0805997d>] _switch_to+0x6d/0x9c 15b07a80: [<081b3371>] schedule+0x2e5/0x574 15b07ae0: [<081b3d37>] schedule__timeout+0x4f/0xbc 15b07b20: [<080c1a85>] do__select+0x255/0x2e4 15b07ba0: [<080c1d75>] sys__select+0x231/0x43c 15b07c20: [<0805f591>] handle__syscall+0xa9/0xc8 15b07c80: [<0805e65a>] userspace+0x1ae/0x2bc 15b07ce0: [<0805f11e>] new__thread_handler+0xaa/0xbc 15b07d20: [<00dde420>] 0xdde420 The process ID I gave was one internal to UML, that of the init process. If you don't know what processes are running on the system, you can get a list of them with sysrq t: host% uml_mconsole debian2 sysrq t The output looks in part like this: apache S 00000246 0 253 238 \ 252 (NOTLB) 14f03b10 00000001 bfffe0cc 0013a517 00000246 14f03b10 \ 000021a0 144d8000 14e75860 1448c740 144db98c 144db8d4 0805e941 00000001 \ 12002000 00000000 00000033 00000025 bfffe0cc 0013a517 00000246 bfacf178 \ 0000007b 0013a517 Call Trace: 144db990: [<0805f039>] switch__to_skas+0x39/0x74 144db9c0: [<08059955>] _switch_to+0x45/0x9c 144dba00: [<081b38a1>] schedule+0x2e5/0x574 144dba60: [<081b4289>] schedule__timeout+0x71/0xbc 144dba90: [<08187cf9>] inet__csk_wait_for_connect+0xc5/0x10c 144dbad0: [<08187de1>] inet__csk_accept+0xa1/0x150 144dbb00: [<081a529a>] inet__accept+0x26/0xa4 144dbb30: [<08165e10>] sys__accept+0x80/0x12c 144dbbf0: [<081667dd>] sys__socketcall+0xbd/0x1c4 144dbc30: [<0805f591>] handle__syscall+0xa9/0xc8 144dbc90: [<0805e65a>] userspace+0x1ae/0x2bc 144dbcf0: [<0805f1e0>] fork__handler+0x84/0x9c 144dbd20: [<00826420>] 0x826420 As with sysrq output, this will also be recorded in the kernel message log. This tells us we have an apache process whose process ID is 253. This also dumps the stack of every process on the system in exactly the same format as with the stack command. So why have the stack command when sysrq t gives us the same information and more? The reason is that the real intent of this command is to temporarily wake up a particular thread within the UML instance so it will hit a breakpoint, letting you examine the thread with the debugger. To do this, you must have the UML instance running under gdb, either from the start or by attaching to the instance later. You put a breakpoint on the show_regs() call in _switch_to, which is currently in arch/um/kernel/process_kern.c: do { current->thread.saved_task = NULL ; CHOOSE_MODE_PROC(switch_to_tt, switch_to_skas, prev, \ next); if(current->thread.saved_task) show_regs(&(current->thread.regs)); next= current->thread.saved_task; prev= current; } while(current->thread.saved_task);This call is what actually dumps out the stack. But since you put a breakpoint there, gdb will stop before that actually happens. At this point, gdb is sitting at the breakpoint with the desired thread in context. You can now examine that thread in detail. Obviously this is not useful for the average UML user. However, it is immensely useful for someone doing kernel development with UML who is seeing processes hang. Most commonly, it's a deadlock of some sort, and figuring out exactly what threads are holding what locks, and why is essential to debugging it. Waking up a particular thread and making it hit a breakpoint is very helpful. This sort of thing had been possible in tt mode for a long time, but not in skas mode until this functionality was implemented. In tt mode, every UML process or thread has a corresponding host process, and that process includes the UML kernel stack. This makes it possible to see the kernel stack for a process by attaching gdb to the proper host process. In skas mode, this is not the case. The UML kernel runs entirely within a single process, using longjmp to switch between kernel stacks on context switches. gdb can't easily access the kernel stacks of processes that are not currently running. Temporarily waking up a thread of interest and making it hit a breakpoint is a simple way to fix this problem. Sending an Interrupt to a UML InstanceThe int command is implemented locally within uml_mconsole. It sends an interrupt signal (SIGINT) to the UML instance that uml_mconsole is communicating with. It operates by reading the instance's pid file and sending the signal to that process. Normally, that instance will be running under gdb, in which case, the interrupt will cause the UML instance to stop running and return control to gdb. At that point, you can use gdb to examine the instance as you would any other process. If the UML instance is not running under gdb, the signal will cause it to shut down. Getting HelpFinally, there is a help command, which will display a synopsis of the available MConsole commands: host% uml_mconsole debian help OK Commands: version Get kernel version help Print this message halt Halt UML reboot Reboot UML config <dev>=<config> - Add a new device to UML; same syntax as command line config <dev> - Query the configuration of a device remove <dev> - Remove a device from UML sysrq <letter> - Performs the SysRq action controlled by \ the letter cad - invoke the Ctrl-Alt-Del handler stop - pause the UML; it will do nothing until it receives \ a 'go' go - continue the UML after a 'stop' log <string> - make UML enter <string> into the kernel \ log proc <file> - returns the contents of the UML's \ /proc/<file> Additional local mconsole commands: quit - Quit mconsole switch <socket-name> - Switch control to the given \ machine log -f <filename> - use contents of <filename> as \ UML log messages mconsole-version - version of this mconsole program The first section shows requests supported by the MConsole driver within the UML kernel. These are available to all clients, although perhaps in a different form. The second section lists the commands supported locally by this particular client, which may not be available in others. There is no predetermined set of requests within the MConsole protocol. Requests are defined by the driver and can be added or removed without changing the protocol. This provides a degree of separation between the client and the UML kernelthe kernel can add more commands and existing clients will be able to use them. This separation can be seen in the help message above. The first section was provided by the UML kernel and merely printed out by the uml_mconsole client. When a new request is added to the driver, it will be added to the kernel's help string, and it will automatically appear in the help text printed by the uml_mconsole client. Running Commands within the UML InstanceAn oft-requested MConsole feature is the ability to run an arbitrary command within a UML instance. I oppose this on the basis that there are perfectly good ways to run commands inside a UML instance, for example, by logging in and executing a command within a shell. A design ethic in the Linux kernel community holds that only things that need to be done in the kernel should be done there. The existence of other ways to run commands within a UML is proof that this functionality doesn't need to be in the kernel. Thus, I have refused to implement this or to merge other people's implementations. Nevertheless, a patch implementing this ability does exist, and it has a following in the UML community. With a suitably patched UML, it works like this: host% uml_mconsole debian exec "ps uax > /tmp/x" OK The command has been started successfully. The command's output isn't returned back to the MConsole client because it would be complicated to start a process from a kernel thread, capture its output, and return it to the outside. Thus, if you want the output, you need to save it someplace, as I did above by redirecting the output to /tmp/x, and then retrieve it. This is convenient, but I would claim that, with a little foresight on behalf of the host administrator, essentially the same thing can be done in other ways. The most straightforward way to do this is simply to log in to the UML and run the commands you need. Some people make a couple of common objections to this method.
I have what I believe to be solid answers to these objections. First, with an ssh key in the appropriate place in the UML filesystem, parsing the login and password prompts is unnecessary because there aren't any. Second, logging in over a UML console doesn't modify any network counters. Dedicating this console to the admin makes it possible to have a root shell permanently running on it, making even ssh unnecessary, and also not modifying the wtmp or utmp files because there's no login. Third, I don't think any of the alternatives are any more robust against disabling or manipulation than MConsole exec. An ssh login can be disabled by the UML root user deleting the ssh key. The console with a permanent root shell can be disabled by editing the UML instance's /etc/inittab. But MConsole exec can be disabled by moving or replacing the commands that the host administrator will run. The desire for something like MConsole exec is a legitimate one, and all of the current solutions have limitations. I believe that the longterm solution may be something like allowing a host process to migrate into the UML instance, allowing it to do its work inside that environment. In this case, the UML environment wouldn't be as opaque to the host as it is now. It would be possible to create a process on the host, guaranteeing that it is running the correct executable, and then move it into the UML instance. It would then see the UML filesystem, devices, processes, and so on and operate in that environment. However, it would retain ties to the host environment. For example, it would retain the file descriptors opened on the host before the migration, and it would be able to accept input and send output through them. Something like this, which can be seen as a limited form of clustering, seems to me to suffice and has none of the limitations of the other solutions. The uml_mconsole ClientWe've already seen a great deal of the uml_mconsole client, as it has been used to illustrate all of the MConsole discussion to date. However, there are some aspects we haven't seen yet. We have seen the format of the output of a successful request, such as this: host% uml_mconsole debian version OK Linux usermode 2.6.13-rc5 #29 Fri Aug 5 19:12:02 EDT 2005 \ i686 It always starts with OK or ERR to simplify automating the determination of whether the request succeeded or failed. This is how a failure looks: host% uml_mconsole debian remove ubda ERR Device is currently open Because the /dev/ubda device is currently mounted, the removal request fails and is reported with ERR followed by the human-readable error message. An important part of uml_mconsole that we haven't seen is its internal command line. Every example I have used so far has had the command in the argument list. However, if you ran uml_mconsole with just one argument, a umid for a running UML instance, you would see something like this: host% uml_mconsole debian (debian) At this point, you can run any MConsole command. The prompt tells you which UML instance your request will be sent to. You can change which UML you are talking to by using the switch local command: (debian) switch new-debian Switched to 'new-debian' (new-debian) At this point, all requests will go to the new-debian UML instance. Finally, there is a local command that will tell you what version of the client you are running: (new-debian) mconsole-version uml_mconsole client version 2 Whether you're using uml_mconsole in single-shot mode, with the command on the uml_mconsole command line, or you're using its internal command line, commands intended for the UML MConsole driver are generally passed through unchanged. A single-shot command is formed by concatenating the command-line argument vector into a single string with spaces between the arguments. The one exception to this is for commands that take filenames as arguments. Currently, there is only one time this happenswhen indicating the files that a block device will be attached to. These may be specified as relative paths, which can cause problems when the UML instance and uml_mconsole process don't have the same working directory. A path relative to the uml_mconsole process working directory will not be successfully opened by the UML instance from its working directory. To avoid this problem, uml_mconsole makes such paths absolute before passing the request to the UML instance. The MConsole ProtocolThe MConsole protocol, between the MConsole client and the MConsole driver in the UML kernel, is the glue that makes the whole thing work. I'm going to describe the protocol in enough detail that someone, sufficiently motivated, could implement a client. This won't take too long since the protocol is extremely simple. As with any client-server protocol, the client forms a request, sends it to the server, and at some later point gets a reply from the server. The request structure contains
In C, it looks like this: #define MCONSOLE_MAGIC (0xcafebabe) #define MCONSOLE_MAX_DATA (512) #define MCONSOLE_VERSION 2 struct mconsole_request { u32 magic; u32 version; u32 len; char data[MCONSOLE_MAX_DATA]; };The command goes into the data field as a string consisting of space-separated wordsexactly what the uml_mconsole client reads from its command line. The length of the command, the index of the NULL-terminator, is put in the len field. In Perl, forming a request looks like this: my $MCONSOLE_MAGIC = 0xcafebabe; my $MCONSOLE_MAX_DATA = 512; my $MCONSOLE_VERSION = 2; my $msg = pack("LiiA*", $MCONSOLE_MAGIC, $MCONSOLE_VERSION, \ length($cmd), $cmd);Once the request is formed, it must be sent to the server in the UML MConsole driver over a UNIX domain socket created by the driver. On boot, UML creates a subdirectory for instance-specific data, such as this socket. The subdirectory has the same name as the UML instance's umid, and its parent directory is the umid directory, which defaults to ~/.uml. So, a UML instance with a umid of debian will have its MConsole socket created at ~/.uml/debian/mconsole. The umid directory can be changed with the umid= switch on the UML command line. The request is sent as a datagram to the MConsole socket, where it is received by the driver and handled. The response will come back over the same socket in a form very similar to the request: struct mconsole_reply { u32 err; u32 more; u32 len; char data[MCONSOLE_MAX_DATA]; };err is the error indicatorif it is zero, the request succeeded and data contains the reply. If it is nonzero, there was some sort of error, and the data contains the error message. more indicates that the reply is too large to fit into a single reply, so more reply packets are coming. The final reply packet will have a more field of zero. As with the request, the len field contains the length of the data in this packet. In Perl, the response looks like this: ($err, $more, $len, $data) = unpack("iiiA*", $data);where $data is the packet read from the socket. The use of a UNIX domain socket, as opposed to a normal IP socket, is intentional. An IP socket would allow an MConsole client to control a UML instance on a different host. However, allowing this would require some sort of authentication mechanism built into the protocol, as this would enable anyone on the network to connect to a UML instance and start controlling it. The use of a UNIX domain socket adds two layers of protection. First, it is accessible only on the UML instance's host, so any users must be logged in to the host. Second, UNIX domain sockets are protected by the normal Linux file permission system, so that access to it can be controlled by setting the permissions appropriately. Rather than invent another authentication and authorization mechanism, the use of UNIX domain sockets forces the use of existing mechanisms. If remote access to the UML instance is required, executing the uml_mconsole command over ssh will use ssh authentication. Similarly, the file permissions on the socket make up the MConsole authorization mechanism. The MConsole Perl LibraryAs is evident from the Perl snippets just shown, uml_mconsole is not the only MConsole client in existence. There is a Perl client that is really a library, not a standalone utility. Part of the UML test suite, it is used to reconfigure UML instances according to the needs of the tests. In contrast to the uml_mconsole client, this library has a method for every MConsole request, rather than simply passing commands through to the server unchanged. Requests Handled in Process and Interrupt ContextsThere is a subtlety in how MConsole requests are handled inside the driver that can affect whether a sick UML will respond to them. Some requests must be handled in a process context, rather than in the MConsole interrupt handler. Any request that could potentially sleep must be handled in a process context. This includes config and remove, halt and reboot, and proc. These all call Linux kernel functions, which for one reason or another might sleep and thus can't be called from an interrupt handler. These requests are queued by the interrupt handler, and a special worker thread takes care of them at some later time. If the UML is sufficiently sick that it can't switch to the worker thread, such as if it is stuck handling interrupts, or the worker thread can't run, then these requests will never run, and the MConsole client will never get a reply. In this case, another mechanism is needed to bring down the UML instance in a semicontrolled manner. For this, see the final section of this chapter, on controlling UML instances With Signals from the host. MConsole NotificationsSo far, we have seen MConsole traffic initiated only by clients. However, sometimes the server in the UML kernel can initiate traffic. A notification mechanism in the MConsole protocol allows asynchronous events in the UML instance to cause a message to be sent to a client on the host. Messages can result from the following events.
The client that receives these notifications may be a different client than you would use to control the UML. In fact, the uml_mconsole client is incapable of receiving MConsole notifications. In order to generate notifications, a switch on the UML command line is needed to specify the UNIX socket to which the instance will send notifications. This argument on the command line specifies the file /tmp/notify, which must already exist, as the notification socket for this UML instance: mconsole=notify:/tmp/notify Using this small Perl script, we can see how notifications come back from the UML instance: use UML::MConsole; use Socket; use strict; my $sock = "/tmp/notify"; !defined(socket(SOCK, AF_UNIX, SOCK_DGRAM, 0)) and die "socket failed : $!\n"; !defined(bind(\*SOCK, sockaddr_un($sock))) and die "UML::new bind failed : $!\n"; while(1){ my ($type, $data) = UML::MConsole->read_notify(\*SOCK, undef); print "Notification type = \"$type\", data = \"$data\"\n"; }By first running this script and then starting the UML instance with the switch given above, we can see notifications being generated. The first one is the socket notification telling us that the MConsole request socket is ready: Notification type = "socket", data = \ "/home/jdike/.uml/debian/mconsole" Once the instance has booted, we can log in and send messages to the host through the /proc/mconsole file: UML# echo "here is a user notification" > /proc/mconsole This results in the following output from the notification client: Notification type = "user notification", \ data = "here is a user notification" These notifications all have a role to play in an automated UML hosting environment. The socket notification tells when a UML instance is booted enough to be controllable with an MConsole client. When this message is received, the instance can be marked as being active and the control tools told of the location of the MConsole socket. The panic and hang notifications are needed in order to know when the UML should be restarted, in the case of a panic, or forcibly killed and then restarted, in the case of a hang. The user notifications have uses that are limited only by the imagination of the administrator. I implemented them for the benefit of workloads running inside a UML instance that need to send status messages to the host. In this scenario, whenever some milestone is reached or some significant event occurs, a user notification would be sent to the client on the host that is keeping track of the workload's progress. You could also imagine having a tool such as a log watcher or intrusion detection system sending messages to the host through /proc/mconsole whenever an event of interest happens. A hosting provider could also use this ability to allow users to make requests from inside the UML instance. |
