This section looks at a few of the techniques used to prevent rootkits.
A system that constantly monitors the kernel system call table, and refreshes the table whenever modifi-cations are detected, can stop many rootkits before they get a chance to do any damage. This feature should be accompanied by a capability to inform the user when changes are detected. The use of an informative hook detector while surfing the Internet is a great way to flag malicious web pages.
Unfortunately, most rootkits initially install only files and boot configuration changes, leaving the actual infection to occur with the next reboot. If the rootkit is positioned to load before the hook prevention software, there is a possibility kernel hook prevention will provide nothing more than a false sense of security.
Most of the services employed by the operating system are started during initial boot. Monitoring and reporting service startup after initial boot can provide protection against service-based malware. Services are not usually started unless the operator is aware of it. For instance, starting a program or installing new software are good reasons to expect a new service to be loaded, whereas surfing the Internet is not. Provided these conditions are clearly described, a simple dialog asking the user to authorize the loading of services can provide maximum protection with minimal operator interaction. This prevention technique should be accompanied by a white list, updated whenever the operator specifies “allow always.”
Unfortunately, this prevention technique has a vulnerability similar to kernel hook prevention. A service load prevention system cannot prevent the loading of services during the boot process, which is where most rootkits target their initialization.
This protection mechanism is similar to service load prevention, but driver load prevention will catch a lot more rootkits. If a driver is being loaded sometime after boot, the operator can be strongly advised to block the action unless he or she is installing new software. Moreover, there is no need to advise the operator at all if the driver loading technique is unconventional. For instance, to the best of my knowledge, the only drivers that are loaded using ZwSetSystemInformation with SystemLoadAnd CallImage are rootkits. Preventing this loading technique should not require operator interaction.
Unfortunately, this prevention technique has the same limitation as service load prevention, with the exception of unconventional driver loading, which can safely be blocked at all times.
Properly formatted software will separate code from data. When the program is loaded, the code and data are placed into separate sections of memory. Programs that don’t follow this convention have a high probability of being malicious - so high in fact that blocking the execution of these programs can be performed without operator interaction.
Unfortunately, searching data segments for code can be very process intensive. In addition, the possibility of false positives can plague an otherwise excellent security mechanism. Nonetheless, any form of software limited only by false positives and processing power is destined to overcome its limitations over time, so expect this detection method to gather momentum quickly.
The HTML exploit detailed in Chapter 10 is a good example of stack execution. Specifically, the payload used to bind port 28876 is contained in the local variable ShellcodeBytes. After the web page is loaded and the variable is placed on the stack, the CrashAndBurn function initiates a buffer overflow that returns program execution to the code placed on the stack. This form of code execution is only performed by malicious software, so blocking stack execution has no possibility of false positives.