Section 14.1. Challenges to Kernel Debugging

14.1. Challenges to Kernel Debugging

Debugging a modern operating system involves many challenges. Virtual memory operating systems present their own unique challenges. Gone are the days when we could replace a processor with an in-circuit emulator. Processors have become far too fast and complex. Moreover, pipeline architectures hide important code-execution details, partly because memory accesses on the bus can be ordered differently from code execution, and particularly because of internal caching of instruction streams. It is not always possible to correlate external bus activity to internal processor instruction execution, except at a rather coarse level.

Some of the challenges you will encounter while debugging Linux kernel code are:

• Linux kernel code is highly optimized for speed of execution in many areas.

• Compilers use optimization techniques that complicate the correlation of C source to actual machine instruction flow. Inline functions are a good example of this.

• Single-stepping through compiler optimized code often produces unusual and unexpected results.

• Virtual memory isolates user space memory from kernel memory and can make various debugging scenarios especially difficult.

• Some code cannot be stepped through with traditional debuggers.

• Startup code can be especially difficult because of its proximity to the hardware and the limited resources available (for example, no console, limited memory mapping, and so on).

The Linux kernel has matured into a very high-performance operating system capable of competing with the best commercial operating systems. Many areas within the kernel do not lend themselves to easy analysis by simply reading the source code. Knowledge of the architecture and detailed design are often necessary to understand the code flow in a particular area. Several good books are available that describe the kernel design in detail. Refer to Section 14.6.1, "Suggestions for Additional Reading," for recommendations.

GCC is an optimizing compiler. By default, the Linux kernel is compiled with the -O2 compiler flag. This enables many optimization algorithms that can change the fundamental structure and order of your code.^[1] For example, the Linux kernel makes heavy use of inline functions. Inline functions are small functions declared with the inline keyword, which results in the function being included directly in the execution thread instead of generating a function call and the associated overhead.^[2] Inline functions require a minimum of -O1 optimization level. Therefore, you cannot turn off optimization, which would be desirable for easier debugging.

^[1] See the GCC manual referenced at the end of this chapter in Section 14.6.1, "Suggestions for Additional Reading" for details on the optimization levels.

^[2] Inline functions are like macros, but with the advantage of compile-time type checking.

In many areas within the Linux kernel, single-stepping through code is difficult or impossible. The most obvious examples are code paths that modify the virtual memory settings. When your application makes a system call that results in entry into the kernel, this results in a change in address space as seen by the process. In fact, any transition that involves a processor exception changes the operational context and can be difficult or impossible to single-step through.