Panic! UNIX System Crash Dump Analysis |
By Chris Drake, Kimberley Brown |
| |
Publisher | : Prentice Hall PTR |
Pub Date | : May 19, 1995 |
ISBN | : 0-13-149386-8 |
Pages | : 496 |
| Copyright |
| | Figures |
| | Tables |
| | Code Examples |
| | Acknowledgments |
| | Introduction |
| | | What happened ? |
| | | What will Panic! teach you? |
| | | So many flavors of UNIX |
| | | The audience |
| | | Conventions used |
| | | Contacting the authors |
| | | Welcome to Panic! |
|
| | Part 1: Getting Started |
| | | Chapter 1. My System Has Crashed! |
| | | What is a system crash? |
| | | What conditions cause panics? |
| | | A word about bad traps |
| | | The panic() routine |
| | | How do you know if your system has panic'ed? |
| | | Capturing system crash information |
| | | What is a program crash in comparison? |
|
| | | Chapter 2. My System Is Hung! |
| | | What is a system hang? |
| | | What conditions cause hangs ? |
| | | How do you know if your system is hung? |
| | | What is a program hang in comparison? |
| | | Capturing system hang information |
|
| | | Chapter 3. The savecore Program |
| | | What is savecore? |
| | | How does savecore work? |
| | | Disk space requirement & locations |
| | | Security issues |
| | | Solaris 1: How to set up savecore |
| | | Solaris 2: How to set up savecore |
|
| | | Chapter 4. Hey! We Got One! |
| | | What to do when your system has crashed |
| | | Is the system still usable? |
| | | Turn off savecore? (How many dumps will you need?) |
| | | Saving the crash to tape for shipment or archives |
|
| | | Chapter 5. Crashing Your Own System |
| | | Crash your Solaris 2 system |
| | | Crash your Solaris 1 system |
|
| | | Chapter 6. Initial Analysis Without adb |
| | | Identifying the UNIX release & hardware architecture |
| | | The message buffer, msgbuf |
| | | Process status utilities: ps & pstat |
| | | Network status: netstat |
| | | NFS status: nfsstat |
| | | Address resolution protocol status: arp |
| | | Interprocess communication status: ipcs |
| | | The crash program |
| | | Summary |
|
| | | Chapter 7. Introduction to adb |
| | | Other debuggers |
| | | adb hardware & software requirements |
| | | The distribution of adb |
| | | The different uses of adb & kadb |
| | | adb macros & /usr/lib/adb |
| | | General startup syntax |
| | | Security issues |
| | | Other helpful files |
|
| | | Chapter 8. adb: The Gory Details |
| | | Basic commands |
| | | Displaying data |
| | | Locations & sizes |
| | | Miscellaneous commands |
| | | Summary |
|
| | | Chapter 9. Initial Analysis Using adb |
| | | Starting an adb session |
| | | System identification |
| | | Boot time, crash time, and uptime |
| | | Panic strings |
| | | The message buffer, msgbuf |
| | | Stack tracebacks |
| | | Summary |
|
| | | Chapter 10. The /usr/include Header Files |
| | | What is a header file? |
| | | The /usr/include directories |
| | | /usr/kvm/sys |
| | | /usr/share/src/uts |
| | | /usr/ ucbinclude |
| | | Summary |
|
| | | Chapter 11. Symbol Tables |
| | | Namelists & the nm command |
| | | Using adb to look at tiny's variables |
| | | A tiny summary |
|
| | | Chapter 12. adb Macros: Part One |
| | | The macro library |
| | | Reading and understanding macros |
| | | The utsname macro |
| | | The bootobj macro |
| | | Summary |
|
| | | Chapter 13. adb Macros: Par t Two |
| | | The msgbuf and msgbuf.wrap macros |
| | | The cpus, cpus.nxt, & cpu macros |
| | | Summary |
|
| | | Chapter 14. adb Macros: Writing Your Own |
| | | Exercise 1: Initial information |
| | | Exercise 2: DNLC, the directory name lookup cache |
| | | Exercise 3: Swap information |
| | | Extra Credit Challenge: Which process on which CPU? |
| | | Possible solutions |
|
|
| | Part 2: Advanced Studies |
| | | Chapter 15. Introduction to Assembly |
| | | High-level vs. low-level languages |
| | | Assembly languages |
| | | Basic CPU structure (all CPUs are similar) |
| | | Instruction execution |
| | | Instruction types |
| | | Instruction formats and addressing modes |
| | | Addressing and registers |
| | | Data in memory |
| | | On to SPARC! |
|
| | | Chapter 16. Introduction to SPARC |
| | | Basic characteristics of SPARC assembly language |
| | | SPARC instructions |
| | | SPARC registers |
| | | SPARC instruction types |
| | | Delayed Control Transfer Instructions |
| | | Looking at instructions in memory |
| | | Want to learn more? |
|
| | | Chapter 17. Stacks |
| | | A generic stack |
| | | The frame structure |
| | | Instructions that affect windows & frames |
| | | What have we got so far? |
|
| | | Chapter 18. Stack Tracebacks |
| | | Compiling with optimization |
| | | The trouble with more than six arguments |
|
| | | Chapter 19. A Kernel Overview |
| | | Major sections |
| | | Entering the kernel |
| | | Scheduling: processes and threads |
| | | File systems |
| | | Files, inodes, and processes |
| | | Memory Management |
| | | This was just a kernel overview |
|
| | | Chapter 20. Virtual Memory |
| | | The free list |
| | | Swap space |
| | | Page faults |
| | | Working set, or resident set |
| | | Keeping track of pages |
| | | Keeping track of process space |
| | | Anonymous memory |
| | | Kernel functions |
| | | Virtual memory routines |
| | | Address spaces |
| | | Segments |
| | | Pages |
| | | The hat layer |
|
| | | Chapter 21. Scheduling |
| | | SunOS 4.x |
| | | Solaris 2 |
| | | How you change it |
|
| | | Chapter 22. File Systems |
| | | Basic disk structure |
| | | The old original |
| | | VFS functions |
| | | UNIX File System (UFS) |
| | | Other file systems |
|
| | | Chapter 23. Hardware Devices and Drivers |
| | | Drivers and device control |
| | | Driver functions |
| | | Real hardware |
| | | Drivers and crashes |
|
| | | Chapter 24. Interprocess Communication |
| | | Semaphores |
| | | Messages |
| | | Shared memory |
| | | Common functions |
| | | Why all this? |
|
| | | Chapter 25. STREAMS |
| | | STREAMS structure |
| | | Data structures |
| | | Queues |
| | | Message structures |
| | | Data blocks |
| | | Pipes |
| | | Basic functions |
| | | Support functions |
| | | Digging around inside |
|
| | | Chapter 26. Trap Handling |
| | | Kinds of traps |
| | | Trap sequence |
| | | Trap frames |
| | | Trap types |
|
| | | Chapter 27. Watchdog Resets |
| | | What is a watchdog? |
| | | Can you get a core file? |
| | | What do you do next ? |
| | | Watchdog analysis |
| | | Summary |
| | | For further reference |
|
| | | Chapter 28. Interrupts |
| | | SPARC systems |
| | | Priority levels |
| | | Serial devices |
| | | Vectored interrupts |
| | | Polled interrupts |
| | | Interrupts in tracebacks |
|
| | | Chapter 29. Multiprocessor Kernels |
| | | Data protection |
| | | SunOS 4.x multiprocessor systems |
| | | Solaris 2 |
|
|
| | Part 3: Case Histories |
| | | Chapter 30. Network Troubles |
| | | Initial analysis |
| | | Check the instruction |
| | | Bug check |
| | | Resolution |
|
| | | Chapter 31. A Stomped-on Module |
| | | Strings output |
| | | Analysis using adb |
| | | Walking the stack by hand |
| | | The ipcaccess () routine |
| | | Loading the semsys module with modload |
| | | Remember to use the same OS! |
| | | Is it a hardware failure? |
| | | How about a software problem? |
| | | Using nm to query symbol values |
| | | adb's search command at work |
| | | Conclusion |
|
| | | Chapter 32. Hanging Instead of Swapping |
| | | Initial information |
| | | Process status |
| | | Stack tracebacks for every process |
| | | Resolution |
|
| | | Chapter 33. Panic'ed Pipes |
| | | Always get initial information |
| | | Walking the stack by hand |
| | | Examining assembly code: fifo_rdwr() |
| | | Calling parameters |
| | | Examining assembly code: vno_rw() |
| | | Don't work too hard! Use SunSolve! |
| | | Resolution |
|
| | | Chapter 34. A Sleeping Dragon |
| | | Initial information |
| | | Walking the stack by hand |
| | | Using the threadlist macro |
| | | Examining mutex locks |
| | | Data address not found? Dig deeper! |
| | | Additional analysis tools |
| | | Which processes were waiting for locks? |
| | | Loadable kernel modules |
| | | Conclusion |
|
| | | Chapter 35. Once Is Not Enough |
| | | The first captured crash |
| | | The second captured crash |
| | | The other crashes |
| | | The solution |
| | | Conclusion |
|
| | | Chapter 36. Life Without A Root Directory |
| | | Get initial information |
| | | Invoking mutex_enter() |
| | | What process was involved? |
| | | Going back through time |
| | | What's still useful in the stack traceback? |
| | | Why was FrameMaker involved? |
|
| | | Chapter 37. Disk Woes in the Wee Hours |
| | | Get initial information |
| | | Look for patterns |
| | | Inodes & vnodes |
| | | Crash 11: A closer look |
| | | Crash 12: A closer look |
| | | Crashes 13 & 14: A closer look |
| | | Resolution |
|
|
| | Appendix A. SPARC: The Gory Details |
| | | All CPUs are similar |
| | | The SPARC processor |
| | | Integer Unit (IU) |
| | | Floating-Point Unit (FPU) |
| | | Coprocessor (CP) |
| | | Windows & use of SPARC registers |
| | | In closing |
|
| | Appendix B. SPARC Instruction Set |
| | | Instruction set summary |
| | | Memory access instructions |
| | | Arithmetic / logical / shift instructions |
| | | Control transfer instructions |
| | | State register instructions |
| | | Floating-point unit instructions |
| | | Coprocessor instructions |
| | | Synthetic instructions |
| | | Always keep an instruction set reference handy! |
|