Table of Contents | Solaris Performance and Tools: DTrace and MDB Techniques for Solaris 10 and OpenSolaris

Solaris™ Performance and Tools: DTrace and MDB Techniques for Solaris 10 and OpenSolaris
By Richard McDougall, Jim Mauro, Brendan Gregg
...............................................
Publisher: Prentice Hall
Pub Date: July 20, 2006
Print ISBN-10: 0-13-156819-1
Print ISBN-13: 978-0-13-156819-8
Pages: 496

Table of Contents | Index

Part One: Observability Methods

Chapter 1. Introduction to Observability Tools

Section 1.1. Observability Tools

Section 1.2. Drill-Down Analysis

Section 1.3. About Part One

Chapter 2. CPUs

Section 2.1. Tools for CPU Analysis

Section 2.2. vmstat Tool

Section 2.3. CPU Utilization

Section 2.4. CPU Saturation

Section 2.5. psrinfo Command

Section 2.6. uptime Command

Section 2.7. sar Command

Section 2.8. Clock Tick Woes

Section 2.9. mpstat Command

Section 2.10. Who Is Using the CPU?

Section 2.11. CPU Run Queue Latency

Section 2.12. CPU Statistics Internals

Section 2.13. Using DTrace to Explain Events from Performance Tools

Section 2.14. DTrace Versions of runq-sz, %runocc

Section 2.15. DTrace Probes for CPU States

Chapter 3. Processes

Section 3.1. Tools for Process Analysis

Section 3.2. Process Statistics Summary: prstat

Section 3.3. Process Status: ps

Section 3.4. Tools for Listing and Controlling Processes

Section 3.5. Process Introspection Commands

Section 3.6. Examining User-Level Locks in a Process

Section 3.7. Tracing Processes

Section 3.8. Java Processes

Chapter 4. Disk Behavior and Analysis

Section 4.1. Terms for Disk Analysis

Section 4.2. Random vs. Sequential I/O

Section 4.3. Storage Arrays

Section 4.4. Sector Zoning

Section 4.5. Max I/O Size

Section 4.6. iostat Utility

Section 4.7. Disk Utilization

Section 4.8. Disk Saturation

Section 4.9. Disk Throughput

Section 4.10. iostat Reference

Section 4.11. Reading iostat

Section 4.12. iostat Internals

Section 4.13. sar -d

Section 4.14. Trace Normal Form (TNF) Tracing for I/O

Section 4.15. DTrace for I/O

Section 4.16. Disk I/O Time

Section 4.17. DTraceToolkit Commands

Section 4.18. DTraceTazTool

Chapter 5. File Systems

Section 5.1. Layers of File System and I/O

Section 5.2. Observing Physical I/O

Section 5.3. File System Latency

Section 5.4. Causes of Read/Write File System Latency

Section 5.5. Observing File System "Top End" Activity

Section 5.6. File System Caches

Section 5.7. NFS Statistics

Chapter 6. Memory

Section 6.1. Tools for Memory Analysis

Section 6.2. vmstat(1M) Command

Section 6.3. Types of Paging

Section 6.4. Physical Memory Allocation

Section 6.5. Relieving Memory Pressure

Section 6.6. Scan Rate as a Memory Health Indicator

Section 6.7. Process Virtual and Resident Set Size

Section 6.8. Using pmap to Inspect Process Memory Usage

Section 6.9. Calculating Process Memory Usage with ps and pmap

Section 6.10. Displaying Page-Size Information with pmap

Section 6.11. Using DTrace for Memory Analysis

Section 6.12. Obtaining Memory Kstats

Section 6.13. Using the Perl Kstat API to Look at Memory Statistics

Section 6.14. System Memory Allocation Kstats

Section 6.15. Kernel Memory with kstat

Section 6.16. System Paging Kstats

Section 6.17. Observing MMU Performance Impact with trapstat

Section 6.18. Swap Space

Chapter 7. Networks

Section 7.1. Terms for Network Analysis

Section 7.2. Packets Are Not Bytes

Section 7.3. Network Utilization

Section 7.4. Network Saturation

Section 7.5. Network Errors

Section 7.6. Misconfigurations

Section 7.7. Systemwide Statistics

Section 7.8. Per-Process Network Statistics

Section 7.9. TCP Statistics

Section 7.10. IP Statistics

Section 7.11. ICMP Statistics

Chapter 8. Performance Counters

Section 8.1. Introducing CPU Caches

Section 8.2. cpustat Command

Section 8.3. cputrack Command

Section 8.4. busstat Command

Chapter 9. Kernel Monitoring

Section 9.1. Tools for Kernel Monitoring

Section 9.2. Profiling the Kernel and Drivers

Section 9.3. Analyzing Kernel Locks

Section 9.4. DTrace lockstat Provider

Section 9.5. DTrace Kernel Profiling

Section 9.6. Interrupt Statistics: vmstat -i

Section 9.7. Interrupt Analysis: intrstat

Part Two: Observability Infrastructure

Chapter 10. Dynamic Tracing

Section 10.1. Introduction to DTrace

Section 10.2. The Basics

Section 10.3. Inspecting Java Applications with DTrace

Section 10.4. DTrace Architecture

Section 10.5. Summary

Section 10.6. Probe Reference

Section 10.7. MDB Reference

Chapter 11. Kernel Statistics

Section 11.1. C-Level Kstat Interface

Section 11.2. Command-Line Interface

Section 11.3. Using Perl to Access kstats

Section 11.4. Snooping a Program's kstat Use with DTrace

Section 11.5. Adding Statistics to the Solaris Kernel

Section 11.6. Additional Information

Part Three: Debugging

Chapter 12. The Modular Debugger

Section 12.1. Introduction to the Modular Debugger

Section 12.2. MDB Concepts

Chapter 13. An MDB Tutorial

Section 13.1. Invoking MDB

Section 13.2. MDB Command Syntax

Section 13.3. Working with Debugging Targets

Section 13.4. GDB-to-MDB Reference

Section 13.5. dcmd and Walker Reference

Chapter 14. Debugging Kernels

Section 14.1. Working with Kernel Cores

Section 14.2. Examining User Process Stacks within a Kernel Image

Section 14.3. Switching MDB to Debug a Specific Process

Section 14.4. kmdb, the Kernel Modular Debugger

Section 14.5. Kernel Built-In MDB dcmds

Appendices

Appendix A. Tunables and Settings

Section A.1. Tunable Parameters in Solaris

Section A.2. System V IPC Tunables for Databases

Appendix B. DTrace One-Liners

Section B.1. DTrace One-Liners

Section B.2. DTrace Longer One-Liners

Appendix C. Java DTrace Scripts

Section C.1. dvm_probe_test.d

Section C.2. DVM Agent Provider Interface

Appendix D. Sample Perl Kstat Utilities

Section D.1. A Simple Kstat Walker

Section D.2. A Perl Version of Uptime

Section D.3. A Network Statistics Utility

Section D.4. A Performance Utility for CPU, Memory, Disk, and Net

Bibliography

Index