Section 7.1. Available Tools

7.1. Available Tools

The primary debugging method in Mongrel is logging. By enabling different options or setting up special handlers you can get Mongrel to log helpful data to files found in the log/mongrel_debug directory. Since Mongrel is easily extensible, you can also use the current handlers as the basis for your own specialized debugging filters.

7.1.1. "Dash-Bee" Logging

The main logging people use is "Dash-Bee" or just -B for the mongrel_rails option that enables all of the available logging filters. It's very easy:

mongrel_rails start -e production -B

Once you run your Rails application this way, you'll get tons of information from the log/mongrel_debug directory. Notice that we started in production mode. This is done because in debuggin mode Mongrel is painfully slow, and also Rails development mode does quite a few nasty things to Ruby, which can skew your debugging output.

Right away Mongrel will start logging the following:

• accesses Apache-style access logs statements get printed to the console for you to see what requests are coming in.

• objects.log Used for debugging memory leaks. For each request a listing of every class currently active in object space, the number of objects of that class, the delta when compared with the last run, and a bit of information on the average lengths of objects that answer to the length method.

• files.log Used for finding file leaks, this log contains a list of each currently open file and how many times it's been opened without a close. File leaks are very crippling to Mongrel since Ruby 1.8.x can only handle about 1,024 depending on the system.

• rails.log Many times what you think Rails is receiving is different from what your HTTP client is sending. This log file will tell you exactly what parameters Rails is receiving for each request.

• threads.log Library authors love messing with the internals of Threads (specifically the Thread.current[ ] hash). This can cause unexpected behavior and memory leaks. Mongrel will dump the active threads and their Thread.current contents.

Each of these log files will be written to the mongrel_debug directory for each request. There isn't any locking or coordination done on this logging so you should avoid really thrashing Mongrel in -B mode to get better results.

7.1.2. USR1 Logging

While -B can help you do deeper debugging "back at the lab," it is not viable for many production situations since it's slow and does voodoo hackery on Ruby's internals. To help with two common problems while in production mode, Mongrel has a lighter type of debug logging called USR1 Logging. This mode is a toggled mode that simply flips a global variable to tell Mongrel to log more information when there's an exception, and to report on a few more key elements.

First, in USR1 mode Mongrel will dump the full request of any client that violates the HTTP specification. You can then use this to see if you should block this client, fix the client, or request that Mongrel's parser get an update.

Second, it logs at 10-second intervals the number of requests waiting for different Rails actions. This helps you spot the Rails actions and controllers that are blocking your Mongrel processes from continuing. It is a very light logging that will only go to your mongrel.log file (or console if not in daemon mode).

Using USR1 logging is very simple:

killall -USR1 mongrel_rails

On POSIX systems this will tell all your Mongrel processes to start doing USR1 logging. It doesn't do a restart, stop any active connections, or anything that should impact your running application. It's also not a performance hit (maybe 1% or so).

Once you've gathered some information you can use, then you simply run the killall command again and Mongrel will toggle the logging off.

Zed Sez

Have you ever worked with that one programmer who insists that his code is absolute perfection handed from Gaiai in little baskets woven from gold? You know the guy. He's the one that's constantly using big fancy words like "efferent coupling" and "semantic inference," and claims Haskell/Scheme/Lisp/ML is the greatest language ever.

You probably also have experienced when Super-Software-Architect-Master-Coder writes crap-buggy code yet defends it to his death. He'll insist that everyone else change to make his junk work. Quite often these so-called "master coders" are actually fairly incompetent and hide behind extensive education to cover their constant failures. I've worked with several who would rather quit than admit that they were wrong. Very unprofessional.

These people's problem is they suffer from Potpourri Turd Syndromea belief that their you-know-what don't stink and smells like fine dew on freshly cut grass. Whenever there's a bug, they go running like kids in a candy store to other people's code trying to find fault and just assume that it's nothing they wrote.

The developers that I consider good always assume that what they write is the first suspect when a bug comes through the door. They will spend quite a bit of time building evidence to rule out possible causes, try to find causes of defects, and have no problem admitting when their designs are flawed.

When you run into a problem with your application, always assume it's your fault first. Mongrel's not perfect, but its code is minuscule compared to the size of Rails and most likely even your own application code. Mongrel also powers many large and medium deployments without any problems. If there's an error, the evidence already says it's in your code, so bite the bullet and start investigating it as if it's your problem.

7.1.3. Customized Log Configuration

A more advanced usage of the logging capabilities in Mongrel is to use a mongrel.conf file and the -S option to configure just one of the debugging handlers. This lets you log only the part of the request you need to determine a problem while letting Mongrel run a bit quicker. All you have to do is create a mongrel.conf file with the appropriate Configurator#debug function call.

As an example, we'll configure just access logging and rails.log parameter logging:

debug "/", what = [:access, :rails]

Place this line in a mongrel.conf and then tell Mongrel to run this file with the -S option:

mongrel_rails start -e production -S mongrel.conf

You could configure your own custom handlers in a special mongrel_debug.conf that you run only in desperation to get special information.