What s Next for DeadlockDetection?

What's Next for DeadlockDetection?

DeadlockDetection is a fairly complete utility, and I've used it successfully to track down quite a few multithreaded deadlocks. As always, however, I encourage you to look for ways that you can extend DeadlockDetection to make it more useful. Here are some of my ideas for enhancements to DeadlockDetection:

• Create a stand-alone application to manipulate the DEADLOCKDETECTION.INI file. Your program could be even nicer if it allowed you to set the DeadDetExt DLL and validated that the selected DeadDetExt DLL exported the correct functions.

• You could optimize the hook functions better if they weren't doing any logging. In that case, not all the register values need to be copied.

• Right now, DeadlockDetection just skips hooking a couple of DLLs that it knows about. A mechanism for specifying on a program-by-program basis which DLLs to skip would be nifty.

Debugging War Story: Uncommitted Transactions with Pooled COM Objects

The Battle

My good friend Peter Ierardi told me about an interesting multithreading bug he encountered. He was working on a large DCOM project that used a multithreaded DCOM service to coordinate database transactions. The DCOM service managed the transactions by creating a pool of database-centric in-process COM objects, which were used to write data into and read data from his relational database management system (RDBMS). The component-to-component communication took place through Microsoft Message Queue Server (MSMQ). Even though there were explicit, transactional commits, the data didn't seem to be written to the database. The DCOM service would retry three to five times, and the data would finally appear, as if by magic. Obviously, the excessive retries were taking a toll on the application's performance, and the fact that the data wasn't being written to the database was cause for alarm.

The Outcome

After some heavy debugging sessions, Peter found that the DCOM service was performing the reads and writes on separate, nonsynchronized threads. The read occurred before a separate instance of the database COM object had written the data. This behavior wasn't evident during debugging sessions because the debugger was forcing the proper timing and synchronization. He eventually uncovered the problem through appropriate labeling of object instances in the event log.

The Lesson

The big lesson Peter said he learned from this bug was that in a large-scale distributed application such as the one he was working on, you can't assume that the debug environment will accurately represent the release environment. He solved the problem by adding appropriate synchronization code and wrapping the component-to-component communications, which originally went through MSMQ individually, into the same transaction along with the database writes such that the messages would be sent only upon transactional commits.

What was happening in Peter's bug was that the MSMQ reads/writes, not surprisingly, were cycling much faster than the database reads/writes. Even though Peter and his team had walked through and planned all the multithreading carefully, they were still bitten by some initial misunderstanding about how much faster certain operations outside their project would take in the real world.