What Tools Do You Use?
| When I do a presentation at a conference or a company, the most common question I get is: "What tools do you use?" As someone who's spent a major portion of his career writing software development tools, everyone is interested in what I'm using. As a tool freak, I'm more than happy to share. I'll use any tool that can save me five minutes because time is truly money. Throughout this book, I've been discussing various tools as they come up in the discussion. In this portion of the chapter, I want to list some of the tools that I haven't discussed or that require a more detailed explanation. If you're looking for general documentation and usage tools, see Chapter 2. For add-ins and macros, see Chapter 7. Everything from SysInternals!All through the book, I've referred to particular tools from Bryce Cogswell and Mark Russinovich at www.sysinternals.com, but anything at the Web site will be a lifesaver. All the tools there are free, so the price is certainly right! While you might not have heard of Bryce before, you've definitely heard of Mark. He's the coauthor, along with David Solomon, of Windows Internals, 4th Edition (Microsoft Press, 2005), the book on how Windows works. In addition to Mark's writing, he's the developer that broke the Sony rootkit story, which you can read about at http://www.sysinternals.com/blog/2005/10/sony-rootkits-and-digital-rights.html. As I was writing this book, both SysInternals and Winternals, Bryce and Mark's commercial software companies, were acquired by Microsoft. Even with all that, Mark's real claim to fame is that he used to be my neighbor in New Hampshire. Process ExplorerSimply put, Process Explorer is what TaskManager hopes to be when it grows up. If you want to see anything about running processes on your computer, Process Explorer is the tool. In the "Know Where Your Assemblies Load" section in Chapter 2, I discussed the simple steps on how to use Process Explorer to view your relocated DLLs in your address space. Although that's a great trick, there's much more to Process Explorer that I want to cover here. One of Process Explorer's fortes is that it lets you look at the open handles for a process. In the upper window, select your process and press Ctrl+H to cause the lower window to show you all the handles. It can't get much simpler than that. By default, Process Explorer shows you only the named handles. To see the unnamed handles, select Show Unnamed Handles And Mappings from the View menu. I will discuss why you want to name your handles and the strategies for doing so in the "The !handle Command" section of Chapter 6. Because Process Explorer will update the Handle View as the process creates and deletes handles, you can see the exact state at any time. A great debugging trick is to sort by name in the Handle View by clicking on the Name column. If you see the same name repeatedly, you're looking at a handle leak, and by naming your handles, you've made it trivial to find exactly which one is being leaked. I've found countless handle-leak problems in our clients' applications by using nothing more than the Handle View window in Process Explorer. By double-clicking any handle in the Handle View, you can see the detailed information about that handle. For example, if the handle is an Event, you can see its type and signaled state. This information is invaluable for looking at your handle state and potentially looking at deadlocks based on synchronization handles. Right-clicking a handle will show two menu items, the Properties and Close Handle commands. When I asked Mark why he put that feature in, he said, "Because I could." When I laughed and said that it was pretty dangerous, he said it was my job to come up with a reason for having it. The key reason for randomly closing handles in Process Explorer is so you can sneak into your manager's office and close half the handles in Outlook to keep him from sending you annoying e-mail. I figure that's a plenty good enough reason. The upper window in Process Explorer, the Process View, is where you can see all sorts of very important information about the process itself. If you press the Delete key after highlighting a process, that is a true death kill. If you've ever run into the issue of which Task Manager won't terminate a process, you'll find that Process Explorer is the true process assassin and will kill any type of process you have rights to. Make sure that you have the Confirm Kill command on the Options menu selected so you don't accidentally start terminating important processes, such as Csrss.exe (Client Server Runtime Subsystem) or Lsass.exe (Local Security Authority Service). Of course, if you kill either of those two processes, you'll actually know about it because Windows will stop running. Double-clicking a process or right-clicking and selecting Properties from the shortcut menu will get you the Properties dialog box for the process. The numerous tabs are self-explanatory, but I want to mention a couple highlights of which you might not be aware. The first is in the Performance Graph tab on which you see the process CPU utilization, Private Bytes history, and I/O Bytes history. The CPU utilization and I/O Bytes history are self-explanatory, but Private Bytes deserves elaboration. The working set for a process, which is the Mem Usage column in the Processes tab in TaskManager, is all the physical memory the process needs at the current moment to execute. That's a nice number, but it includes all the modules and files mapped into the address space along with all allocated memory. The Private Bytes are just the allocated memory for the process. Thus, if you see the Private Bytes increasing over time, you may be looking at a memory leak. If Private Bytes, which is a standard performance counter, are increasing at the same rate as the .NET CLR memory counter # Bytes in All Heaps, the memory being used is .NET memory. If the Private Bytes are increasing at a sharper rate than the # Bytes in All Heaps, that's a sign of a potential native code memory leak. The Threads tab in the process Property page will show you the native call stacks for the threads in your process. However, to get the best call stacks, you'll first need to tell Process Explorer about your Symbol Server. Select Configure Symbols from the Options menu, and in the Configure Symbols dialog box that pops up, the first text box, Dbghelp.dll path, needs to point to the latest version of Dbghelp.dll from your WinDBG installation. Make sure to point at the proper "bitness" and CPU version when specifying the location. In the Symbols Path text box, type the same value you set for the _NT_SYMBOL_PATH environment variable I discussed in "Set Up a Symbol Store" in Chapter 2. What makes Process Explorer's stack walking so interesting is that it will show you the call stack from user mode through to kernel mode. Without running a kernel debugger, Process Explorer is the only tool that shows you the real stack in your application. As with closing handles, the Threads tab in the process Properties dialog box allows you to kill a thread. I guess this is another one of those "because I can" moments for Mark. One nice trick of the Process View is that it will place a background color on the process based on key information. If you go into the Configure Highlighting dialog box, you'll see the legend for all the colors. I do want to point out that if you're on a 64-bit machine, processes that contain .NET will be highlighted in the default yellow only if they are 64-bit processes. In other words, 32-bit processes, such as Visual Studio or those compiled as 32-bit, running in Windows on Windows (WoW), will not have the highlighting. However, if you select the process and look for MSCoree.dll in the DLL View, you can check if .NET is running in that process. Have you ever tried to delete a file and received the dreaded message, "Error Deleting File or Folder, Cannot delete Foo: Access is denied. Make sure the disk is not full or write-protected"? You know you have access to the file, so it's not a security error, but some process has the file open and you have no idea which one. Process Explorer comes to the rescue with the Process Explorer Search dialog box accessible by pressing Ctrl+F. In the dialog box, type in the partial name of the file you want to delete, and the Process Explorer Search dialog box will show you all DLLs and handles with that name. Select the handle, and the main Process Explorer window will jump to show you that handle in the process that has it open. Right-click the handle in the main Process Explorer window and select Close Handle. Now you can delete the file. Of course, you may end up crashing the application that had it open, so alternatively, you can choose to kill the whole process instead. Using the Process Explorer Search dialog box is also an easy way to see which of the many ASP.NET worker processes is hosting a particular ASP.NET site. Simply type in the name of the site in the Process Explorer Search dialog box, and you'll find the instance immediately. I've already mentioned a few hints for configuring Process Explorer, but the best new feature in the 10.0 version was the ability to set up column groups. Choosing Select Columns from the View menu brings up the dialog box in which you can select the columns that appear in the various list views in Process Explorer. The Process Image, Process Performance, and .NET tabs of the Select Columns dialog box are interesting because those contain the columns you can select for the Process View. One weakness of Process Explorer is that there's no way to export the column groups from one machine to the rest of your computers. Mark may add that feature in a future version, but a little poking around in the registry and a hint from Mark showed me where they were stored. In the .\ProcessExplorer\ ProcessExplorerColumnSets.reg file that installs with the book's source code, are my column settings for Process Explorer. I've listed the column groups in Tables 4-1 and 4-2 so you can see the most important performance counters at a glance and why I have them turned on. Because Process Explorer sets up the column sets with accelerator keys, it's trivial to switch between them. If you chose to use my .reg file, it will remove your first four existing column sets and replace them with mine.
Table 4-1 lists the columns I have turned on for Process Explorer Column Sets. Table 4-2 shows the unique values for each particular Column Set. Only the checked items are used; all other check-box items in the Select Column dialog tabs are clear. One major question I've gotten many times about Process Explorer is how a single binary manages to run natively on Windows 9x (yes, Mark still supports Windows 95!), 32-bit versions of Windows 2000, Windows XP, Windows Vista, and Windows Server 2003, and the x64 and IA64 versions of Windows XP, Windows Vista, and Windows Server 2003? When you consider that Process Explorer is loading a device driver, which has to be CPU specific, it's obviously performing a little magic to get a single EXE to run without an installation. The magic is all done through binary resources in the resources for the main binary. Procexp.exe is the 32-bit native application. When Procexp.exe runs on an x64 operating system, it detects the operating system and extracts from the 32-bit resources the x64 version of Process Explorer, called, appropriately enough, Procexpx64.exe. The x64 EXE is written to the disk, and Procexp.exe starts Procexpx64.exe. You can see this by looking at the Process Explorer Process View on an x64 machine. You'll see the hierarchy of Procexp.exe starting Procexpx64.exe. As you can probably guess, the device driver Process Explorer uses is also embedded as a binary resource and is extracted and loaded when the appropriate bit version starts. Process MonitorFor years, we've been using FileMon (http://www.sysinternals.com/utilities/filemon.html) and RegMon (http://www.sysinternals.com/utilities/regmon.html) from SysInternals to look at all the file and registry accesses on our computers. Mark and Bryce were looking at the state of their tools and decided to combine FileMon and RegMon and add more process-related activity; thus Process Monitor was born. Like all SysInternals tools, it runs on all 32-bit and 64-bit based Windows operating systems from Windows 2000 forward. What makes Process Monitor so interesting is that the filtering has taken a major jump in improvement. Prior tools did destructive filtering, which actually didn't collect data when filters were applied, but Process Monitor keeps the data so you don't lose what's already been monitored. Additionally, the filters are drastically improved so you can narrow down the display on anything you want. My favorite filtering enhancement is that when creating filters, instead of relying on your typing the names for a process, the filters' drop-down list will include all the processes currently running on the computer. Additionally, you can filter the output on items directly from the list view by right-clicking and selecting the item you want to filter on from the possible values. Process Monitor allows you to save your particular filters so you can apply them again. This is incredibly convenient when you want to monitor multiple processes that make up a solution. The list of items that you can filter on contains all the different data items Process Monitor collects. If you want to filter on all RegRenameKey API calls, you can do so. In addition to the great new filtering, another new feature I'm sure everyone will find useful is the ability to see the full stack on any of the events collected. As does Process Explorer, the Process Monitor call stacks show you both the user-mode and kernel-mode sides. It's so trivial to see that your process is querying for a file or registry key over and over again that even a manager can figure out that doing that isn't a good idea. Reflector by Lutz RoederIf it weren't for Reflector (http://www.aisto.com/roeder/dotnet/), no one could do any .NET development at all. With this tool, we have the source code to the Framework Class Library (FCL) in addition to any other .NET code you want to look at. I've referred to Reflector all through the book because I used it to figure out all sorts of interesting problems, especially in Chapter 8, in which I used it to figure out how to write Code Analysis/FxCop rules even though rule development is completely undocumented. What many people aren't aware of is that Reflector offers a plug-in architecture so developers can extend its functionality. Lutz keeps a running list of new Reflector add-ins on his blog at http://www.aisto.com/roeder/Frontier/. There are lots of add-ins up there, including one to decompile to the Boo programming language if you're feeling scary. What I want to do is to show you the add-ins that I find most useful. Keep in mind that Lutz doesn't worry too hard about backwards compatibility with add-ins, so if you get a new version of Reflector, your add-ins may break. You'll want to keep the add-ins you rely on in your browser's favorites list so you can check if an updated version that works with the latest and greatest Reflector has been released. The first add-in is probably the most useful: Denis Bauer Reflector.FileDisassembler (http://www.denisbauer.com/NETTools/FileDisassembler.aspx). Whereas Reflector does a great job decompiling individual methods, what happens if you want to decompile a complete assembly? I guess you could manually copy and paste each method into a code editor, but by the time you finish, .NET will be out of style. With Reflector.FileDisassembler, you'll select the assembly in Reflector's tree control and then choose the File Disassembler from the View menu. Reflector.FileDisassembler asks you which directory you want to decompile the assembly to and what project type it is. What makes Reflector.FileDisassembler so amazingly cool is that it will also produce a Visual Studio .NET 2003 project, which will open with a quick conversion in Visual Studio 2005, so you can compile the source directly. Another add-in I like quite a bit is Reflector Diff AddIn by Sean Hederman at http://www.codingsanity.com/diff.htm. If you have two compiled assemblies you want to compare, this is the add-in for you. If you've ever experienced a situation in which you had a computer crash and you lost the source code for an assembly, this add-in will save your life (not that that's happened to me)! The final add-in I like is Kevin Dente's Running Assembly Reflector Add-in from http://Weblogs.asp.net/kdente/articles/438539.aspx. It's so nice to be able to select Open Running Assembly from the File menu and select directly an assembly from the application it is running in instead of wandering through a thousand directories. The last point I want to address related to Reflector is protecting your intellectual property. Because you can see anything and everything with Reflector, your company managers and lawyers will twist themselves into knots if you show them Reflector. If you have intellectual property you need to protect, consider using an obfuscator. There are numerous obfuscators on the market, but the one I like the best is Brent Rector's Demeaner (http://www.wiseowl.com/). Don't take my word for itwhat do you think Lutz Roeder uses to protect Reflector? When using an obfuscator, it's a one-way street. Once you've obfuscated the Intermediate Language (IL), you can no longer debug the code because the binary no longer matches the PDB file. However, if you have a key algorithm you must protect, you have to do something. If you are going down the obfuscation route, you need to make sure that you've tested your application completely, because once it goes through the obfuscator, it's a black box. Some obfuscators, such as Demeaner, offer tools to help with stack trace decoding, but there's no way to load the obfuscated process into the debugger and start debugging. Sells Brothers' RegexDesigner.NETChris Sell's RegexDesigner.NET (http://www.sellsbrothers.com/tools/#regexd) has saved me thousands of hours. As a regular expression aficionado, having an IDE to test all your regular expressions interactively is a wonderful thing. Additionally, once you finish the regular expression, pressing Ctrl+G will generate the code you can paste into your application. You can find other regular expression testers on the Internet, but I find myself using RegexDesigner.NET far more than any of the others. In fact, several of the directories in the source code for the book contain .rep files for the regular expressions used in the code. Regular expressions have always been one of the black art areas of software, but a single book, Mastering Regular Expressions, Third Edition (O'Reilly, 2006), by Jeffrey E. F. Friedl, will show you the way. It's one of those books that every time I pick it up I learn something. You certainly won't learn anything about regular expressions from the Microsoft documentation, so you need this book. To continue your learning about regular expressions, spend some quality time at the Regular Expression Library Web site (http://www.regexlib.com). There are many excellent examples of regular expressions that numerous people have contributed. Although you might not find the regular expression that does exactly what you need, you'll probably find something quite close that you can tweak. Make sure to look at regular expressions contributed by Michael Ash and Steven Smith because they've both posted some quite complicated examples that show the real power of regular expressions. Windows Installer XML (WiX)Microsoft likes to say that .NET installations are outdated and we'll all be doing XCOPY deployment from now on. I don't know about you, but the only XCOPY deployment I've ever seen was HelloWorld.exe. You simply can't copy real applications because there are database connections, registry settings, and all the pieces necessary to be a modern application. When I started the development for the book's code, I wanted a good installation to handle the code in addition to all the add-ins and Visual Studio extensions that I had planned. After trying a few different tools, I settled on the Windows Installer XML (WiX), the free installer product from Microsoft available at http://wix.sourceforge.net. Having never built a modern installer before, I was able to architect the installation quite quickly with Gábor Deák Jahn's excellent WiX tutorial at http://www.tramontana.co.hu/wix/. Additionally, I received answers to my few remaining questions as a result of subscribing to the WiX-Users mailing list. For searching the archive, use http://www.mail-archive.com/wix-users@lists.sourceforge.net/ because it's better than the SourceForge mailing list viewer. What I like so much about WiX is that it makes the developers responsible for installation by making the installation files themselves easy-to-edit XML files. You don't need expensive licenses for special tools. Now if there's a new file the developer needs to have installed, she simply adds it to the XML file. When I added a new directory to the book's code, I could integrate it into the setup in about three minutes total. When I tried this with other tools, it was far more difficult than I felt it should have been. You can look at the source code to see how I did the installation. The .\Install\MainInstall directory contains the installation for the initial .msi file you'll install to get the book's code on your computer. Each top-level directory has a .wxs file that lists that directory's files. The main install is built to include all the .wxs files that MSBUILD finds in the code directories minus those dedicated to the WintellectToolsInstall.MSI. If you look at .\Install\MainInstall\Debugging.wxs, you'll see the steps in comments to incorporate a new top-level directory into the main install. To see a slightly more complicated installation, the .\Install\WintellectTools directory contains the installation code for WintellectToolsInstall.MSI. One of the advanced items I needed to perform in WintellectToolsInstall.MSI was to determine the version of Visual Studio installed. If the Visual Studio Team Developer or Visual Studio Team Suite is not on the machine, I disable the option to install the Code Analysis Rules. There are plenty of comments in the file, so you can see how it all fits together. To give you an idea how powerful WiX is, consider that Microsoft Office, Visual Studio, SQL Server 2005, and almost everything else coming out of Microsoft these days is using WiX. It's a great toolset and one you definitely want to get familiar with. The faster you push installation tasks down to the developer level, the better install experience you will have for your applications. Other ToolsAlthough there are numerous tools on my hard disk, as does any developer, I use some tools far more than others, so I want to give those tools their due here. One tool that I'm constantly using is the free Notepad2 from Florian Balmer (http://www.flos-freeware.ch/notepad2.html). With syntax highlighting supplied by the excellent Scintilla editing library (http://www.scintilla.org/), it starts up approximately a million times faster than Visual Studio. Of course it doesn't have Microsoft IntelliSense or tooltips, but when you want to look at a source file quickly, it's the perfect tool. Because I started developing on PCs with Microsoft DOS 3.2 (I think I just showed my age!), the command line is part of my DNA. The default cmd.exe that comes with Windows is quite weak compared to JPSoft's excellent 4NT (http://www.jpsoft.com). It's a commercial product, but the time savings you get with its superb command line and extra commands will more than pay for itself within a month. Throw on the enhancements to the batch language and the batch file debugger, and you'll wonder how you ever lived without it. The new Windows PowerShell (http://www.microsoft.com/windowsserver2003/technologies/management/powershell/default.mspx) appears to be very interesting, but I haven't fully gotten into it. The learning curve can be a bit steep, but it's lacking some of the basic command-line behaviors from 4NT and cmd.exe that I'm finding very hard to break. However, I'm willing to bet that by this time next year, I'll be doing quite a bit of work in the PowerShell. If your primary focus is ASP.NET applications or Web Services, seeing the data going back and forth between IIS and the browser is a great way to learn how things work. Additionally, sometimes you want to change the data to test hard to test scenarios. To see every byte running across a system, there's always WireShark (http://www.wireshark.org/), which is the complete network protocol analyzer. It's simply amazing what you can see with WireShark. Since most of us are interested in more the HTTP traffic with our ASP.NET applications, another tool, Fiddler (http://www.fiddlertool.com/fiddler/) is a HTTP Debugging Proxy. With Fiddler, you can change any of the HTTP data easily. There are many features in Fiddler to break on specific data and customize it extensively. My favorite feature of Fiddler is that you can export a .WebTest project directly from it so your logging can be played back with the Visual Studio testing tools. The final tool I want to mention is one of those tools that looks like magic to me. I've written commercial profilers and looked extremely hard at the .NET Profiling API, but I can't figure out how the incredible developers at SciTech were able to make their .NET Memory Profiler (http://memprofiler.com/) so amazingly fast. If you have ever wondered which object is in which garbage-collected generation, this is the tool for you. You can look at your memory in almost any way you want and do it fast. It has overhead, but it's lower than that of any of the other .NET memory-tracking tools. If you have an application that's using a bit too much memory, now you can easily find exactly which objects are bubbling up to generation 2 but being released. Since generation 2 garbage collections don't come around that often, you can end up in situations in which you're using more memory than you need, so you start paging and thrashing as memory is swapped in and out of the paging file. My favorite view of all in .NET Memory Profiler is the real-time view. You can watch your memory usage and see when garbage collections trigger certain operations. Add in comment calls to the .NET Memory Profiler API, and you'll be able to see exactly who's doing what to whom and what memory you're using. The real-time view is worth the price of the entire tool! |
EAN: 2147483647
Pages: 99