Killer Switches
|
As I mentioned in Chapter 2, there are some new compiler switches in Microsoft Visual C++ .NET that you should always have turned on. In this section, I'll introduce the /RTCx and /GS switches and explain why they're so important.
The Run-Time Check Switches
If the only feature added to the Visual C++ .NET compiler were the /RTCx (Run-Time Error Checks) switches, I would still tell everyone that it's a mandatory upgrade. As the name implies, the four RTC switches watch your code, and when you have certain errors at run time, the debugger pops up and tells you about them. Figure 17-6 shows an error in which some code overwrote past the end of a local variable. As you can see in the message box, the particular local that was blown is shown as well. The part you can't see is that this message box pops up at the end of the function where the error occurred, so it's trivial to find and fix the problem! The RTC switches, by the way, are allowed only in debug builds.
Figure 17-6: The /RTCs error report
The first switch, /RTCs, whose error report is shown in Figure 17-6, does all sorts of run-time stack checking. It starts by initializing all local variables to nonzero values. This helps you find those pesky release build problems that don't show up in debug builds—specifically the case in which you have an uninitialized pointer variable on the stack. All local variables are filled with 0xCC, the INT 3 (breakpoint) opcode.
The second helpful stack-checking feature is that /RTCs does stack pointer verification around every function call you make to combat against calling convention mismatches. For example, if you declare a function as __cdecl, but it's exported as __stdcall, the stack will be corrupted upon returning from the __stdcall function. If you've been paying attention to compiler switches over the years, you might have guessed that the first two stack-checking options for /RTCs do the same thing as the /GZ switch in Microsoft Visual C++ 6.
Fortunately for us, Microsoft extended the /RTCs switch to also do overrun and underrun checking of all multibyte local variables such as arrays. It does this by adding four bytes to the front and end of those arrays and checking them at the end of the function to ensure those extra bytes are still set to 0xCC. The local checking works with all multibyte locals except those that require the compiler to add padding bytes. In almost all cases, padding is added only when you use the __declspec(align) directive, the /Zp structure member alignment switch, or the #pragma pack(n) directive.
The second switch, /RTCu, checks your variable usage and will pop the warning if you use any without initializing them first. If you've been a loyal Bugslayer over the years, you might be wondering why this switch is important. Since all loyal readers (like you) are already compiling their code with warning level 4 (/W4) and treating all warnings as errors (/WX), you know that compiler warnings C4700 and C4701 will always tell you at compile time where you are definitely using and where you might be using variables without initialization, respectively. With the /RTCu switch, those who aren't will still be told they have bugs in their code. What's interesting about the /RTCu switch is the code to check for uninitialized variables is inserted if the compiler detects a C4700 or C4701 condition. The third switch, /RTC1, is just shorthand for combining /RTCu and /RTCs.
The final switch, /RTCc, checks for data truncation assignments—for example, if you try to assign 0x101 to a char variable. Like the /RTCu, if you're compiling with /W4 and /WX, data truncation will produce a C4244 error at compilation time. If you get an /RTCc error, you have to either mask off the bits you need or cast to the appropriate value. The project Property Pages dialog box, shown in Figure 17-7, allows you to only set /RTCu, /RTCs, or /RTC1 in the Basic Runtime Checks option. In order to turn on /RTCc, you'll need to select the Smaller Type Check option above the Basic Runtime Checks option, as shown in Figure 17-7.
Figure 17-7: Setting the /RTCx switches
At first, I couldn't see why the /RTCc switch was not turned on by the /RTC1. A little thinking showed that /RTCc can show errors on legal C code such as the following:
char LoByte(int a) { return ((char)a) ; } If /RTCc were included in /RTC1, people might think that all the Run-Time Error Checks switches are reporting false positives. However, I vote for always turning on /RTCc because I want to know about any potential problems whenever I run my code.
With the switch descriptions out of the way, I want to turn to the notification you get when you do have an error. When running your programs outside the debugger, the run-time checking code uses the standard C run-time assertion message box. For those of you writing services or code that can't have a user interface, you'll need to redirect the message box using the _CrtSetReportMode function with the _CRT_ASSERT report type parameter. You might think there would be a single, standard way for the /RTCx switches to notify the user, but that's not the case. When running under the debugger, there's a completely different way to do the notifications.
If you happen to look at the Exceptions dialog box in the Visual Studio .NET IDE, you might notice several new classes of exceptions added. The interesting new exception class is Native Run-Time Checks. As you look at it in the Exceptions dialog box, you'll recognize the four different exceptions as matching up with the /RTCx switches. That's your hint that when your program encounters a run-time check while running under the debugger, your program will throw a special exception so that the debugger can handle it.
Controlling Run-Time Check Output
While the default method of output will suffice for many situations, you might want to handle the error output yourself. Listing 17-6 shows a sample custom error handler. The parameter list for run-time check error handlers is a little different in that it takes variable parameters. Evidently, Microsoft is planning to add quite a few different run-time error checks in the future to account for this flexibility. Since your custom handler gets the same parameters as the default version, you can show the errors with variable information and everything else. As you can see from Listing 17-6, it's up to you how you choose to inform the user. The code from Listing 17-6 is with this book's sample files in the RTCHandling sample, so you can play with the error handling all you want.
Listing 17-6: /RTCx custom error reporting
 |
/*///////////////////////////////////////////////////////////////////// FUNCTION : HandleRTCFailure DESCRIPTION : The Run Time Checking (RTC) handler function when NOT running under the debugger. When running under a debugger, this function is ignored. Consequently, you can't debug this puppy at all! PARAMETERS : iError - The type of error as reported through _RTC_SetErrorType. Note that I don't use this parameter. szFile - The source filename where the error occurred. iLine - The line where the error occurred. szModule - The module where the error occurred. szFormat - The printf-style format string for the variable parameters. Note that I don't use this parameter except to get the values out of the variables parameters. ... - The "variable" length parameter list. There are only two values passed here. The first is the RTC Error number. 1 = /RTCc 2 = /RTCs 3 = /RTCu The second is the string that appears when the debugger shows the message. This is only important for /RTCs and /RTCu as those show the variable where the problem happened. RETURN VALUES : TRUE - Cause a _DebugBreak to be called after this function returns. FALSE - Continue execution. /////////////////////////////////////////////////////////////////////*/ // Turn off run time checks for this function so I don't go recursive // on myself. #pragma runtime_checks("", off) // The critical section to protect against reentrancy in the // HandleRTCFailure function. CRITICAL_SECTION g_csRTCLock ; int HandleRTCFailure ( int /*iError*/ , const char * szFile , int iLine , const char * szModule , const char * szFormat , ... ) { // Just say no to reentrant code! EnterCriticalSection ( &g_csRTCLock ) ; // Get the two variable length parameters. I guess they plan on // adding a ton of these RTC checks in the future. va_list vl ; va_start ( vl , szFormat ) ; // The first one is the number of the error that occured. _RTC_ErrorNumber RTCErrNum = va_arg ( vl , _RTC_ErrorNumber ) ; // The second is the additional description of the error. char * szErrorVariableDesc = va_arg ( vl , char * ) ; va_end ( vl ) ; TCHAR szBuff [ 512 ] ; // Get the error description text based off the error number. const char *szErr = _RTC_GetErrDesc ( RTCErrNum ) ; // Make sure szFile and szModule have something in them. if ( NULL == szFile ) { szFile = "Unknown File" ; } if ( NULL == szModule ) { szModule = "Unknown Module" ; } // If it's any error other than /RTCc, I can show some cool info // that includes the variable in question! if ( 1 != RTCErrNum ) { _stprintf ( szBuff , _T ( "%S\n%S\nLine #%d\nFile:%S\nModule:%S" ) , szErr , szErrorVariableDesc , iLine , szFile , szModule ) ; } else { // Build the string. _stprintf ( szBuff , _T ( "%S\nLine #%d\nFile:%S\nModule:%S" ) , szErr , iLine , szFile , szModule ) ; } int iRes = TRUE ; if ( IDYES == MessageBox ( GetForegroundWindow ( ) , szBuff , _T ( "Run Time Check Failure" ) , MB_YESNO | MB_ICONQUESTION ) ) { // Returning 1 means DebugBreak will happen when this // function returns. iRes = 1 ; } else { iRes = 0 ; } // Pop out of the lock. LeaveCriticalSection ( &g_csRTCLock ) ; return ( iRes ) ; } #pragma runtime_checks("", restore) |
Setting your custom error handler function is trivial; just pass it as the parameter to _RTC_SetErrorFunc. There are a few other functions to assist you when handling run-time check errors. The first, _RTC_GetErrDesc, retrieves the description string for a particular error. _RTC_NumErrors returns the total number of errors supported by the current version of the compiler and run time. One function, which I find a little dangerous, is _RTC_SetErrorType. You can use this function to turn off error handling for any or all of the specific run-time checks.
Since the run-time checks rely on the C run time, if your program doesn't use the C run time, you might think you would completely lose the benefits of the RTC switches. If you are wondering why you would ever have a program without the C run time, think about ATL and building with _ATL_MIN_CRT. Without the C run time, you need to call _RTC_Initialize if you've used __MSVC_RUNTIME_CHECKS. You must also provide a function named _CRT_RTC_INIT, which returns your custom error handler.
When I first started playing with a custom output handler, I immediately ran into a small problem. I couldn't debug my handler! If you think about it, you can see why it happens. As I've already discussed, the run-time checking code can determine whether you are running under a debugger and display the output either in the debugger or through the normal C run-time assertion dialog box. When you're running under a debugger, the run-time checking code sees that it's under a debugger and just generates the special exception code to talk to the debugger, completely bypassing your custom output handler.
|
Probably the most common security error as well as an inflection point for bugs is related to handling good old null terminated strings. The problem lies in the original definition of the C run time string handling functions: there's no way to indicate how long a string buffer is. For example, strcpy takes an input buffer and an output buffer and blindly copies the date from the input buffer into the output buffer—no matter that the output buffer might be only half the size. Not only is that a memory overrun but one of the key security holes that virus writers take advantage of to overwrite the return value on the stack in order to get code executing.
You could code review yourself to death looking for these errors, but you'll still miss them. Fortunately, some very bright people at Microsoft realized it was time to make a change. A new library, STRSAFE, rides to the rescue to save your string processing from yourself. STRSAFE is part of the July 2002 Platform SDK and is included with Visual Studio .NET 2003.
The whole idea is to include STRSAFE.H in your precompiled header so that you'll have access to a whole slew of functions that take output buffer lengths and ensure those functions don't copy any more than the specified number of characters. When you first compile after including STRSAFE.H, you'll find that all the error-prone string functions are now deprecated and your builds will stop until you fix them.
Unfortunately, STRSAFE came out after I had written nearly all the code for this book. As you'll find with your projects, retrofitting STRSAFE on your code is much like taking the ride to convert ANSI code to Unicode. It takes quite a bit of time, but it's worth it. If the code is not fully converted by the time the book ships, I'll have it moved over soon and posted on Wintellect's Web site.
|
The Buffer Security Check Switch
The run-time checks are very cool, but another switch that you should always turn on is /GS, the Buffer Security Check switch. Unlike the /RTCx switches, /GS should be turned on for both debug and release builds. The purpose of /GS is to monitor the return address for a function to see whether it's overwritten, which is a common technique used by viruses and Trojan horse programs to take over your application. /GS works by reserving space on the stack before the return address. At the function entry, the function prolog fills in that spot with a security cookie XOR'd with the return address. That security cookie is computed as part of the module load so it's unique to each module. When the function exits, a special function, _security_check_cookie, checks to see whether the value stored at the special spot is the same as it was when entering the function. If the values are different, the code pops up a message box and terminates the program. If you want to see the security code in action, read the source files SECCINIT.C, SECCOOK.C, and SECFAIL.C in the C run-time source code.
As if the security-checking capability of the /GS switch weren't enough, the switch is also a wonderful debugging aid. Even though the /RTCx switches will track numerous errors, a random write to the return address will still sneak through. With the /GS, you get that checking in your debug builds as well. Of course, the Redmondtonians were thinking of us when they wrote the /GS switch, so you can replace the default message box function with your own handler by calling _set_security_error_handler. If you do whack the stack, your handler should call ExitProcess after logging the error.
|
EAN: 2147483647
Pages: 177