Compatibility Testing During Development

   

During a deployment of Windows within a large organization, there might be concerns about the compatibility of in-house applications. These programs are frequently very important to the flow of normal business and critical in the deployment process. The testing ideas presented in this section are useful for verifying these internal applications, but they can also be used as part of a software evaluation process when considering new programs for the organization.

Testing applications for compatibility with any operating system can be a complicated process. Fortunately, the Application Verifier (AppVerifier) tool developed by the Windows Application and Customer Experience group at Microsoft can assist with this task. AppVerifier encompasses several tools that are specifically designed to test for commonly encountered application compatibility issues and some very subtle kernel issues. These tests can also reveal compatibility issues related to the requirements found in the Designed for Windows XP Application Specification.

AppVerifier is a useful tool for identifying some of the common security issues that applications might create in the Windows operating system, such as writing information to incorrect locations within the registry or the file system, where the information can later be modified by a malicious program. Using the Application Verifier will not prevent every possible security or compatibility issue, but it does provide an easy opportunity to avoid and correct the most commonly identified problems.

Using Application Verifier

Perhaps the first thing to note about AppVerifier is that it is not an automated test program for your applications. AppVerifier will attach to a program and perform its tests whenever you run the program. It's possible to use AppVerifier and an automated test procedure simultaneously . AppVerifier attaches a stub , or small piece of code, to the executable program you are testing so that anytime it's run, the AppVerifier tests you have selected will be engaged.

Using AppVerifier to test an application is a relatively simple process. You choose the program files that you want to test. Then choose the tests you want to perform. Some of the tests available to you are the following:

  • Heap corruption detection.

    This test performs regular checks of the heap and adds guard pages at the end of each allocation to catch possible heap overruns.

  • Locks usage checking.

    This test looks for common errors with locks. The output is displayed in a separate debugger application. Note that this test can cause access violations if an error is found.

  • Invalid handle usage detection.

    Checks for common problems with handles. The output is displayed in a separate debugger application. Note that this test can cause access violations if an error is found.

  • Thread stack size checking.

    This test disables stack growth. This will cause a stack overflow exception if the initial allocation was too small.

  • LogStartAndStop.

    This option simply enters log information when the application starts or stops. This helps to make the logs easier to read when reviewing test data.

  • FilePaths.

    This test monitors the application's attempts to obtain file path information to determine whether the program uses hard-coded paths or a nonstandard method of gathering the information. Note that this test can cause the application to crash if an improper method of determining file paths is used.

  • HighVersionLie.

    In the past, many applications were written to run on a single version of Windows. This test will return a very high version number when the application attempts to determine which version of Windows it's running in.

  • RegistryChecks.

    This test monitors the application's use of the system registry for any inappropriate or dangerous calls. Any problems detected will be logged.

After choosing the tests you want to perform, click Options to configure your tests, as shown in Figure 18-2. Then start the application by clicking the Run button in AppVerifier or by starting the program normally. Exercise the application by trying to use all of the functionality in the program to generate the best data for the AppVerifier logs. After closing the application, view the test results in the AppVerifier log file: click View Logs.

Figure 18-2. Select the options you want to use for testing the application.

graphics/f18xo02.jpg

The test settings you specify in AppVerifier for a particular application will remain active anytime you run the program until it is removed from the list of applications in AppVerifier. This helps to run programs repeatedly while working out issues.

The first four tests in AppVerifier look for issues that might be found at the kernel level. Because of this, the best output from these tests can be acquired only with the use of a separate kernel debugger. The kernel tests are designed to generate access violation errors when they encounter an error in the program being tested so that the kernel debugger will break in at precisely that point in the application's execution. If you run an application through AppVerifier without a debugger attached and one of the kernel tests finds an error, the application will appear to crash.

To run the app using a debugger, just set all the options and tests desired in AppVerifier and then launch the application with a debugger according to the directions for that debugger. For example, to debug Myapp.exe with NTSD (the Windows XP system debugger), go to a command line and type ntsd myapp.exe .

Any debugger can be used. The assumption is that the user running the tests is familiar with using a debugger. If you are not comfortable with using a debugger, you should have problems investigated by an experienced developer, who can then run the application with a debugger.

Testing for Logo Compliance

The Designed for Windows logo program identifies products that have been proven to maintain a high level of compatibility with Microsoft Windows. Application Verifier contains several tests that directly relate to the Designed for Windows logo program to make testing easier for every independent software vendor planning to submit a product for the logo. These tests are identified in the user interface by a number at the end of the test name , as shown in Figure 18-3.

Figure 18-3. You can use Application Verifier to test for Windows logo compliance.

graphics/f18xo03.jpg

The numbering used in the interface indicates the specific requirement within the Designed for Windows XP Application Specification that the test setting refers to. For example, the WindowsFileProtection (1.5) test applies to section 1, requirement 5, of the Designed for Windows XP Application Specification, Support Fast User Switching and Remote Desktop, because correct use of system paths is one step toward supporting fast user switching in Windows XP and later.

Using Application Verifier during the development of applications destined for the Designed for Windows logo is strongly suggested. This tool is able to detect approximately 90 percent of the problems that Microsoft finds in products that fail the Designed for Windows logo auditing process. Using Application Verifier as a normal step in your development cycle means that you have eliminated the majority of issues that can block your product from receiving the logo. It also helps to ensure a high-quality user experience for your customers.


   
Top


Introducing Microsoft Windows Server 2003
Introducing Microsoft Windows Server(TM) 2003
ISBN: 0735615705
EAN: 2147483647
Year: 2005
Pages: 153

flylib.com © 2008-2017.
If you may any questions please contact us: flylib@qtcs.net