Building the Security Test Plan

The test plan itself should outline which application and application components are being tested, what the security assumptions are for each component, what security aspects of each component require testing, and what the expected results are. The process involves the following steps:

Decompose the application into its fundamental components.
Identify the component interfaces.
Rank the interfaces by potential vulnerability.
Ascertain the data used by each interface.
Find security problems by injecting faulty data.

Figure 14-1 shows how you can effectively decompose an application.

Figure 14-1

Decomposing an application into its components and interfaces.

Let s look at each aspect of building a security test plan.

Decompose the Application

The first step of building a security test plan requires you to identify the application structure. Most applications are built from components such as ASP pages or ASP.NET pages, DLLs, EXEs, and object component technologies such as COM and each exposes discrete functionality. In fact, if the component does not export any functionality, why is it in the application?

To itemize the application components, you can usually determine the bill of goods by talking to the application setup people their setup code must copy all the required functional pieces, after all. However, it s not quite as simple as that: the application might reuse components that are already part of the operating system or that require features inherent in other applications. Although it s important that you test your own code, no application resides in a vacuum, and you must consider the security ramifications of your code in the environment. Because of this, you will also need to determine the external components your application uses.

Example technologies used to build applications include the following:

Executable files, including DLLs and EXEs
Services
ISAPI applications and filters
ATL server component
Script files, such as command files (.BAT and .CMD), Microsoft Visual Basic Scripting Edition (VBScript), Microsoft JScript, and Perl
Active Server Pages (ASP) and ASP.NET pages (ASPX)
CGI scripts
HTML pages
COM components

Identify Component Interfaces

The next step is to determine the interfaces exposed by each component. This is possibly the most critical step because exercising the interface code is how you find security bugs. The best place to find what interfaces are exposed by which components is in the functional specifications. Otherwise, ask the developers or read the code. Of course, if an interface is not documented, you should get it documented in the specifications.

Example interfacing and transport technologies include

TCP and UDP sockets
Wireless data
NetBIOS
Mailslots
Dynamic Data Exchange (DDE)
Named Pipes
Shared memory
Other named objects Named Pipes and shared memory are named objects such as semaphores and mutexes
The Clipboard
Local procedure call (LPC) and remote procedure call (RPC) interfaces
COM methods, properties, and events
EXE and DLL functions
System traps and input/output controls (IOCTLs) for kernel-mode components
The registry
HTTP requests
Simple Object Access Protocol (SOAP) requests
Remote API (RAPI), used by Pocket PCs
Console input
Command line arguments
Dialog boxes
Database access technologies, including OLE DB and ODBC
Store-and-forward interfaces, such as e-mail using SMTP, POP, or MAPI, or queuing technologies such as MSMQ
Environment (environment variables)
Files
LDAP sources, such as Active Directory
Hardware devices, such as infrared using Infrared Data Association (IrDA), universal serial bus (USB), COM ports, Firewire, and so on

Rank Interfaces by Their Relative Vulnerability

You need to prioritize which interfaces get tested first, simply because more vulnerable interfaces should be tested thoroughly. The process of determining the relative vulnerability of an interface is to use a simple point-based system. Add up all the points for each interface, based on the descriptions in Table 14-1, and list them starting with the highest number first. Those at the top of the list are most vulnerable to attack, might be susceptible to the most damage, and should be tested more thoroughly.

Table 14-1 Points to Attribute to Interface Characteristics
Interface Characteristic	Points
The process hosting the interface or function runs as a high privileged account such as SYSTEM (Microsoft Windows NT and later) or root (UNIX and Linux systems) or some other account with administrative privileges.	2
The interface handling the data is written in C or C++.	1
The interface takes arbitrary-sized buffers or strings.	1
The recipient buffer is stack-based.	2
The interface has no access control list (ACL) or has weak ACLs.	1
The interface or the resource has good, appropriate ACLs.	-2
The interface does not require authentication.	1
The interface is, or could be, server-based.	1
The feature is installed by default.	1
The feature is running by default.	1
The feature has already had security vulnerabilities.	1

Note that if your list of interfaces is large and you determine that some interfaces cannot be tested adequately in the time frame you have set for the product, you should seriously consider removing the interface from the product and the feature behind the interface. If you can t test it, you can t ship it.

Ascertain Data Used by Each Interface

The next step is to determine the data accessed by each interface. Table 14-2 shows some example interface technologies and where the data comes from. This is the data you will modify to expose security bugs.

Table 14-2 Example Interface Technologies and Data Sources
Interface	Data
Sockets, RPC, Named Pipes, NetBIOS	Data arriving over the network
Files	File contents
Registry	Registry key data
Active Directory	Nodes in the directory
Environment	Environment variables
HTTP data	HTTP headers, form entities, query strings, Multipurpose Internet Mail Extensions (MIME) parts, XML payloads, SOAP data and headers
COM	Method and property arguments
Command line arguments	Data in argv[] for C or C++ applications, data held in WScript.Arguments in Windows Scripting Host (WSH) applications, and the String[] args array in C# applications

Now that we have a fully decomposed functional unit, a ranked list of interfaces used by the components, and the data used by the interfaces, we can start building test cases. The method we ll use is called fault injection.

Find Security Problems by Injecting Faulty Data

The next step is to build test cases to exercise the interfaces by using fault injection. Fault injection involves perturbing the environment such that the code handling the data that enters an interface behaves in an insecure manner. I like to call fault injection lyin and cheatin because your test scripts create bogus situations and fake data designed to make the code fail.

You should build one or more test cases per interface, using the STRIDE threat model for guidance. (For a refresher on STRIDE, see Chapter 2.) For example, if the interface requires authentication, build test scripts to attempt to work around the authentication so as to spoof identity and, potentially, elevate privileges.


	I ll cover some techniques besides fault injection that expose other security issues, such as tampering with data and information disclosure threats, later in this chapter.

The easiest threats to test for are denial of service (DoS) threats, which make the application fail. If your test code can make the application fail and issue an access violation, you ve identified a DoS threat, especially if the application is a networked service.


	The application has suffered a DoS attack if you can make a networked service fail with an access violation. The development team should take these threats seriously, because they will have to fix the bug after the product ships if the defect is discovered.


	Be aware that two kinds of DoS attacks exist. The first, which is easy to test for, causes the application to stop running because of an access violation or similar event. In the second case, which is not easy to test for because it requires a great deal of hardware and preparation, an application fails slowly and response times get worse as the system is attacked by many machines in a distributed testing manner.

Figure 14-2 shows techniques for perturbing an application s environment.

Figure 14-2

Techniques to perturb applications to reveal security vulnerabilities and reliability bugs.

When designing security tests, keep this diagram close at hand. It will help you determine which test conditions you need to create. Let s look at each category.

The Data and the Container

Data often exists in containers for example, a file contains bytes of data. You can create security problems by perturbing the data or the container (the file itself). Changing the name of the container, in this case the filename, is changing the container but not the data itself. Generally, on-the-wire data does not have a container, unless you consider the network to be the container. I ll leave that philosophical conundrum to you!

Perturbing the container

You can perturb a container in a number of ways. You can deny access to the container; this is easily achieved by setting the Deny access control entry (ACE) on the object prior to running the test. Restricted access is somewhat similar to no access. For example, the application requires read and write access to the object in question, but an ACE on the object allows only read access. Some resources, such as files, can have their access restricted using other techniques. In Windows, files have attributes associated with them, such as read-only.

Another useful test relies on the application assuming the resource already exists or not. Imagine that your application requires that a registry key already exist how does the application react if the key does not exist? Does it take on an insecure default?

Finally, how does the application react if the container exists but the name is different? A special name case, especially in UNIX, is the link problem. How does the application react if the name is valid but the name is actually a link to another file? Refer to Chapter 8, Canonical Representation Issues, for more information about symbolic links and hard links in UNIX and Windows.

Note in Figure 14-2 the link from the container name to the data section. This link exists because you can do nasty things to container names too, such as change the size of the name or the contents of the container name. For example, if the application expects a filename like Config.xml, what happens if you make the name overly long (shown in Figure 14-2 as Data-Size-Long), such as Myreallybigconfig.xml, or too short (Data-Size-Short), such as C.xml? What happens if you change the name of the file (that is, the contents of the container name) to something random (Data-Contents-Random), like rfQy6-J.87d?

Perturbing the data

Data has two characteristics: the nature of the contents and the size of the data. Each is a potential target and should be maliciously manipulated. Applications can take two types of input: correctly formed data and incorrectly formed data. Correctly formed data is just that it s the data your application expects. Such data rarely leads to the application raising errors and is of little interest to a security tester. Incorrectly formed data has numerous variations. Let s spend some time looking at each in detail.

Random data

Random data is a series of arbitrary bytes sent to the interface, or written to a data source, that is then read by the interface. In my experience, utterly incorrect data, although useful, does not help you find as many security holes as partially incorrect data, because many applications perform some degree of data input validation.

To create a buffer full of utterly random but printable data in Perl, you can use the following code:

srand time; my $size = 256; my @chars = ( A .. Z , a .. z , 0..9, qw( ! @ # $ % ^ & * - + = )); my $junk = join ( ", @chars[ map{rand @chars } (1 .. $size)]);

In C or C++, you can use CryptGenRandom to populate a user-supplied buffer with random bytes. Refer to Chapter 6, Cryptographic Foibles, for example code that generates random data. If you want to use CryptGenRandom to create printable random data, use the following C++ code:

/* * PrintableRand.cpp */ #include windows.h" #include wincrypt.h" DWORD CreateRandomData(LPBYTE lpBuff, DWORD cbBuff, BOOL fPrintable) { DWORD dwErr = 0; HCRYPTPROV hProv = NULL; if (CryptAcquireContext(&hProv, NULL, NULL, PROV_RSA_FULL, CRYPT_VERIFYCONTEXT) == FALSE) return GetLastError(); ZeroMemory(lpBuff, cbBuff); if (CryptGenRandom(hProv, cbBuff, lpBuff)) { if (fPrintable) { char *szVal abcdefghijklmnopqrstuvwxyz" 0123456789" ~`!@#$%^&*()_-+={}[];: <>,.? \\/"; DWORD cbValid = lstrlen(szValid); // Convert each byte (0-255) to a different byte // from the list of valid characters above. // There is a slight skew because strlen(szValid) is not // an exact multiple of 255. for (DWORD i=0; i<cbBuff; i++) lpBuff[i] = szValid[lpBuff[i] % cbValid]; // Close off the string if it s printable. // The data is not zero-terminated if it s not printable. lpBuff[cbBuff-1] = \0 ; } } else { dwErr = GetLastError(); } if (hProv != NULL) CryptReleaseContext(hProv, 0); return dwErr; } void main(void) { BYTE bBuff[16]; CreateRandomData(bBuff, sizeof bBuff, FALSE); }

You can also find this sample code on the companion CD in the folder Secureco\Chapter 14\PrintableRand. The real benefit of using this kind of test one that uses junk data is to find certain buffer overrun types. The test is useful because you can simply increase the size of the buffer until the application fails or, if the code is robust, continues to execute correctly with no error. The following Perl code will build a buffer that continually increases in size:

# Note the use of _ in $MAX below. # I really like this method of using big numbers. # They are more readable! 128_000 means 128,000. # Cool, huh? my $MAX = 128_000; for (my $i=1; $i < $MAX; $i *= 2) { my $junk = A x $i; # Send $junk to the data source or interface. }


	Sometimes it s difficult to determine whether a buffer overrun is really exploitable. Therefore, it s better to be safe and just fix any crash caused by a long string.

Probably the most well-known work on this kind of random data is a paper titled Fuzz Revisited: A Re-examination of the Reliability of UNIX Utilities and Services by Barton P. Miller, et al, which focuses on how certain applications react in the face of random input. The paper is available at citeseer.nj.nec.com/2176.html. The findings in the paper were somewhat alarming:

Even worse is that some of the bugs we reported in 1990 are still present in the code releases of 1995. The failure rate of utilities on the commercial versions of UNIX that we tested (from Sun, IBM, SGI, DEC and NEXT) ranged from 15-43%.

Chances are good that some of your code will also fail in the face of random garbage. But although Fuzz testing is useful and should be part of your test plans, it won t catch many classes of bugs. If random data is not so useful, what is? The answer is partially incorrect data.

Partially incorrect data

Partially incorrect data is data that s accurately formed but that might contain invalid values. It takes more work to build such data because it requires better knowledge of the data structures used by the application. For example, if a Web application requires a specific (fictitious) header type, TIMESTAMP, setting the data expected in the header to random data is of some use, but it will not exercise the TIMESTAMP code path greatly if the code checks for certain values in the header data. Let s say the code checks to verify that the timestamp is a numeric value and your test code sets the timestamp to a random series of bytes. Chances are good that the code will reject the request before much code is exercised. Hence, the following header is of little use:

TIMESTAMP: H7ahbsk(0kaaR

But the following header will exercise more code because the data is a valid number and will get past the initial validity-checking code:

TIMESTAMP: 09871662

This is especially true of RPC interfaces compiled using the /robust Microsoft Interface Definition Language (MIDL) compiler switch. If you send random data to an RPC interface compiled with this option, the packet won t make it beyond the validation code in the server stub. The test is worthless. But if you can correctly build valid RPC packets with subtle changes, the packet might find its way into your code and you can therefore exercise your code and not MIDL-generated code.


	The /robust MIDL compiler switch does not mitigate all malformed data issues. Imagine an RPC function call that takes a zero-terminated string as an argument but your code expects that the data be numeric only. There is no way for the RPC marshaller to know this. Therefore, it s up to you to test your interfaces appropriately by testing for wrong data types.

Let s look at another example a server listening on port 1777 requires a packed binary structure that looks like this in C++:

#define MAX_BLOB (128) typedef enum { ACTION_QUERY, ACTION_GET_LAST_TIME, ACTION_SYNC} ACTION; typedef struct { ACTION actAction; // 2 bytes short cbBlobSize; // 2 bytes char bBlob[MAX_BLOB]; // 128 bytes } ACTION_BLOB;

However, if the code checks that actAction is 0, 1, or 2 which represent ACTION_QUERY, ACTION_GET_LAST_TIME, or ACTION_SYNC, respectively and fails the request if the structure variable is not in that range, the test will not exercise much of the code when you send a series of 132 random bytes to the port. So, rather than sending 132 random bytes, you should create a test script that builds a correctly formed structure and populates actAction with a valid value but sets cbBlobSize and bBlob to random data. The following Perl script shows how to do this. This script is also available on the companion CD in the folder Secureco\Chapter 14.

# PackedStructure.pl # This code opens a TCP socket to # a server listening on port 1777; # sets a query action; # sends MAX_BLOB letter A s to the port. use IO::Socket; my $MAX_BLOB = 128; my $actAction = 0; # ACTION_QUERY my $bBlob = A x $MAX_BLOB; my $cbBlobSize = 128; my $server = 127.0.0.1 ; my $port = 1777; if ($socks = IO::Socket::INET->new(Proto=>"tcp", PeerAddr=>$server, PeerPort => $port, TimeOut => 5)) { my $junk = pack ssa128",$actAction,$cbBlobSize,$bBlob; printf Sending junk to $port (%d bytes)", length $junk; $socks->send($junk); }


	All the Perl samples in this chapter were created and executed using ActiveState Perl 5.6.1 from www.activestate.com.

Note the use of the pack function. This Perl function takes a list of values and creates a byte stream by using rules defined by the template string. In this example, the template is "ssa128", which means two signed short integers (the letter s twice) and 128 arbitrary characters (the a128 value). The pack function supports many data types, including Unicode and UTF-8 strings, and little endian and big endian words. It s a useful function indeed.


	The pack function is very useful if you want to use Perl to build test scripts to exercise binary data.

The real fun begins when you start to send overly large data structures. This is a wonderful way to test code that handles buffers, code that has in the past led to many serious buffer overrun security vulnerabilities.

Taking it further use different sizes

You can have much more fun with the previous example Perl code because the structure includes a data member that determines the length of the buffer to follow. This is common indeed; many applications that support complex binary data have a data member that stores the size of the data to follow. To have some real fun with this, why not lie about the data size? Look at the following single line change from the Perl code noted earlier:

my $cbBlobSize = 256; # Lie about blob size.

This code sets the data block size to 256 bytes. However, only 128 bytes is sent, and the server code assumes a maximum of MAX_BLOB (128) bytes. This might make the application fail with an access violation if it attempts to copy 256 bytes to a 128-byte buffer, when half of the 256 bytes is missing. Or you could send 256 bytes and set the blob size to 256 also. The code might copy the data verbatim, even though the buffer is only 128 bytes in size. Another useful trick is to set the blob size to a huge value, as in the following code, and see whether the server allocates the memory blindly. If you did this enough times, you could exhaust the server s memory. Once again, this is another example of a great DoS attack.

my $cbBlobSize = 256_000; # Really lie about blob size.

I once reviewed an application that took usernames and passwords and cached them for 30 minutes, for performance reasons. The cache was an in-memory cache, not a file. However, there was a bug: if an attacker sent a bogus username and password, the server would cache the data and then reject the request because the credentials were invalid. However, it did not flush the cache for another 30 minutes. So an attacker could send thousands of elements of invalid data, and eventually the service would stop or slow to a crawl as it ran out of memory. The fix was simply not caching anything until credentials were validated. I also convinced the application team to reduce the cache time-out to 15 minutes.

If the code does fail, take special note of the value in the instruction pointer (EIP) register. If the register contains data from the buffer you provided in this case, a series of As the return address on the stack has been overwritten, making the buffer overrun exploitable. Figure 14-3 shows the EIP register after some code copied a buffer of As, creating an overrun. Notice its value in 0x41414141; 41 is the hexadecimal value of the ASCII character A.

Figure 14-3

The Registers dialog box in the Microsoft Visual C++ 6 debugger showing a corrupted EIP register a sure sign of an exploitable buffer overrun.

What s the EIP Register?

When function A calls function B, the next address to execute once function B returns is placed on the stack. When function B returns, the CPU takes the address off the stack and places it in the EIP register, the instruction pointer. The address held in EIP determines at what address code execution should continue.

Partially incorrect data is powerful and can uncover many bugs, but it does take work to construct good test scripts, because knowledge of the data structure is required.


	A useful test case is to perturb file and registry data read by code that expects the data to be no greater than MAX_PATH bytes or Unicode characters in length. MAX_PATH, which is defined in many Windows header files, is set to 260.

Special-Case Data

Three special-case data types can be used: NULL data, zero-length data, and data that is the wrong type. Many of these work only with certain technologies. For example, SQL includes NULL data, which is nonexistent or unknown, and zero-length data, which is an empty string. Also, SQL tables contain columns, and columns have data types. You might uncover some security bugs with these special case data, but you ll mostly find DoS bugs because a server did not handle the data and failed.


	You could consider that Unicode and ANSI characters are different data string types. Your test plans should use ANSI strings where Unicode is expected and vice versa.

Other special cases exist that relate only to data on-the-wire: replayed data, out-of-sync data arrival, and data flooding. The first case can be quite serious. If you can replay a data packet or packets and gain access to some resource, or if you can make an application grant you access when it should not, you have a reasonably serious error that needs to be fixed. For example, if your application has some form of custom authentication that relies on a cookie, or some data held in a field that determines whether the client has authenticated itself, replaying the authentication data might grant others access to the service, unless the service takes steps to mitigate this.

Out-of-sync data involves sending data out of order. Rather than sending Data₁, Data₂, and Data₃ in order, the test application sends them in an incorrect order, such as Data₁, Data₃, and Data₂. This is especially useful if the application performs some security check on Data₁, which allows Data₂ and Data₃ to enter the application unchecked. Some firewalls have been known to do this.

Finally, here s one of the favorite attacks on the Internet: simply swamping the service with so much data, or so many requests, that it becomes overwhelmed and runs out of memory or some other restricted resource and fails. Performing such stress testing often requires a number of machines and multithreaded test tools. This somewhat rules out Perl (which has poor multithreaded support), leaving C/C++ and specialized stress tools.


	A useful tool for fault injection, especially if you re not a tester who does much coding, is the ClickToSecure, Inc., product named Hailstorm. This tool allows a tester to construct arbitrarily complex data to send to various networking interfaces. It also supports data flooding. You can find more information about the tool at www.clicktosecure.com.

Before Testing

You need to set up application monitoring prior to running any test. Most notably, you should hook up a debugger to the machine in case the application breaks. Don t forget to use Performance Monitor to track application memory and handle usage. If the application fails or memory counts or handle counts increase, attackers could also make the application fail, denying service to others.


	Other tools to use include Gflags.exe, available on the Windows 2000 and Windows .NET CDs, which allows you to set system heap options; Oh.exe, which shows handle usages; and dh.exe, which shows process heap usage. The second and third of these tools are available in the Windows 2000 and Windows .NET resource kits.


	If the application performs exception handling, you might not see any errors occur in the code unless you have a debugger attached. Why? When the error condition occurs, it is caught by the exception-handling code and the application continues operation. If a debugger is attached, the exception is passed to the debugger first.

Also, use the event log, because you might see errors appear there, especially if the application is a service. Many services are configured to restart on failure.

Building Tools to Find Flaws

Finally, you need to build tools to test the interfaces to find flaws. There is a simple rule you should follow when choosing appropriate testing tools and technologies: use a tool that slips under the radar. Don t use a tool that correctly formats the request for you, or you might not test the interface correctly. For example, don t use Visual Basic to exercise low-level COM interfaces because the language will always correctly form strings and other data structures. The whole point of performing security testing by using fault injection is to create data that is invalid.


	If a security vulnerability is found in your code or in a competitor s code by an external party and the external party makes exploit code available, use the code in your test plans. You should run the code as regularly as you run other test scripts.

I ve already discussed some ways to build fault-injection data. Now we need to look at how to get the data to the interfaces. In the next few sections, I ll look at some ways to test various interface types.

Testing Sockets-Based Applications

I ve already shown Perl-based test code that accesses a server s socket and sends bogus data to the server. Perl is a great language to use for this because it has excellent socket support and gives you the ability to build arbitrarily complex binary data by using the pack function. You could certainly use C++, but if you do, I d recommend you use a C++ class to handle the socket creation and maintenance. Your job is to create bogus data, not to worry about the lifetime of a socket. One example is the CSocket class in the Microsoft Foundation Classes (MFC). C# and Visual Basic .NET are also viable options. In fact, I prefer to use C# and the System.Net.Sockets namespace. You get ease of use, a rich socket class, memory management, and threading. Also, the TcpClient and TcpServer classes help by providing much of the plumbing for you.

Testing HTTP-Based Applications

To test HTTP-based applications, once again I would use Perl for a number of reasons, including Perl s excellent socket support, HTTP support, and user-agent support. You can create a small Perl script that behaves like a browser but is in fact Perl; it takes care of some of the various headers that are sent during a normal HTTP request. The following sample code shows how to create an HTTP form request that contains invalid data in the form. The Name, Address, and Zip fields all contain long strings. The code sets a new header in the request, Timestamp, to a bogus value too.

# SmackPOST.pl use HTTP::Request::Common qw(POST GET); use LWP::UserAgent; # Set the user agent string. my $ua = LWP::UserAgent->new(); $ua->agent( HackZilla/v42.42 WindowsXP ); # Build the request. my $url = http://127.0.0.1/form.asp"; my $req = POST $url, [Name => A x 128, Address => B x 256, Zip => C x 128]; $req->push_header( Timestamp: => 1 x 10); my $res = $ua->request($req); # Get the response. # $err is the HTTP error and $_ holds the HTTP response data. my $err = $res->status_line; $_ = $res->as_string; print Error! if (/Illegal Operation/ig $err != 200);

This code is also available on the companion CD in the folder Secureco\Chapter 14. As you can see, the code is small because it uses various Perl modules, Library for WWW access in Perl (LWP), and HTTP to perform most of the underlying work, while you get on with creating the malicious content.

Here s another variation. In this case, the code exercises an ISAPI handler application, test.dll, by performing a GET operation, setting a large query string in the URL, and setting a custom header (Test-Header) handled by the application, made up of the letter H repeated 256 times, followed by carriage return and line feed, which in turn is repeated 128 times. The following code is also available on the companion CD in the folder Secureco\Chapter 14:

# SmackQueryString.pl use LWP::UserAgent; $bogushdr = ( H x 256) . \n\r ; $hdr = new HTTP::Headers(Accept => text/plain , User-Agent => HackZilla/42.42 , Test-Header => $bogushdr x 128); $urlbase = http://localhost/test.dll?data= ; $data = A x 16_384; $url = new URI::URL($urlbase . $data); $req = new HTTP::Request(GET, $url, $hdr); $ua = new LWP::UserAgent; $resp = $ua->request($req); if ($resp->is_success) { print $resp->content; } else { print $resp->message; }

Also consider using the .NET Framework HttpGetClientProtocol or HttpPostClientProtocol classes to build test applications. Like HTTP::Request::Common in Perl, these two classes handle much of the low-level protocol work for you.

The buffer overrun in Microsoft Index Server 2.0, which led to the CodeRed worm and is outlined in Unchecked Buffer in Index Server ISAPI Extension Could Enable Web Server Compromise at www.microsoft.com/technet/ security/bulletin/MS01-033.asp, could have been detected if this kind of test had been used. The following scripted URL will make an unpatched Index Server fail. Note the large string of As.

$url = http://localhost/NULL.ida? . ( A x 260) . =Z ;

Testing Named Pipes Applications

Perl includes a Named Pipe class, Win32::Pipe, but frankly, the code to write a simple Named Pipe client in C++ is small. And, if you re using C++, you can call the appropriate ACL and impersonation functions when manipulating the pipe, which is important. You can also write a highly multithreaded test harness, which I ll discuss in the next section.

Testing COM, DCOM, ActiveX, and RPC Applications

To help you test, draw up a list of all methods, properties, events, and functions, as well as any return values of all the COM, DCOM, ActiveX, and RPC applications. The best source of information for this is not the functional specifications, which are often out of date, but the appropriate Interface Definition Language (IDL) files.

Assuming you have compiled the RPC server code by using the /robust compiler switch refer to the RPC information in Chapter 10, Securing RPC, ActiveX Controls, and DCOM, if you need reminding why using this option is a good thing you ll gain little from attempting to send pure junk to the RPC interface, because the RPC and DCOM run times will reject the data unless it exactly matches the definition in the IDL file. In fact, if you do get a failure in the server-side RPC run time, please file a bug with Microsoft! So, you should instead exercise the function calls, methods, and properties by setting bogus data on each call from C++. After all, you re trying to exercise your code, not the RPC run-time code. Follow the ideas laid out in Figure 14-2.

For low-level RPC and DCOM interfaces that is, those exposed for C++ applications, rather than scripting languages consider writing a highly multithreaded application, which you run on multiple computers, and stress each function or method to expose possible timing issues, race conditions, multithread design bugs, and memory or handle leaks.

If your application supports automation that is, if the COM component supports the IDispatch interface you can use C++ to set random data in the function calls themselves, or you can use any scripting language to set long data and special data types.

Remember that ActiveX controls can often be repurposed unless they are tied to the originating domain. If you ship one or more ActiveX controls, consider the consequence of using the control beyond its original purpose.

Testing File-Based Applications

You need to test in a number of ways when handling files, depending on what your application does with a file. For example, if the application creates or manipulates a file or files, you should follow the ideas in Figure 14-2, such as setting invalid ACLs, precreating the file, and so on. The really interesting tests come when you create bogus data in the file and then force the application to load the file. The following simple Perl script creates a file named File.txt, which is read by Process.exe. However, the Perl script creates a file containing a series of 0 to 32,000 As and then loads the application.

my $FILE = file.txt"; my $exe = program.exe"; my @sizes = (0,256,512,1024,2048,32000); foreach(@sizes) { printf Trying $_ bytes\n"; open FILE, > $FILE or die $!\n"; print FILE A x $_; close FILE; # Note the use of backticks like calling system(). `$exe $FILE`; }

If you want to determine which files are used by an application, you should consider using FileMon from www.sysinternals.com.

Testing Registry-Based Applications

Registry applications are simple to test, using the Win32::Registry module in Perl. Once again, the code is short and simple. The following example sets a string value to 1000 As and then launches an application, which loads the key value:

use Win32::Registry; my $reg; $::HKEY_LOCAL_MACHINE->Create( SOFTWARE\\AdvWorks\\1.0\\Config",$reg) or die $^E"; my $type = 1; # string my $value = A x 1000; $reg->SetValueEx( SomeData","",$type,$value); $reg->Close(); `process.exe`;

Or, when using VBScript and the Windows Scripting Host, try

Set oShell = WScript.CreateObject( WScript.Shell ) strReg = HKEY_LOCAL_MACHINE\SOFTWARE\AdvWorks\1.0\Config\NumericData" oShell.RegWrite strReg, 32000, REG_DWORD" Execute process.exe, 1=active window. True means waiting for app to complete. iRet = oShell.Run( process.exe", 1, True) WScript.Echo process.exe returned & iRet

Don t forget to clean up the registry between test passes. If you want to determine which registry keys are used by an application, consider using RegMon from www.sysinternals.com.


	You might not need to thoroughly test all securable objects including files in NTFS file system (NTFS) partitions and the system registry for security vulnerabilities if the ACLs in the objects allow only administrators to manipulate them. This is another reason for using good ACLs they help reduce test cases.

Testing Command Line Arguments

No doubt you can guess how to test command line applications based on the previous two Perl examples. Simply build a large string and pass it to the application by using backticks, like so:

my $arg= A x 1000; `process.exe -p $args`; $? >>= 8; print process.exe returned $?";

Of course, you should test all arguments with invalid data. And in each case the return value from the executable, held in the $? variable, should be checked to see whether the application failed. Note that the exit value from a process is really $? >>8, not original $?.

The following sample code will exercise all arguments randomly and somewhat intelligently in that it knows the argument types. You should consider using this code as a test harness for your command line applications and adding new argument types and test cases to the handler functions. You can also find this code on the companion CD in the folder Secureco\Chapter 14.

# ExerciseArgs.pl # Change as you see fit. my $exe = process.exe"; my $iterations = 100; # Possible option types my $NUMERIC = 0; my $ALPHANUM = 1; my $PATH = 2; # Hash of all options and types # /p is a path, /i is numeric, and /n is alphanum. my %opts = ( p => $PATH, i => $NUMERIC, n => $ALPHANUM); # Do tests. for (my $i = 0; $i < $iterations; $i++) { print Iteration $i"; # How many args to pick? my $numargs = 1 + int rand scalar %opts; print ($numargs args) ; # Build array of option names. my @opts2 = (); foreach (keys %opts) { push @opts2, $_; } # Build args string. my $args = "; for (my $j = 0; $j < $numargs; $j++) { my $whicharg = @opts2[int rand scalar @opts2]; my $type = $opts{$whicharg}; my $arg = "; $arg = getTestNumeric() if $type == $NUMERIC; $arg = getTestAlphaNum() if $type == $ALPHANUM; $arg = getTestPath() if $type == $PATH; # arg format is / argname : arg # examples: /n:test and /n:42 $args = $args . / . $whicharg . :$arg"; } # Call the app with the args. `$exe $args`; $? >>= 8; printf $exe returned $?\n"; } # Handler functions # Return a numeric test result; # 10% of the time, result is zero. # Otherwise it s a value between -32000 and 32000. sub getTestNumeric { return rand > .9 ? 0 : (int rand 32000) - (int rand 32000); } # Return a random length string. sub getTestAlphaNum { return A x rand 32000; } # Return a path with multiple dirs, of multiple length. sub getTestPath { my $path="c:\\"; for (my $i = 0; $i < rand 10; $i++) { my $seg = a x rand 24; $path = $path . $seg . \\"; } return $path; }

In Windows, it s rare for a buffer overrun in a command line argument to lead to serious security vulnerabilities, because the application runs under the identity of the user. But such a buffer overrun should be considered a code-quality bug. On UNIX and Linux, command line buffer overruns are a serious issue because applications can be configured by a root user to run as a different, higher-privileged identity, usually root, by setting the SUID (set user ID) flag. Hence, a buffer overrun in an application marked to run as root could have disastrous consequences even when the code is run by a normal user. One such example exists in Sun Microsystems Solaris 2.5, 2.6, 7, and 8 operating systems. A tool named Whodo, which is installed as setuid root, had a buffer overrun, which allowed an attacker to gain root privileges on Sun computers. Read about this issue at www.securityfocus.com/bid/2935.

Testing XML Payloads

As XML becomes an important payload, it s important that code handling XML payloads is tested thoroughly. Following Figure 14-2 (on page 372), you can exercise XML payloads by making tags too large or too small or by making them from invalid characters. The same goes for the size of the XML payload itself make it huge or nonexistent. Finally, you should focus on the data itself. Once again, follow the guidelines in Figure 14-2.

You can build malicious payloads by using Perl modules, .NET Framework classes, or the Microsoft XML document object model (DOM). The following example builds a simple XML payload by using JScript and HTML. I used HTML because it s a trivial task to build the test code around the XML template. This code fragment is also available on the companion CD in the folder Secureco\Chapter 14.

<!-- BuildXML.html --> <XML > <user> <name/> <title/> <age/> </user> </XML> <SCRIPT> // Build long strings // for use in the rest of the test application. function createBigString(str, len) { var str2 = new String(); for (var i = 0; i < len; i++) str2 += str; return str2; } var user = template.XMLDocument.documentElement; user.childNodes.item(0).text = createBigString( A", 256); user.childNodes.item(1).text = createBigString( B", 128); user.childNodes.item(2).text = Math.round(Math.random() * 1000); var oFS = new ActiveXObject( Scripting.FileSystemObject ); var oFile = oFS.CreateTextFile( c:\\temp\\user.xml ); oFile.WriteLine(user.xml); oFile.Close(); </SCRIPT>

View the XML file once you ve created it and you ll notice that it contains large data items for both name and title and that age is a random number. You could also build huge XML files containing thousands of entities.

If you want to send the XML file to a Web service for testing, consider using the XMLHTTP object. Rather than saving the XML data to a file, you can send it to the Web service with this code:

var oHTTP = new ActiveXObject( Microsoft.XMLHTTP ); oHTTP.Open( POST", http://localhost/PostData.htm", false); oHTTP.send(user.XMLDocument);

Building XML payloads by using the .NET Framework is trivial. The following sample C# code creates a large XML file made of bogus data. Note that getBogusISBN and getBogusDate are left as an exercise for the reader!

static void Main(string[] args) { string file = @"c:\1.xml"; XmlTextWriter x = new XmlTextWriter(file, Encoding.ASCII); Build(ref x); // Do something with the XML file. } static void Build(ref XmlTextWriter x) { x.Indentation = 2; x.Formatting = Formatting.Indented; x.WriteStartDocument(true); x.WriteStartElement( books", ); for (int i = 0; i < new Random.Next(1000); i++) { string s = new String( a , new Random().Next(10000)); x.WriteStartElement( book", ); x.WriteAttributeString( isbn", getBogusISBN()); x.WriteElementString( title", ", s); x.WriteElementString( pubdate", ", getBogusDate()); x.WriteElementString( pages", ", s); x.WriteEndElement(); } x.WriteEndElement(); x.WriteEndDocument(); x.Close(); }

Some in the industry claim that XML will lead to a new generation of security threats, especially in cases of XML containing script code. I think it s too early to tell, but you d better make sure your XML-based applications are well-written and secure, just in case! Check out one point of view at www.computerworld.com/ rckey259/story/0,1199,NAV63_STO61979,00.html.

Testing SOAP Services

Essentially, a SOAP service is tested with the same concepts that are used to test XML and HTTP SOAP is XML over HTTP, after all! The following sample Perl code shows how you can build an invalid SOAP request to launch at the unsuspecting SOAP service. This sample code is also available on the companion CD in the folder Secureco\Chapter 14.


	SOAP can be used over other transports, such as SMTP and message queues, but HTTP is by far the most common protocol.

# TestSoap.pl use HTTP::Request::Common qw(POST); use LWP::UserAgent; my $ua = LWP::UserAgent->new(); $ua->agent( SOAPWhack/1.0 ); my $url = http://localhost/MySOAPHandler.dll ; my $iterations = 10; # Used by coinToss my $HEADS = 0; my $TAILS = 1; open LOGFILE, >>SOAPWhack.log or die $!; # Some SOAP actions - add your own, and junk too! my @soapActions=( , junk , foo.sdl ); for (my $i = 1; $i <= $iterations; $i++) { print SOAPWhack: $i of $iterations\r"; # Choose a random action. my $soapAction = $soapActions[int rand scalar @soapActions]; $soapAction = S x int rand 256 if $soapAction eq junk ; my $soapNamespace = http://schemas.xmlsoap.org/soap/envelope/"; my $schemaInstance = http://www.w3.org/2001/XMLSchema-instance"; my $xsd = http://www.w3.org/XMLSchema"; my $soapEncoding = http://schemas.xmlsoap.org/soap/encoding/"; my $spaces = coinToss() == $HEADS ? : x int rand 16384; my $crlf = coinToss() == $HEADS ? \n : \n x int rand 256; # Make a SOAP request. my $soapRequest = POST $url; $soapRequest->push_header( SOAPAction => $soapAction); $soapRequest->content_type( text/xml ); $soapRequest->content( <soap:Envelope . $spaces . xmlns:soap=\" . $soapNamespace . \ xmlns:xsi=\" . $schemaInstance . \ xmlns:xsd=\" . $xsd . \ xmlns:soapenc=\" . $soapEncoding . \"><soap:Body> . $crlf . </soap:Body></soap:Envelope> ); # Perform the request. my $soapResponse = $ua->request($soapRequest); # Log the results. print LOGFILE [SOAP Request]"; print LOGFILE $soapRequest->as_string . \n"; print LOGFILE [WSDL response]"; print LOGFILE $soapResponse->status_line . ; print LOGFILE $soapResponse->as_string . \n"; } close LOGFILE; sub coinToss { return rand 10 > 5 ? $HEADS : $TAILS; }

When you build code like this, you should consider changing settings, such as the following:

Setting boundary condition values for parameter data types. For example, if an application requires a value in the range 0 10, try -1 and 11.
Various UTF-8 and Unicode settings. Refer to Chapter 12, Securing Web-Based Services, for ideas.
URL canonicalization in the Web service URL. Refer to Chapter 12 for ideas.
Various malformed, missing, and invalid Accept-Charset, Accept-Encoding, Accept-Language, Content-Encoding, and Content-Language values.
Unescaped and escaped XML entities in values.
Type mismatches int instead of string, for example.
Malformed fragments content length says 2K, but it s really 1K, for example.
Extraneous headers.
Binary, not text, garbage.
Extremely large payloads.
Case variations. (XML is case-sensitive.)
Extra SOAP headers.
Nonexistent SOAP methods.
Using too many parameters in a SOAP method.
Too few parameters in a SOAP method.

Finally, you could also use the .NET Framework class SoapHttpClientProtocol to build multithreaded test harnesses.

Testing for Cross-Site Scripting and Script-Injection Bugs

In Chapter 12, I discussed cross-site scripting and the dangers of accepting user input. In this section, I ll show you how to test whether your Web-based code is susceptible to some forms of scripting attacks. The methods here won t catch all of them, so you should get some ideas, based on some attacks, from Chapter 12 to help build test scripts.

Identify all points of input into a Web service, and set every field, header, or query string to include some script. For example,

><script>alert(window.location);</script>

The input strings used depend on where the server uses the input text when creating output. The text might wind up in the middle of a script block already inside a string. So something like the following would work:

 ; alert(document.cookie);

Or your text could be placed inside an attribute, and if the application filters the < and > characters that surround the attribute, you won t see the injection. So something like this would work:

 onmouseover= alert(document.cookie);

If the text is placed in an attribute and the code does not filter the < and > characters, you could try this:

 ><script>alert(document.cookie);</script>

But maybe there s a tag you need to close off first, so use something akin to this:

 ></a><script>alert(document.cookie);</script>

Finally, add one or more carriage returns to the input some Web sites don t scan input across multiple lines.

If a dialog box appears in the browser with the directory or URL of the HTML page or with a cookie value, you have a potential cross-site scripting bug that needs fixing. Refer to Chapter 12 for remedies.

The following Perl script works by creating input for a form and looking for the returned text from the Web page. If the output contains the injected script, you should investigate the page because the page might be susceptible to cross-site scripting vulnerabilities. Note that this code will not find all issues. The cross-site scripting vulnerability might not appear in the resulting page it might appear a few pages away. So you need to test your application thoroughly.

# CSSInject.pl use HTTP::Request::Common qw(POST GET); use LWP::UserAgent; # Set the user agent string. my $ua = LWP::UserAgent->new(); $ua->agent( CSSInject/v1.36 WindowsXP ); # Various injection strings my @css = ( ><script>alert(window.location);</script> , \"; alert(document.cookie); , \ onmouseover=\ alert(document.cookie);\ \ , \"><script>alert(document.cookie);</script> , \"></a><script>alert(document.cookie);</script> ); # Build the request. my $url = http://127.0.0.1/form.asp"; my $inject; foreach $inject (@css) { my $req = POST $url, [Name => $inject, Address => $inject, Zip => $inject]; my $res = $ua->request($req); # Get the response. # If we see the injected script, we may have a problem. $_ = $res->as_string; if (index(lc $_, lc $inject) != -1) { print HHMM! Possible CSS issue in $url\n ; } }

This sample code is also available on the companion CD in the folder Secureco\Chapter 14. You should also test for another case: many Netscape browsers support an alternate way to use script instead of using <script> tags. Script can be executed using the &{} format. For example, &{alert('document.cookie');} will display the Web site s cookie, and there are no tags!

Some issues outlined in Malicious HTML Tags Embedded in Client Web Requests at www.cert.org/advisories/CA-2000-02.html would have been detected using code like that in the listing just shown.

The most authoritative reference on cross-site scripting is Cross-Site Scripting Overview at www.microsoft.com/technet/ itsolutions/security/topics/csoverv.asp.