Step 2: Creating Logical Groups Within the Application Structure

Step 2: Creating Logical Groups Within the Application Structure

The first thing to do after crawling a site is to look at the URL path structure of the Web application. In most cases, the path structure reveals information about application functionality. For example, let's consider the crawler results from http://www.example.com/, a few URLs of which are:

http://www.example.com/index.asp

http://www.example.com/login/login.asp

http://www.example.com/login/secure/transaction.asp?id=AD678GWRT67344&user=bob&doit=add

http://www.example.com/download/download.cgi?file=tools.exe

http://www.example.com/download/download.cgi?file=setup.exe

http://www.example.com/scripts/search.cgi

http://www.example.com/scripts/mail.pl

http://www.example.com/admin/administrator.asp

http://www.example.com/news/index.asp

http://www.example.com/public/default.asp

http://www.example.com/servlet/showtrans?id=13445466

http://www.example.com/servlet/setcustinfo

http://www.example.com/contacts/

http://www.example.com/profile/

http://www.example.com/private/

http://www.example.com/logoff/logoff.asp

What can we infer from this result? Some things are apparent. The /login/ section is some sort of entry point to areas of the application that are restricted for registered application users. The /logoff/ section is for clearing the user session after the user has finished using the application. The /download/ section hosts resources made available for all users. Files such as setup.exe and tools.exe can be downloaded from the site by the script download.cgi.

Let's see what happens when we make a request for the /private/ area. In the browser, if we click on this particular link, we get the HTTP authentication dialog box shown in Figure 8-7.

Figure 8-7. Authentication dialog box for example.com

If we try sending the same HTTP request by using netcat, we get this response:

nc www.example.com 80

HEAD /private/ HTTP/1.0

HTTP/1.1 401 Authorization Required

Date: Mon, 18 Mar 2002 09:40:24 GMT

Server: Microsoft-IIS/5.0

WWW-Authenticate: Basic realm="special directory"

Connection: close

The HTTP response code from the server is 401, which indicates that the /private/ area is password protected. Here the authentication required is HTTP Basic authentication, where the user and password string is sent after encoding it with Base64 encoding. Now we know that in order to proceed further, we need some valid user credentials. The Web hacker notes to herself either to use some previously cracked accounts here or to try to enter this page by using an automated brute force script.

Another interesting directory is the /scripts/ directory. It has a few server-side scripts, such as mail.pl and search.cgi. By simply looking at the script names, we infer that mail.pl probably is used to send e-mail and that search.cgi probably is used to perform some keyword searches. The /admin/ directory hosts a resource called administrator.asp, which appears to be a Web site administration page available only to the site administrator with proper credentials.

As we look at the collection of URLs as a whole, another thought emerges. Grouping ASP files, CGI files, and Java servlets in one Web application isn't commonly done. Their collective presence suggests that this Web application is hosted by more than one Web server platform. The ASP and perhaps even Perl and CGI pages are hosted by a Windows server running IIS, whereas the Java servlets are hosted by another server. Recall from Chapter 6 that various areas of a Web application can be hosted on various platforms and that these platforms are linked to function as a cohesive unit.

There are no fixed rules when it comes to extracting such information. Past experience and knowledge about commonly followed Web application development practices is a great help when you're performing such an analysis. In the following chapters, we discuss various attack scenarios, which further illustrate the concepts introduced here. Each of the links in a Web application can have a potential security flaw, which an attacker can exploit. Our focus here is to associate crawled URLs with their possible functional uses and perhaps also to identify areas of potential abuse. In this way, we can associate all such inferences with the Web resources found. Figure 8-8 shows the Funnel Web Profiler's tree view of directories and files found on http://www.foundstone.com/. What we need to do is tack on the logical inferences with a list of this type.

Figure 8-8. Funnel tree

Step-2 Wrap-Up

We demonstrated how to look at URL strings and Web resources to figure out their individual purposes in the Web application. In Step 3 we describe how to compile all the information discovered and reorganize it into a more usable format.