The Basics of Technology Identification

The Basics of Technology Identification

The first step in identifying technologies that comprise a Web application is to look at the HTTP header returned by the Web server. In most cases, the HTTP header can accurately identify the front-end Web server being used. A simple HTTP HEAD or GET request can be issued by using a utility such as Netcat to obtain the HTTP header. An HTTP header of a server running Apache looks like this:

# nc www.example.com 80

GET / HTTP/1.0

HTTP/1.1 200 OK

Date: Mon, 04 Feb 2001 19:29:37 GMT

Server: Apache/1.3.19 (Unix)

Set-Cookie: Apache=192.168.51.1.308631012850977729; path=/; expires=Tue, 04-Feb-02 19:29:37 GMT

Last-Modified: Mon, 10 Dec 2001 04:48:34 GMT

ETag: "94b5d-1ef-3c143ea2"

Accept-Ranges: bytes

Content-Length: 495

Connection: close

Content-Type: text/html

The Server: field in the HTTP header reveals that it is an Apache 1.3.19 server and that it also supports SSL, mod_perl, and PHP 4.0. Not all Web servers are so verbose in their descriptions of the technologies being used. Even Apache can be configured to return only the bare minimum HTTP headers needed by the Web browser. An HTTP header from another Apache server is:

# nc www.example.com 80

GET / HTTP/1.0

HTTP/1.1 200 OK

Date: Mon, 04 Feb 2001 19:12:35 GMT

Server: Apache/1.3.12 (Unix)

Connection: close

Content-Type: text/html

Here we have no idea about the types of application scripts that are running on this system. We need to observe more distinguishing characteristics in order to narrow the possibilities and identify the technologies present.

The next step in refining our guesses about the technologies is to observe the various elements of the URL. In Chapter 5 we talked about the URL structure and the role that each element plays in the URL. The starting point within the URL is the type of resource or file being requested. The file types or extensions identify the server platform.

The following list shows the file extensions used in URLs and the platforms with which they are usually associated.

Extension	Technology	Server Platform
.pl	Perl CGI script	Generic; usually Web servers running on Unix
.asp	Active Server Pages	Microsoft IIS
.aspx	ASP+	Microsoft .NET
.php	PHP script	Generic; usually interfaced with Apache Web servers
.cfm	ColdFusion	Generic; usually interfaced with Microsoft IIS
.nsf	Lotus Domino	Lotus Domino server

Because Web servers are so flexible in terms of configuration, you may be able to obfuscate the file types in order to conceal information about the technology being used.

Beyond this point, technology identification becomes an art. File types such as Java Server Pages (.jsp) don't give any clues about the front-end Web server or the Java application being used to execute the JSP files. The reason is that all popular Java application servers conform to J2EE standards. You must dig through clues embedded in the URLs, HTTP headers, and sometimes even HTML content in order to accurately identify the technologies behind the curtain. The best way to proceed is to learn by example. Let's look at some URL examples and consider how to identify or guess the technologies being used.

Examples

URL: http://www1.example.com/homepage.nsf?Open

The file extension .nsf indicates that this server is most likely a Lotus Domino server. A quick look at the HTTP header generated by this request confirms this supposition:

# nc www1.example.com 80

GET / HTTP/1.0

HTTP/1.1 302 Found

Server: Lotus-Domino/5.0.5

Date: Mon, 04 Feb 2001 17:52:59 GMT

Location: homepage.nsf?Open

Connection: close

Content-Type: text/html

Content-Length: 305

URL: http://www2.example.com/software/buy.jhtml;jsessionid=ZYQFN5W HKORD5QFIAE0SFF GAVAAUIIV0

The HTTP header from this site identifies it as Microsoft-IIS/4.0. However, we know that IIS 4.0 doesn't natively support .jhtml files, which are actually Java HTML pages. Thus an application server is working in conjunction with IIS 4.0. The string ";jsessionid=xxxxx" is a telltale characteristic of the ATG Dynamo Application Server, which is used in this case to serve Java HTML files and execute Java Servlets. ATG's main Web site itself has a similar URL: http://www.atg.com/en/index.jhtml;jsessionid=1GA5D1FQUNUS4CQCQBJCGWQKAKAFUIV0?_requestid=2331.

URL: http://www3.example.com/cgi-bin/ncommerce3/ExecMacro/webstore/ home.d2w/report

This URL is typical of IBM's Net.Data e-commerce platform. The strings "ncommerce3" and "ExecMacro" are the key strings that reveal the technology type. Here "home.d2w" is an application macro written in the IBM Net.Data scripting language and "report" is one of the many methods that the application macro provides. The underlying front-end Web server is detected by the HTTP header.

URL: http://www4.example.com/ category.jsp?id=21&StoreSession=PC1q Nwwm0xqCFOWHZcYxZaZ21laYQEfOetnSjrYtrsxSC1V7b|3886513130244820/ 167838525/6/7001/7001/7002/7002/7001/-1

This example is peculiar. The file being requested is a Java Server Pages file. The underlying server is a Netscape Enterprise 4.1 server, which is identified by the HTTP header. However, the URL "signature" isn't that of an ordinary Netscape Enterprise server. Instead, the URL signature is typical of BEA WebLogic, whose session identifiers have the form:

Session=xxxxxxxxxxxxxx|######/#####/#/7001/7001/7002/7002/7001/#

The first part of the session identifier is an alphanumeric string, followed by a "|" symbol and some numeric strings separated by "/"s. The numbers 7001 and 7002 refer to the HTTP and SSL TCP ports, which WebLogic uses internally to communicate with front-end Web servers. If WebLogic is running on ports other than 7001 and 7002, they are reflected differently in the URLs generated from that site.

URL: http://www5.example.com/site/index/0,10017,2578,00.html

This URL also is peculiar: It ends with numbers separated by commas and the extension .html. It doesn't seem to be a static HTML page because, if we browse the site, the numbers keep on changing but the overall URL template remains the same. This result indicates that the content displayed is generated dynamically and that the numbers correspond to various content collections and categories. This URL is typical of Vignette's Story Server, a popular content server used with front-end Web servers such as the Netscape Enterprise server and Microsoft IIS.

If we were to look at the HTML code generated by the Vignette Story Server, we find HTML comments such as the following one clearly indicating the application server type and version:

<!  Vignette V/5 Sat Feb 02 05:07:43 2001  >

Vigenette's own Web site uses the Vignette Story Server, and the URL is similar to the one shown in the example head or: http://www.vignette.com/CDA/Site/0,2097,1-1-1489-2890,00.html.

More Examples

The preceding examples should have given you an idea of what it takes to identify Web server and application technologies. As you come across different types of Web application servers, you will tend to recognize various URL signature patterns generated by these servers. However, in certain cases technology identification isn't as straightforward as in the examples just discussed. Consider these URLs:

         http://www6.example.com/report.cgi?page=3

         http://www7.example.com/ui/Login.jsp

         http://www8.example.com/webapp/wcs/stores/servlet/Display?storeId=10001&langId=-1&catalogId=10001&categoryId=10052&clearance=0&catTree=10052

         https://www9.example.com/OA_HTML/store.jsp?section=101&prod_ses=j=4081:Guest:US:jtfpfalse:jtfpi-1:671:504:75123~zv=75123~zs=t~zp=504~zo=2~zm=101~zj=Guest~zi=504

The first URL seems to be straightforward. An initial guess is that it is running Apache with the application written in Perl. But, to identify the technologies behind the URLs, we need to dig deeper for clues. Let's begin by observing what the HTTP header for each request looks like.

URL: http://www6.example.com/report.cgi?page=3

nc www6.example.com 80

GET /report.cgi?page=3 HTTP/1.0

HTTP/1.1 200 OK

Server: Apache/1.3.6

Date: Mon, 04 Feb 2001 19:19:58 GMT

Connection: Keep-Alive

Content-Length: 1270

Content-Type: text/html

Set-Cookie: ASPSESSIONIDGGQGGCVC=KELHFOFDIHOIPLHJEBECNDME; path=/

Cache-control: private

If we look closely enough at the HTTP header, we see that what we have is contrary to what we assumed. Our initial assumption was that www6.example.com was an Apache server with the application report.cgi most likely written in Perl. The cookie being returned by the Web server tells us otherwise. The cookie is an Active Server Pages session cookie, with the name ASPSESSIONIDGGQGGCVC and the value KELHFOFDIHOIPLHJEBECNDME. Apache servers never send such cookies. This leads us to conclude that the IIS server is obfuscated by changing the server type string to "Apache/1.3.6" and mapping Active Server Pages extensions as .cgi instead of .asp.

Cookies returned by the Web server and the application server provide us with more signatures to look at!

URL: http://www7.example.com/ui/Login.jsp

This URL is too generic to identify the type of Java application server being used, so we again resort to inspecting the HTTP header for more clues:

nc www7.example.com 80

GET /ui/Login.jsp HTTP/1.0

HTTP/1.1 200 OK

Server: Netscape-Enterprise/3.6 SP2

Date: Mon, 04 Feb 2001 19:26:16 GMT

Pragma: nocache

Content-Type: text/html

Expires: 0

Set-Cookie:

WebLogicSession=PF7gWP6zdhQp0JaTuSJrhc5yBNVTGvyM2K1Jd2prFc3Dc2Ct2|35632677440752739/1743

30161/6/7001/7001/7002/7002/7001/-1; path=/

Connection: close

Again, the cookie string comes to our aid. The cookie WebLogicSession tells us that the system is running on a BEA WebLogic application server working in conjunction with a front-end Netscape Enterprise server. The structure of the WebLogic session cookie is the same as the WebLogic session identifier shown previously. The following list displays some known cookie types and the application servers associated with them.

Server	Cookie
Apache	Apache=202.86.136.115.308631021850797729
IIS	ASPSESSIONIDGGQGGCVC=KELHFOFDIHOIPLHJEBECNDME
ATG Dynamo	JSESSIONID=H4TQ0BVCTCDNZQFIAE0SFFOAVAAUIIV0
IBMNet.Data	SESSION_ID=307823,wFXBDMkiwgAnRyij+iK1fg87gsw8e/ TUDq2n4VZKc+UyjEZq
ColdFusion	CFID=573208, CFTOKEN=86241965

For www8.example.com and www9.example.com, looking at HTTP headers doesn't reveal anything about the application server being used. We now need to use some arm-twisting tactics on the application server to make it talk!