Using Databases and Web Servers to Manage Your Security Data

Going beyond just checking server logs, you also want to be able to analyze the output of the security programs discussed earlier in this book. The best way to do this is to import the results into a database. The rest of the tools in this chapter are designed to let you import and view security data in a database. To use these tools you'll need a database program and a Web server running to review the results. Though there are other options supported, the database recommended for these programs is MySQL and the Web server is Apache with PHP. You should set up these programs before attempting to install any of these tools. The basic installation and configuration of each of these prerequisite servers are described briefly here.

Setting Up a MySQL Server

MySQL is an open source SQL-compliant database that is gaining acceptance in the corporate world for its power and flexibility. While this book does not intend to teach you all the ins and outs of running a MySQL database, the following information will help you set up and execute some basic administrative tasks on a MySQL database so you can use the analysis tools.

Download the latest version of MySQL from www.mysql.com or use an RPM from your OS and distribution disks. Make sure it is at least version 4.0 or later.

Note: If you already have MySQL database version 4.0 or later installed, skip to Step 4.
Unpack the file and issue the usual compilation commands in the directory it creates:
```
 
```
```
 ./configure make make install 
```
Run the install script located in the /scripts directory by typing:
```
 
```
```
 mysql_install_db 
```
This initializes your database program and gets it ready for use.
Create a MySQL user and group for the database to use to perform tasks. Do this by issuing the following commands:
```
 
```
```
 groupadd mysql useradd g mysql mysql 
```

Set the ownership and file permissions so that MySQL can operate using the following commands:

 chown R root /usr/local/mysql chown R mysql /usr/local/mysql/var chgrp R mysql /usr/local/mysql cp /usr/local/mysql/support-files/my-medium.cnf /etc/   my.cnf

Edit the file /etc/ld.so.conf and add the following lines:
```
 
```
```
 /usr/local/mysql/lib/mysql  /usr/local/lib 
```
Save the file.
As root, type:
```
 
```
```
 ldconfig v 
```
As root, set an admin user for the MySQL database by typing:
```
 
```
```
 /usr/local/mysql/bin/mysqladmin u root password  123456  
```
where you replace 123456 with your password. Be sure to write down your password and keep it in a safe place.

When you are finished with this, change back to the user you logged in as by typing "exit" at the command line.
You will want to set up MySQL to start as a daemon and run all the time rather than having to start it manually. You can do this by placing the following line at the end of the rc.local file found in /etc/r.cd/.
```
 
```
```
 mysqld user=mysql & 
```
This will start MySQL as a system process every time you reboot.
Finally, you will want to lock down MySQL so that it doesn't become a security hole on your system. By default, MySQL has pretty weak security. While MySQL security isn't the subject of this book, here are a few things you can do.
- Delete the standard users, unless you have a program that uses them.
- Make sure that the root user can only connect from a small number of hosts .
- Set up some rules on your firewall to only allow connections on a limited number of ports and from a limited number of machines to your MySQL server.
- Create system accounts for running programs. Don't use either the system root account or the MySQL root account (two different things) unless you absolutely have to (the NPI tool does require this, unfortunately ). This chapter includes examples of application-specific accounts to create in each package description wherever possible.

MySQL is now ready to use. Type mysql at a command prompt and you will be prompted for a user name and password to enter the standard MySQL command line, where you can use standard SQL commands on your MySQL databases. See the sidebar for some basic MySQL commands.

Basic MySQL Commands

To log in to MySQL, type mysql “u username “p password , replacing username and password with the ones from the account on the MySQL database.

Note: This is different than your system login. This logs you into MySQL and puts you at the MySQL prompt mysql> where you can type commands. Be sure to put a semi- colon at the end of commands before pressing Enter to execute them.

The following are some basic commands to navigate and search a MySQL database.

`show databases;`	Displays all the databases available on that MySQL server.
`use` `database-name;`	Makes the database-name specified the active database so you can perform actions on it.
`show tables;`	Lists all the tables that exist in a database.
`select query from` `Tablename;`	Shows records that match the `query` in `Tablename` . There are a number of operands you can use in your `query` statement. Using an `*` (asterisk) as your query statement will list all records in that table.

Setting Up the Apache Web Server

The advanced analysis tools in this chapter require a Web server to act as both the configuration interface and the output mechanism. Again, this short section is not intended to be a comprehensive guide on how to run and operate a Web server; it is merely intended to get you up and running with what you need to use the security tools. If you intend to do use this server for more than ACID and NCC or to use it in high-volume environments, you should definitely do further reading on Web server administration. Running a Web server does involve some security issues, and you should make sure these servers are locked down, running minimal services, and keep them frequently patched. If you want to run IIS or an alternate Web server, it should work as long as it is compliant with PHP 4.0 or later.

Download the latest version of Apache from www.apache.org. If you can get it from your OS distribution disks or already have it installed, make sure you have at least version 1.3 or later.

Note: If you already have Apache version 1.3 or later installed, skip to Step 3.
Unpack the program and issue the following commands:
```
 
```
```
 ./configure prefix=/www enable so activate-module-src/modules/php4/libphp4.a make make install 
```
These commands set the default directory to /www and enable the proper modules you will need.
Run the Web server by typing apachectl start at the command line. This starts the http daemon and sets it up to run as a system process.

You can stop Apache at any time by issuing the same command with a stop parameter.

Other Linux and UNIX variants may have different ways of handling starting and stopping. Consult your documentation or online resources on how to accomplish this.
Check your Web server installation by opening a Web browser and putting in the IP address of your server, or specify localhost if you are working directly on that machine. If it brings up a sample Apache Web page, then you have successfully installed your Web server. The root directory of your Web server, where you can put documents you want to be publicly viewable, is /usr/local/apache2/htdocs/ on a Mandrake Linux system; different distributions may vary slightly.
Next, set Apache to run automatically when the system is rebooted (you don't want to have to manually restart the Web server every time you reboot). To do this, go to the directory where all the startup scripts reside; on Mandrake Linux, this is the /etc/rc.d directory. Each rc. file represents a different run level. Add the following lines to the rc4.d and rc5.d files:
```
 
```
```
 ../init.d/httpd S85httpd ../init.d/httpd K85httpd 
```
You can test this by rebooting your system and verifying that the httpd process displays when you the list processes running with the ps -ax command.
You should lock down Apache to prevent its abuse. Web servers are some of the most common targets for attackers , so if you are going to allow access to this machine from outside your network you need to make sure it is secure. The following are some basic tips for good Web server security.
- Run a vulnerability scanner against your Web server to make sure that it is all patched up and doesn't have any obvious security holes right after you have finished this installation and configuration process.
- Protect any non-public Web directories with some access control. The quickest and easiest way to do this is using .htaccess files.
- Encrypt communications from clients to the browser with SSL any time you are dealing with sensitive information (security data definitely counts in this category). If you are accessing your server from outside your local network, that is, across the Internet, consult your Web server documentation or the Internet on how to set this up.

Again, this isn't a comprehensive coverage of Web server security, but make sure you do these things before making your server public.

Setting Up PHP

PHP is a scripting language designed for use in Web pages. It is an interpreted language, which means it doesn't have to be compiled to work, so you can just insert your PHP script into a directory that recognizes PHP and it will run when accessed. This makes it easy to write code embedded into Web pages. Most current Web servers recognize PHP, though it may have to be configured on installation to do this.

Because of these features, PHP has become the language of choice for many Web-based applications. You will need it for the three remaining tools in this chapter (ACID, NPI, and NCC). PHP should have been installed as part of the configure directives in the Apache instructions described earlier. To check if PHP is installed on your system and what version it is, type php “v at the command line. If it is there, you should get some output with the version number. However, if you were unable to install it as part of your Apache installation or you want to reload the latest version, use the following procedure.

Download the latest version of PHP from www.php.net or use the RPMs on your OS installation disks. If you use the ones provided with your OS, make sure that you have version 4.0 or later.
Unpack the distribution.
From the installed directory, run the following compile commands:
```
 
```
```
 ./configure prefix=/www/php mysql=/usr/local/mysql   with-apxs2=/www/bin/apxs with-zlib-dir=/usr/local   with-gd  (all on one line)  make make install 
```
These configure statements enable several modules that are needed for the tools in this chapter.
Edit the Web server configuration file, httpd.conf, usually found in /www. Add the following lines, and then save the file.
```
 
```
```
 LoadModule php4_module modules/libphp4.so AddType application/x-httpd-php.php 
```
To make sure PHP is running properly, use a text editor to create a small script file called test.php. Enter the following in the file and then save it.
```
 
```
```
 <?php phpinfo(); ?> 
```
This file is a short PHP script that will display some basic system information when run.
Copy the test file into the /www/htdocs directory. Type in the URL or IP address of your machine and then enter /test.php. You should see the PHP version displayed on a Web page. If you do, then your PHP-enabled Web server is ready to go.

ACID (Analysis Console for Intrusion Databases)

The Analysis Console for Intrusion Databases (ACID) is a program designed to make better use of data from intrusion detection devices. It was written by Roman Danyliw and others for the AirCERT program run by Carnegie Mellon University. They are part of the larger CERT (Computer Emergency Response Team) organization. CERT has been instrumental in protecting the Internet and the organizations using it for many years . CERT tracks incidents of computer crime and sends out notices to a mailing list whenever a large incident happens. The CERT mailing list is kind of an early warning system for any large outbreaks or attacks happening on the Internet. As such, it can be very useful to a system administrator. You can visit the CERT site and sign up for their mailing list at www.cert.org.

The AirCERT project is placing IDS sensors at various organizations and studying overall trends in intrusion activity and behavior. They wrote ACID to facilitate this process. Because they open sourced the code for the project, you can use it for your benefit without doing anything as part of AirCERT.

The idea behind ACID is to port all your intrusion detection data into a database where it can be sorted and organized by priority. ACID gives you a Web-based control panel of sorts to view and manipulate these results.

ACID uses just about any SQL database and any Web server, and supports multiple sensors for the input data. It also accepts both raw Snort alerts and syslog-compliant log files. ACID currently only works directly with one IDS, Snort, but you can import logs into the ACID database from any device that will output in a syslog-type format using a utility called Logsnorter, which is available on the ACID Web site.

ACID has quite a few prerequisite programs. Besides a database, a Web server, and PHP, which have been covered earlier in this chapter, you need the following libraries and subprograms.

ADOdb

This package provides the database abstraction layer so PHP can use a standard interface to a multitude of databases, including MySQL. Download it from http://php.weblogs.com/adodb, unpack it into your /www/htdocs or applicable Web root directory, and it should be ready to go. No further installation is needed.

PHPLOT

This package lets you create charts using ACID. If you want to use this capability, download the module from www.phplot.com. Unpack it in the /www/htdocs directory, and just like ADOdb, it should be ready to use.

JpGraph

This program upgrades PHP to do color graphs. You'll need it, along with PHPLOT above, if you want to be able to graph your Snort data. Download it from www.aditus.nu/jpgraph/ and unzip it into your Web root directory (for example, /www/htdocs). It will create its own subdirectory and be available when needed by ACID.

GD

This package has the image manipulation libraries for PHP, which are also needed for graphing. If you installed PHP according to the instructions given earlier in this chapter, then you should already have this utility. Otherwise, download it from www.boutell.com/gd/ and install it in your /www/php directory. If you didn't compile PHP with the commands given earlier, you also need to make sure you have the following libraries for GD.

libpng . This provides PNG-format support for GD. You can get it at www.libpng.org/pub/png/ or from your OS distribution disks.
libjpeg-6b . This is a JPEG library for PHP. You can get it at www.ijg.org/ or from your OS distribution disks.
zlib . This provides compression support for GD. You can get it www.ijg.org/ or from your OS distribution disks.

Configuring Snort for MySQL

ACID assumes you have one or more Snort sensors up and running to feed it data. If you haven't already built your Snort sensors, refer back to Chapter 7. You must have your Snort sensors configured to log to a MySQL database. To do this, follow these steps when installing Snort.
- When first compiling Snort, use the following configure statement:
```
 
```
```
 /configure --with-mysql=/usr/local/mysql 
```
  Make sure you specify the directory where MySQL is located on your machine.
- Edit the configuration file, snort.conf. Find the commented out line that starts with #output database . Edit it as follows :
```
 
```
```
 output database: log,mysql,user=  snort  password=  123456  dbname=  snort  host=  localhost  
```
  Change the user snort and password to the correct database name and password you are going to use for ACID. ACID will create a database named "snort," although you can change this name by editing the ACID configuration file. If you are connecting to a local database, leave the host variable as localhost . If you are connecting to a database on a different machine, insert the IP address or hostname here.
Be sure to uncomment the line (delete the # ), and then save the file.

This chapter assumes you will be installing ACID on a separate machine from your Snort sensor. Putting them on the same machine is not only a bad idea from a security standpoint, but will also bog down your Snort sensor to the point of making it unusable. The box running ACID should preferably be located at a separate site from the Snort sensors ”this makes it harder for someone who hacks a Snort sensor to get to your logs. Figure 8.1 illustrates the elements of an ACID-Snort IDS.

Figure 8.1. ACID-Snort Intrusion Detection System

Installing ACID

Once you have all the prerequisite programs loaded, you can finally install ACID.

Get the program file from the book's CD-ROM or download it from the ACID Web site.
Place the tar file in your /www/htdocs directory. Unzip it there and it will create its own directory.
Remove the remaining tar file, as anything left in your root /htdocs directory could be accessed by someone using the Web server.

Configuring ACID

Change directories to the /htdocs/www/acid directory.
Edit the acid_conf.php file. The lines starting with slashes and stars represent comments and instructions on how to complete the configuration. The lines starting with $ are variables and tell the program specific things about your system.

Change each of these $ statements with the parameters for your system. Table 8.4 lists the variables and information about and recommendations for each entry.

Table 8.4. Variables for Configuring ACID

Variable Names	Descriptions
$DBtype	The type of database ACID will be using. The default is mysql, but you can also put postgresql or mssql if you want to use either of those two databases.
$alert_dbname	The IDS that ACID is drawing from. Currently only the Snort native format, snort_log, is supported, though there are plans are for ACID to support other IDS types in the future.
$alert_host	The host on which the alert database is going to be stored. This can be an IP address or host name. If it's running on the same machine, it would be localhost. For better security or performance, you could run the database on a separate machine than the PHP Web server.
$alert_port	Port on which to access the database. If you are hosting it locally, just enter `""` for this value.
$alert_user	The database user name that ACID will use to log the data. Make sure it matches the MySQL user name you created when you set up the database.
$alert_password	The password for the database user. Again, make sure it matches your MySQL password for that user.
$archive_dbname	The name of the database for Snort to archive to. The default of snort_archive is fine unless you are storing multiple databases on this machine and want more descriptive names.
$archive_host	The host where the archive database will be located. If it's on the same machine, this should be `localhost` .
$archive_port	The port to log into the database server. Use `""` if you are logging locally.
$archive_user	The database user to log the archive data under. Typically the same value as $alert_user above, though you could create a separate user for logging the archives.
$archive_password	The password for the database user logging the archive data. Again, usually the same value as $alert_password.
$chartlib_path	Path to the charting modules. This should be `/www/htdocs/jpgraph-1.11/src` .
$chart_file_format	The file format of chart graphics. The default is png. Other valid formats are jpg and gif.

After you have saved the file with these parameters, open a Web browser and enter the hostname or IP address of your Web server followed by /acid/acid_main.php . For example:
```
 
```
```
 http://localhost/acid/acid_main.php 
```
This displays the ACID configuration page. From here on in, you can use the Web interface to finish configuring ACID.
Click on the Create ACID AG button. This will create a database for your Snort data. The default database name is "snort."
Go to http://localhost/acid and you will see the ACID main page for your Snort database (see Figure 8.2).

Figure 8.2. Main ACID Interface

You have now finished configuring ACID and can start using it to manage your IDS systems.

Introduction to Using ACID

When you first log in, the ACID main page displays (see Figure 8.2). The top area of the main database view gives you overall statistics on the database you are viewing, including its name, the time frame of all the records contained in the database, and the date and time last queried.

The section below that has all the summary information on this particular alert group (AG). The AG is the sensor or group of sensors represented in this database. If you want to track different groups of sensors as an entity, such as for different customers or divisions, you need to create a separate database or AG for each one. This is also important when you are running reports and using the archival capabilities of ACID. You can only take search or query actions on an individual AG, not on multiple AGs, so make sure that your sensors are properly organized into distinct AGs. For most companies, it will be sufficient to have just one AG with all your sensors in it. But if you are a consulting company or have large operations with multiple divisions, you may want to put groups of sensors into different AGs so you can track them separately.

In the box on the left of the screen you can see the statistics on this AG: the total number of alerts, the number of unique alerts, and the number of different IP addresses appearing in the database, both by source IP and destination IP. If you have multiple sensors in your ACID network, you can click on the Sensors item to see them listed. You can narrow your search criteria down to the data from just one sensor. This main page also profiles your alert traffic graphically by each protocol and port scan traffic so you can get a flavor for what kind of traffic is being scanned by your NIDS sensor.

Using ACID to Tune and Manage Your NIDS

Before your NIDS can be useful at all, you must tune it to your network to eliminate false positive alerts. ACID can be invaluable in this effort. When you first turn on your NIDS, all of the alert signatures will be active and your database will begin to fill up with alert activity. Most of these alerts are going to be false positives at first. To make the alert data relevant to your network, you need to begin removing some of these signatures to eliminate much of the erroneous activity and to give you only data that is actionable .

Once you have a sufficient number of alerts in the database (at least a thousand on a busy network), you can start analyzing your alert data and eliminating some alert types. Watch your database carefully , as it may not take very long at all for it to fill up, especially with the default Snort rule list.

Open ACID and click on the Unique Alerts button. This shows the most recent alerts categorized by alert type (see Figure 8.3).

Figure 8.3. Most Recent Unique Alerts Listing

On this page, you see the following information on each alert type:

Signature name
Alert classification
Total number of that type of alert in the database
Sensor number that the alert came from
The number of different source IP addresses associated with that alert
The number of different destination IP addresses associated with that alert
The time the alert came in

You can sort any of the columns by clicking on the little arrows at the top of the column. For example, you can sort by the number of alerts, then click on the largest offender. This narrows down the list to only that alert type. Scan the list and see if you can determine if this is really a security issue or a false positive. Are there any discernable patterns? Are they all coming from the same IP address? Are they all going to the same IP address? Are they happening at regular intervals or do they seem random? If this analysis doesn't lead to any conclusions, drill deeper by clicking on individual alerts. This lets you see the actual packet that set off the alert, and is very helpful from a forensic standpoint if you are actually being attacked and are trying to respond or pursue the attackers further.

Also, be forewarned: If sensitive data is crossing your network, you may inadvertently end up viewing it since you are capturing whole packets of data in the alerts. Make sure you are cleared to see this kind of data. It is also very important that your Snort database is properly secured, because anyone who breaks into your database machine would potentially have access to that sensitive information. Another solution to this is to turn down the level of detail captured in the alerts, though this may hinder your ability to use the alerts to track down the culprit.

In the example in Figure 8.3, Web-IIS cmd.exe is the most common alert. By clicking on the alert detail, you can see the actual packet that generated this alert (see Figure 8.4). The source IP is shown, along with all the TCP ports and settings.

Figure 8.4. ACID Alert Detail

Based on the host name, this packet came from an address in Japan ( upper-level domain of .jp). You can use this to determine whether this is an address that should normally be accessing your network. You can also look lower and see the actual packet payload. The left side has the packet data in hexadecimal and the right side converts it to ASCII (if it is possible to display that way). This shows the actual commands the sender was trying to issue against your machine. From looking at these, it seems someone was trying to access the cmd.exe command; in other words, get a command line prompt. This is clearly an attack on your system. Unfortunately, this is probably a scripted attack done by an Internet worm, and these types of attacks happen dozens of times a day, as you can see from the large number of cmd.exe alerts in the database. Still, it's worth keeping an eye on, to see if that IP comes up consistently. You can at least file a complaint with the ISP and make sure the destination machine being attacked is secure against that exploit. You could also take further action against the IP address listed as the source, such as legal prosecution or civil action, if a successful break-in had occurred. At least you now know exactly what kinds of attacks are coming at your network and what they are trying to do. This will better enable you to proactively protect your network and react if it comes under attack.

Other Ways to Analyze Alert Data Using ACID

Who's Being Attacked?

Using ACID, look up the most common IP destination addresses. This shows the IP addresses that are supposedly getting attacked the most, and therefore they are the machines to focus your security efforts on. This also helps you discern false positives from real ones, as you may find one particular machine generating a huge number of alerts from an application it is running. A sudden upsurge in alerts to a particular IP address could point you to an attack on that machine that is underway. Then you can batten down the hatches by running vulnerability scanners , checking patch levels, dropping packets from the offending source IP(s) at the router, and so on.

Who's Doing the Attacking?

Look at the IP source address that appears the most often. Go to the source IP list off the main page; this shows the IP and then the Fully Qualified Domain Name (FQDN). This tells you where the attack is coming from. Sorting by the number of alerts lets you see the worst offenders in terms of generating alerts. If the IP addresses with the most alerts are on your network, you may have an internal culprit or an application that is triggering an alert. Use the process discussed earlier to drill down to the alert detail level and analyze the alerts. If they are from external IP addresses, you will want to determine if this is legitimate traffic bound for your network or actual attacks. Look at the individual alerts to see what they are trying to do. Click on the IP address; this displays a page with additional information on the address and some options to analyze it further (see Figure 8.5). You can perform various functions on that address such as reverse DNS lookup, ARIN lookup, and even a Sam Spade search (similar to the tool studied in Chapter 2) from within ACID. The output of these functions should tell you what organization owns those IPs, any contact e- mails for their network operations center, and abuse e-mail contacts (if available). You can use these contacts to register a complaint about these activities. Also, if you see certain addresses showing up again and again, you can filter these IP addresses at your router or firewall.

Figure 8.5. ACID Source IP Address Detail

What Service Is Being Attacked the Most?

By looking at the most common ports on which alerts are being received, you can see what services are being targeted . If you see a lot of Web-based alerts, you will want to pay closer attention to keeping your Web servers properly locked down. If the alerts show a lot of Windows NetBIOS activity, you will want to look at your Windows permissions and password policies. This will give you a good idea of what services to focus your attention on first.

Using ACID on a Daily Basis

Once you have ACID running and sufficiently tuned for your NIDS configuration, you should get in the habit of checking it at least once a day to see what new alerts have been generated. A good rule of thumb is to check first thing in the morning and again just before you go home. If you have after-hours personnel working, you could also add checking the ACID alert database to their routine.

When you log into the ACID database, you can go immediately to the Snapshot section (see Figure 8.6) and click on Most Recent Alerts to get a quick view of the most current activity. This shows you all the alerts in chronological order. If it is still generating too many alerts to be useful, in the Today's Alerts section select Unique. This shows you all the alerts from today grouped by alert type so you can see which ones are generating the most traffic. The Snapshot section options Last 24 Hours and Last 72 Hours are also useful. These let you search on the most frequent alerts, addresses, and ports during various periods.

Figure 8.6. ACID Snapshot Section

Graphing ACID Data

If you are more of a visual person or you need something graphical to show management, ACID comes with the ability to create graphs and charts based on your alert database. This feature is still experimental, and you must have the PHP graphing modules listed at the beginning of this section. However, this does give you some nice options for outputting graphical summaries of Snort data. You can access the graph feature by clicking on Graph Alert Data just below the Alert statistics box off the main ACID screen. This displays the graphing options. You can arrange the data for your graph by:

Time ( hour , day, month) versus the number of alerts
IP addresses (source or destination) versus the number of alerts
TCP or UDP ports (source or destination) versus the number of alerts

Set your parameters using the pull-down boxes and click on Graph Data. Make sure you have filled in every box or you will get an error message. ACID generates and displays a graph. See Figure 8.7 for an example of an ACID graph

Figure 8.7. ACID Alert Data Graph

Maintaining Your ACID database

As your alert database grows, you will need to perform some maintenance on it periodically to keep it from getting too big and unwieldy. Also, your statistics and graphs will be more accurate if you archive your early alerts, which will contain many false positives. In addition, cleaning out your database from time to time will make your queries process faster.

To archive your alerts, use the query control at the bottom of the main screen. Create a query for the alerts you want to archive, for example, all the alerts generated last year. Then select Archive Alerts as the action for your query. You can selectively archive alerts by date, alert type, or other criteria. You can also choose to just copy the alerts into the archive or to remove them. The archived alerts will be placed in their own database with the name you set in the acid_conf.php file during configuration.

You should archive all of your alerts from the first few months of operation when you are fine-tuning the Snort sensor. After that, the data will have more relevance to actual attacks versus false positives. It is also a good idea to archive at least once a year or perhaps quarterly depending on the volume of alerts being logged. As a general rule of thumb, you don't want more than 100,000 alerts in your database at any given time.

So, you now know how to build a complete Snort intrusion detection network, with multiple sensors logging into a database tool that can be used to analyze the data and do reporting. This will help you better use the intrusion detection data, make the most of your security efforts, and have some tangible reports and graphs to show management. Now, we will look at some tools that can help with your vulnerability scan data.

Flamey the Tech Tip:

Use Names Carefully!

Be careful when talking to your boss about using Snort and ACID at work. Make sure he or she understands that these are valuable management programs, not illegal controlled substances!

NPI (Nessus PHP Interface)

One problem when using the vulnerability scanner Nessus to scan medium- to large- size networks is that the reports can be quite intimidating. The Nessus reporting formats themselves are fine and the HTML is fairly easy to navigate, but when you have a couple of hundred pages of data to sort through, it can be hard to see the important data amongst all the noise. It would be nice to have something to organize the scan data any way you like. To really be able to analyze it, you need to get your data into a database versus the standard flat file format. You also want to be able to access the data easily, such as from a Web-based interface. With this capability, you could quickly get to the most important data and analyze scan results over time to see if your network was getting more secure or less secure.

Fortunately, several products integrate Nessus with a database. A few of the Nessus-to-database programs are NesQuik, Java Nessus Report Manager, and Nessus PHP Interface (NPI). I selected NPI for this book for a number of reasons. First, it is a truly open source product with no commercial ties. Also, it uses MySQL and PHP, which you have already used for other tools such as ACID. Using these applications, NPI offers the ability to port your Nessus data into a database and then view it with a Web browser

NPI, similar to ACID in its design, uses a MySQL database to store the results and a PHP-enabled Web server to view and query the results. Figure 8.8 illustrates the logical components of NPI. One difference between the Snort and Nessus architectures is that with Nessus there are two different parts generating the data: a client initiating the scans and a server running them. In some cases they might be on the same machine, but this figure shows two different physical servers. There is also database server to which the scan data is logged, and a Web server that provides the interface to the data. Again, the database and Web server could be on the same machine or two different boxes.

Figure 8.8. NPI Logical Design

Figure 8.8 demonstates the flow of data and the logical parts in the NPI system. The Nessus client logs into the Nessus server to initiate a scan of a target. The data is stored on the client machine in the native Nessus .nbe format. Once you have the raw file, NPI runs a conversion script on the .nbe file and imports the data into a MySQL database. When the scan data is in the database, you can view and query the database using a Web-based PHP interface from any Web browser. This sounds like a much better way to analyze scan data, so let's get busy and set up NPI.

Installing NPI

Before beginning to install NPI, you need to have your MySQL server and Apache Web server with PHP working. If you haven't already gotten these up and running, refer back to the appropriate sections earlier in this chapter. Once you have your SQL and Web servers ready, take these steps to install NPI.

Get the file from the book's CD-ROM, and unzip and untar the program. Place the contents in its own directory. The directory name doesn't matter as long as all the files are in the one directory. For example, /usr/local/nessus-php will work fine, assuming you have the permissions to write there.

Change into that directory and edit the file nsr-php using a text editor. This file contains variables (those lines that start with $ ) that allow the program to contact your MySQL server. Table 8.5 lists the variables you need to edit and recommendations for their values.

Table 8.5. NPI Variables for the MySQL Server

Variables	Descriptions
$db_host	Set this to the host on which the MySQL server will be running. If the MySQL server is running on the same machine as the Web server, then this is `localhost` . Otherwise enter the IP address or hostname of the MySQL machine.
$db_user	This is the username used to log into the MySQL database. This can be the same as the user name set up in the MySQL installation.
$db_pass	The password corresponding to the above user.
$db_database	This is the name of the database that is created to hold the NPI data. The default database name created by NPI is nessus, but you can change this to whatever you would like.
$db_suuser	This is the administrative user on the MySQL database. This is required by the nsr-php script. The default is root, which should work for most installations. Note that this is not the same as the root user on your operating system.
$db_supass	The MySQL root password corresponding to the $db_suuser account described above.
$your_domain	A list of domains that you want to be stripped out from the output. This will clean your reports up if you just want machine names to appear in the output rather than the full Internet host name. This is an optional variable.

Open the file nsr in a text editor and make the same entries in it as you did in Step 2 (see Table 8.5). Make sure that the path to the Nessus plug-ins directory is correct. In most installations they should be in the default location, which is /usr/local/lib/nessus/plugins. Check to make sure yours are there and if they are not, make the appropriate modification in the nsr file.
Edit the file nessusphp.inc, also changing the variables as in the above two steps.
Create your Nessus database. Do this by running the nsr-php script:
```
 
```
```
 php nsr-php b 
```
This creates the MySQL database called nessus for your scan data. You can log into it and check to make sure the proper tables were created.
Verify that the database was properly created.

a. Log into MySQL using the commands in the MySQL sidebar earlier in this chapter.

b. Type show databases ; (don't forget the semi-colon) at the MySQL command prompt. A list displays showing all the databases, and your newly created Nessus database should be one of them.

c. Switch to that specific database using use NESSUS ; and then type the show tables ; command. You should get the results shown in Figure 8.9. This shows each of the three tables in the nessus database.
Copy the files in the /www subdirectory of your NPI install directory to your Web server's root directory. Create a new directory in your Web server's document root where your NPI files will reside. This is the location from which you will access your Nessus database. Issue the following commands from your nessus-php directory to do this and set the correct permissions for the files.
```
 
```
```
 mkdir /usr/local/apache2/htdocs/nessus-php mv ./www /usr/local/apache2/htdocs/nessus-php chown R www.www /usr/local/apache2/htdocs/nessus-php chmod 755 /usr/local/apache2/htdocs/nessus-php/* 
```
Figure 8.9. Output of the show tables; Command
```
 mysql> show tables; +--------------------+  Tables_in_nessus   +--------------------+  report         scans         scripts       +--------------------+ 3 rows in set (0.00 sec) 
```
Be sure to change the path to your Web document root if it is different from the example. You can change the name of the directory that the nessus-php files are in if you like. This example puts the NPI access page in /nessus-php off your Web document root. It also uses the same system user (www) for both the user and group that will access the MySQL database. For better security, set up a special user with only read access to the data. In that case you would change the www.www to the appropriate read-only user and group. You need to do this for the appropriate MySQL account as well.

You have now installed the NPI program.

Importing Nessus Scans into NPI

Now you are ready to import some Nessus scans into your database.

Run the nsr script on each native Nessus scan file to be imported. (Obviously, you must have some completed scan files available, saved in the native .nbe format.) NPI will also accept and convert the older Nessus format .nsr. Enter the following to run nsr from the command line:
```
 
```
```
 ./nsr  /scans/scan.nbe  
```
Replace /scans/scan.nbe with your scan file name and path. This takes the raw Nessus file and imports it into your database. It also checks your Nessus plug-ins and creates entries in the database for any new plug-ins that may have been added.
You are now ready to view your Nessus scans in your database. Open a browser and enter the IP of your NPI Web server with the path to your Nessus-php index file, for example, http://localhost/nessus-php/. The results should appear in the NPI interface, fully searchable and sortable (see Figure 8.10).

Figure 8.10. NPI Main Screen

Using NPI

You can now browse your scan data like any other database, sort it, and run queries to find specific vulnerabilities, hosts, or other criteria. There are quite a few way to analyze your Nessus data using NPI. You can sort by the:

Host (IP address) with the most vulnerabilities
Most common vulnerability
Most common category of exploits
Most exploitable service (port number)
Scan date or date range
CVE or CAN number

By creating queries with NPI, you can focus on areas that represent the greatest danger to your network and maximize the effects of your remediation activity. You can also quickly rule out certain alerts and/or machines. With NPI, you can manipulate your scan data any way you choose. One downside to the NPI tool is manually loading each scan file into the database. A script could easily be written to automate this process, but some associates and I decided to take it one step further.