Step 4: Pilot

The next major phase in the project is to conduct a pilot test. Use the pilot to evaluate convaluate the operation of the security agents, the choices you made during the first two phases, and your deployment procedures. You can use the test results to make any necessary adjustments before you move to a wider deployment. This section covers the pilot test:

• Scope

• Objectives

Scope

For the pilot test, deploy the agent on a relatively small number of hosts. The number is up to you, but any number between 10 and 50 will work. Make sure to install the agents in Test Mode.

Note

Test Mode is a configuration option in the CSA MC. When Test Mode is turned on for a set of hosts, the agents on the host do not actively enforce policies. Instead, they log only policy violations.

Choose the hosts for the pilot test carefully. If you are not going to install the agent on all of your hosts, make sure to use hosts that are within the scope of the overall project. Also, try to select a few hosts from each of categories on which you intend to deploy the agent. For example, if you plan to install the agent on remote laptops and database servers, try to pilot test on a few of each.

Finally, do your best to test on machines that are operated by "friendly users." Friendly users are people who are more willing to try new technologies. Also, friendly users are ready to provide positive and negative feedback about the technologies they try.

ACME installed the agent in test mode on four remote laptops, four manufacturing desktops, and one e-commerce web server. The web server was a semi-production server because the server team was nervous about installing something new on its production systems. It would, however, become active if one of the others failed. Because it was configured as though it were a regular production server, it served as a good pilot host.

Objectives

You should accomplish the following objectives during the pilot test phase:

• Test software compatibility You always run the risk that CSA, at a basic level, is incompatible with one of your standard software packages. The pilot test is an opportunity to find incompatibilities and resolve them before you expand the deployment.

• Begin tuning The CSA default policies are designed to operate in a wide variety of computing environments. However, no environment is alike. You invariably experience false positives, which are instances where the CSA policy mistakenly treats normal activity on a host as dangerous.

  While your agents are running in Test Mode, use the CSA MC's event log to identify false positives. When you encounter one, you can "tune" it out by configuring CSA to treat it as a normal occurrence. Leave your agents in Test Mode long enough to tune all of the false positives you find.

• Gauge user experience After you have tuned the CSA policy to the point that you see few or no false positives, you should turn off Test Mode. The agent should be in Full Enforcement Mode, so that you can measure the impact is has on the pilot test users' computing experience. You might want to conduct a survey of the users to get their feedback, both positive and negative.

When all of the objectives are complete, evaluate the results of the pilot test. If a procedure didn't work well, try to improve it. If the users had a bad experience with CSA, determine why and address the issue. Essentially, you should fix any problems you encountered during the pilot test before you move on to the tuning phase.

ACME CSA pilot test went well. The test lasted for two weeks, and during that time, it didn't encounter any compatibility issues. It was able to eliminate most of its false positives, and at the same time, ACME became even more familiar with the CSA management interface.

During the last four days of the pilot, ACME took the agents out of Test Mode so that the users could see how they liked CSA. The results from an informal survey were encouraging. The only complaint was that the agent asked the users too many questions that they didn't know how to answer. The project team made some changes to the policies, so that the users would be asked fewer questions in the future.