Introduction


Some of the most common complaints ISSOs make are that management doesn't support them, and—as the famous comedian Rodney Dangerfield is known for saying—"I get no respect." Another complaint is that the cost and benefits of InfoSec cannot be measured.

As for the first two, you get support because you are being paid—and these days, more often than not, quite handsomely—and you have a budget that could have been part of corporate profits. Furthermore, respect is earned. Besides, if you want to be popular, you are definitely in the wrong profession.

One often hears management ask:

  • "What is all this security costing me?"

  • "Is it working?"

  • "Can it be done at less cost?"

  • "Why isn't it working?"

That last question often comes right after a successful denial of service attack or some other attacks on the corporate systems or Web sites. Of course, many ISSOs respond by saying that it can't be measured. That is often said out of the ISSO's ignorance of processes to measure costs or because the ISSO is too lazy to track costs.

The more difficult question to answer is, "What are the measurable benefits of a CIAPP and InfoSec functions that provide support under the CIAPP?" Of course, one could always use the well-worn-statement, "It can only be measured as a success or failure depending on whether or not there have been successful attacks against our systems." The truth is that many attacks go unnoticed, unreported by the users or IT people. Furthermore, separating attacks from "accidents" (human error) is usually not easy; however, metrics can help in the analyses.

What Is a Metric?

To begin to understand how to use metrics to support management of a CIAPP, it is important to understand what is meant by "metrics." For our purposes, a metric is defined as a standard of measurement using quantitative, statistical, and/or mathematical analyses.

What Is an InfoSec Metric?

An InfoSec metric is the application of quantitative, statistical, and/or mathematical analyses to measuring InfoSec functional trends and workload—in other words, tracking what each function is doing in terms of level of effort (LOE), costs, and productivity.

There are two basic ways of tracking costs and benefits. One is by using metrics relative to the day-to-day, routine operations of each InfoSec function. These metrics are called level of effort (LOE) and are the basic functions noted in the ISSO's charter of responsibilities and accountabilities. Examples would be daily analyses of audit trail records of a firewall; granting users access to systems; and conducting noncompliance inquiries. In more financial terms, these are the recurring costs.

The other way of tracking costs and benefits is through formal project plans. In other words, if the tasks being performed are not the normal LOE tasks, then they fall under projects. Remember that functions are never-ending, daily work, while projects have a beginning and ending date with a specific objective. In more financial terms, these are the nonrecurring costs.

So, in order to efficiently and effectively develop a metrics management program, it is important to establish that philosophy and way of doing business. Everything that an ISSO and staff do can be identified as fitting into one of these two categories: LOE or project.

What Is InfoSec Metrics Management?

InfoSec metrics management is the managing of a CIAPP and related InfoSec functions through the use of metrics. It can be used where managerial tasks must be supported for such purposes as backing the ISSO's position on budget matters, justifying the cost-effectiveness of decisions, or determining the impact of downsizing on providing InfoSec service and support to customers.

The primary process to collect metrics is as follows:

  • Identify each InfoSec function[1];

  • Determine what drives that function, such as labor (number of people or hours used), policies, procedures, and systems; and

  • Establish a metrics collection process. The collection process may be as simple as filling out a log for later summarization and analysis. The use of a spreadsheet that can automatically incorporate InfoSec statistics into graphs is the preferred method. This will make it easier for the ISSO to use the metrics for supporting management decisions, briefings, etc.

The decision to establish a process to collect statistics relative to a particular InfoSec function should be decided by answering the following questions:

  • Why should these statistics be collected?

  • What specific statistics will be collected?

  • How will these statistics be collected?

  • When will these statistics be collected?

  • Who will collect these statistics?

  • Where at what point in the function's process) will these statistics be collected?

By answering these questions for each proposed metric, the ISSO can better analyze whether or not a metrics collection process should be established for a particular function. This thought process will be useful in helping explain it to the InfoSec staff or management, if necessary. It will also help the ISSO decide whether or not the ISSO should continue maintaining that metric after a specific period of time. Since the IWC ISSO had begun with an analysis of InfoSec requirements (drivers) that led to identification of an ISSO charter that led to the identification of InfoSec functions with process flowcharts, the task of developing metrics will be much easier. That is because each step noted in the InfoSec functions' flowcharts can be a point of quantifying and qualifying costs of performing that specific function.

All metrics should be reviewed, evaluated, and reconsidered for continuation at the end of each year, or sooner—when a requirement changes, a function may also change. Remember that although the collection of the metrics information will help the ISSO better manage the InfoSec duties and responsibilities, a resource cost is incurred in the collection and maintenance of these metrics. These resources include:

  • People who collect, input, process, print, and maintain the metrics for you;

  • Time to collect, analyze and disseminate the information; and

  • The hardware and software used to support that effort.

When using these metrics charts for management briefings, one must remember that the chart format and colors are sometimes dictated by management; however, which type of chart is best for analysis or presentation to management is probably up to the ISSO.

The ISSO should experiment with various types of line, bar, and pie charts. The charts should be kept simple and easy to understand. Remember the old saying, "A picture is worth a thousand words." The charts should need very little verbal explanation.

If the ISSO will use the charts for briefings, the briefing should only comment on the various trends. The reason for this is to clearly and concisely present the material, and not get bogged down in details which detract from the objective of the charts.

One way to determine whether the message of the charts is clear is to have someone look at each chart and describe what it tells him or her. If it is what the chart is supposed to portray, then no changes are needed. If not, the ISSO should then ask the viewer what the chart does seem to represent and what leads him or her to that conclusion. The ISSO must then go back to the chart and rework it until the message is clear and is exactly what the ISSO wants the chart to show. Each chart should have only one specific objective, and the ISSO should be able to state that objective in one sentence, such as "This chart's objective is to show that InfoSec support to IWC is being maintained without additional budget although the workload has increased 13%."

The following paragraphs identify some basic examples of InfoSec metrics that can be collected to assist an ISSO in managing a CIAPP and briefing the management on the CIAPP and the InfoSec organization. By the way, when establishing a briefing to management where the metrics charts will be used, a chart similar to Figure 8.1 (as shown in Chapter 8) should be used to start off the briefing. That chart tracks the requirements (drivers) which can be traced to each function. One may also want to provide more detailed charts tracking specific requirements to specific functions (Figure 9.1).

click to expand
Figure 9.1: An example of tracing a requirement to a specific function.

Of course, as the ISSO, you would want to get more specific and track to a more detailed level of granularity. In fact, the InfoSec staff responsible for leading a specific function should be tasked with developing this chart or charts. That way, the staff will know exactly why they are doing what they do. The next step would be for them to track their workflow, analyze it, and find more efficient ways to do the job. At the same time they would also look at current costs and cost-savings as more efficient ways are found to successfully accomplish their jobs.

The ISSO must remember that the use of metrics is a tool to support many of the ISSO's decisions and actions; however, it is not perfect. Therefore, the ISSO must make some assumptions relative to the statistical data to be collected. That's fine. The ISSO must remember that metrics is not rocket science, only a tool to help the ISSO take better-informed actions and make better-informed decisions. So, the ISSO should never get carried away with the hunt for "perfect statistics," or become so involved in metrics data collection that "paralysis by analysis" takes place.[2]

The spreadsheets and graphs used for metrics management can become very complicated with links to other spreadsheets, elaborate 3-D graphics, etc. That may work for some, but the ISSO should consider the KISS (Keep It Simple, Stupid) principle when collecting and maintaining metrics. This is especially true if the ISSO is just getting started and has no or very little experience with metrics. One may find that the project leads who are developing an "automated statistical collection" application are expending more hours developing the application—which never seems to work quite right—than it would take to manually collect and calculate the statistical information.

It is also important, from a managerial viewpoint, that all charts, statistics, and spreadsheets be done in a standard format. This is necessary so that they can be ready at all times for reviews and briefings to upper management. This standard is indicative of a professional organization and one that is operating as a focused team.

ISSOs who are new to the ISSO position, or management in general, may think that this is somewhat ridiculous. After all, what difference does it make as long as the information is as accurate as possible and provides the necessary information? This may be correct, but in the business environment, standards, consistency, and indications of teaming are always a concern of management. Your charts are indicative of those things.

The ISSO has a hard enough job getting and maintaining management support. The job should not be made more difficult than it has to be.

Another negative impact of nonconformance of format will be that the attendees will discuss the charts and not the information on them. Once "nonconformance to briefing charts standards" is discussed, management has already formed a negative bias. Thus, anything presented will make it more difficult to get the point across, gain the decision desired, and meet the established objective of the briefing.

It is better just to follow the established standards than to argue their validity. It is better to save energy for arguing for those things that are more important. After all, one can't win, and the ISSO does not want to be seen as "a non-team player" more than necessary.

Of course the number, type, collection methods, etc., that the ISSO will use will be dependent on the environment and the ISSO's ability to cost-effectively collect and maintain the metrics.

[1]It is assumed each function costs time, money, and use of equipment to perform.

[2]Dr. Kovacich had used approximately 47 metrics charts at various times to assist in managing several large CIAPPs and InfoSec organizations.




The Information Systems Security Officer's Guide. Establishing and Managing an Information Protection Program
The Information Systems Security Officers Guide: Establishing and Managing an Information Protection Program
ISBN: 0750698969
EAN: 2147483647
Year: 2002
Pages: 204

flylib.com © 2008-2017.
If you may any questions please contact us: flylib@qtcs.net