Data Center Operations

Effective data center operations require strict adherence to formally adopted policies, procedures, and plans. These documents should be used for determining who is granted access to the data center and what access they are granted, determining when facility-based systems are to be scheduled for maintenance, and determining which actions should be taken during an emergency. Some of the areas that should be covered by these policies, procedures, and plans include

• Physical access control

• Facility monitoring

• Roles and responsibilities of data center personnel

• Segregation of duties of data center personnel

• Responding to emergencies and disasters

• Facility and equipment maintenance

• Data center capacity planning

• Data center

One example of why documented policies, plans, and procedures are so important might include a scenario where a generator catches fire while being tested. Without clear procedures and proper training, we probably would witness employees running around in the heat of the moment responding in a way that they think is most appropriate but most likely not working together to solve the problem. With clear emergency response procedures, the decisions would have been thought out ahead of time, and employees would not be forced to make decisions in the heat of the moment. We most likely would witness a more coordinated response.

24 Ensure that physical access control procedures are comprehensive and being followed by security staff.

Physical access control procedures govern employee and guest access to the data center facility. If physical access control procedures are incomplete or not enforced consistently, data center physical access will be compromised.

How

When reviewing physical access control procedures, the auditor should do the following:

• Ensure that access authorization requirements are clearly defined for both employees and guests.

• Verify that guest access procedures include restrictions on taking pictures and outline conduct requirements within the data center.

• Review a sample of both guest access and employee ID authorization requests to ensure that access control procedures are followed.

25 Review facility monitoring procedures to ensure that alarm conditions are addressed promptly.

Facility monitoring procedures ensure that all critical alarm conditions are captured and acted on promptly. They should include a description of the alarm systems that will be monitored, as well as the steps that are to be taken in the event of all reasonably foreseeable alarms, including fire, burglar, water, power outage, data circuit outage, system, and system component alarm conditions. The lack of system monitoring procedures could result in unnecessary risk to information systems and data center facilities.

How

When auditing facility monitoring procedures, the auditor should do the following:

• Ensure that all critical systems and facility alarms are defined as "monitored systems" in the procedure.

• Verify that alarm-condition response is clearly outlined for each type of alarm condition.

The auditor should be able to obtain the actual monitoring procedures as well as monitoring logs from data center facility management.

26 Verify that network, operating system, and application monitoring provides adequate information to identify potential problems.

System monitoring provides insight into potential problems resulting from capacity issues, misconfigurations, and system component failures. Inadequate system monitoring gives rise to the threat of security violations going undetected and system outages.

How

Although this function typically is managed by IT service groups rather than data center personnel, monitoring is a critical component of a sound IT strategy. System monitoring encompasses the monitoring of network devices, intrusion detection systems, operating systems, system hardware, and applications. Whereas intrusion detection system monitoring is focused primarily on monitoring for security violations, network device, operating system, system hardware, and application monitoring is focused primarily on items that can affect the availability of a system, such as hard-disk usage, number of concurrent connections, and so forth. Therefore, when auditing monitoring system procedure, it is important to understand the objective of the system.

The auditor should understand the criticality of specific system components and verify that monitoring systems provide near-real-time information to detect a problem with these system components. Additionally, the auditor should review monitoring logs and reports to identify instances where components being monitored exceed predetermined thresholds and then verify that actions have been taken to remediate the condition. Monitoring logs and reports typically can be obtained from system support groups, network support groups, and security and application monitoring teams.

27 Ensure that roles and responsibilities of data center personnel are clearly defined.

Well-defined employee roles and responsibilities ensure that responsibility and accountability for data center functions are clear. Inadequate roles and responsibilities would result in unclear job boundaries and data center functions going unaddressed, which could increase the risk of system outages.

How

When auditing data center roles and responsibilities, the auditor should ensure that all job functions are covered and that responsibilities associated with job functions are clearly defined. Data center facility management should be able to provide job descriptions, including roles and responsibilities.

28 Verify that duties and job functions of data center personnel are segregated appropriately.

Segregation of duties is a basic security precept of personnel management. The goal is to spread high-risk functions across two or more employees to reduce the risk of fraud or inadvertent errors. One example of segregation of duties is a check-payment procedure where one person is responsible to creating the check, another is responsible for posting the check to the accounting system, and a third is responsible for signing the check. If high-risk functions are not segregated, the data center will have a higher degree of fraud risk.

How

When reviewing the data center's segregation of duties, the auditor should verify that high-risk job functions, such as access authorization, are segregated across two or more employees. These processes should be tracked with logs and forms that can be reviewed to verify that duties are segregated effectively.

29 Ensure that emergency response procedures address reasonably anticipated threats.

When a fire breaks out or a data center floor begins to flood, data center personnel need a clear plan to address the condition and minimize losses. Although used only during the unlikely event of an emergency, emergency response plans are absolutely critical for reducing the risk of an emergency escalating owing to improper response from data center personnel.

How

Data centers are faced with various threats, including

• Fire

• Flood

• Physical or logical intrusion

• Power loss

• System failure

• Telecommunications outages

These and other identified threats should be addressed by emergency response plans. When auditing data centers, the auditor should verify that plans are present for all foreseeable threats and ensure that response procedures are comprehensive and well thought out. Data center operations staff should be able to provide these plans.

30 Verify that data center facility-based systems and equipment are maintained properly.

When not properly maintained, facility-based systems and equipment are prone to premature failure. These breakdowns can cause loss of information and system outages. As a result, maintenance is critical.

How

Critical systems and equipment should be maintained at least semiannually. The auditor should review maintenance logs for critical systems and equipment. The data center facility manager should be able to provide the maintenance logs.

31 Ensure that data center personnel are trained properly to perform their job functions.

Data center personnel cannot be expected to be proficient if they are not afforded job training. When not trained properly, data center personnel are more likely to cause data loss or system outages due to mistakes.

How

Auditors should review training history and schedules during data center audits. When reviewing training, the auditor should ensure that training is relevant to job functions and that all data center personnel are afforded training. Data center management should be able to provide access to training history and schedules. The auditor should review history for the past full year and schedules for the next 6 months.

32 Ensure that data center capacity is planned to avoid unnecessary outages.

Capacity planning ensures that procedures are in place to monitor and analyze factors that could impact the data center's current or future power, network, heating, ventilation, air-conditioning, and space requirements. Inadequate capacity planning could result in data loss, system outages, and/or delays in system deployments.

How

Capacity management is a broad topic that was covered in more detail in Chapter 3. A well-managed data center will be able to forecast how much rack space, network drops, network gear, electricity, and heating, ventilation, and air conditioning, just to name a few, are needed to support current and future operations.

When auditing capacity planning, the auditor should review monitoring thresholds and strategies that data center management uses to determine when facilities, equipment, or networks require upgrading. Data center management should be able to provide the capacity planning strategy, including thresholds for upgrading systems.

33 Verify that procedures are present to ensure secure storage and disposal of electronic media.

Electronic media often contain sensitive information that, if disclosed, would constitute a compromise of information security. As a result, media storage and disposal must be closely controlled. Additionally, improper storage of electronic media could result in accidental corruption of the information stored on the media.

How

The auditor should ensure that the following media storage and disposal controls exist within the data center:

• Electronic media are stored in a dry, temperature-controlled, and secure environment.

• Electronic media containing sensitive information are encrypted and tracked as they move from one location to another.

• Electronic media are degaussed, overwritten with a Department of Defense (DOD)-compliant electronic shredding utility, or physically destroyed prior to disposal.

The auditor should be able to obtain media tracking, storage, and disposal records from data center management. Additionally, the auditor should tour electronic media storage facilities within the data center to verify that appropriate access control and environmental controls are in place. For more information regarding electronic media management audit, see Chapter 3.