Continuous Monitoring Process

Overview

Continuous Monitoring is the fourth phase of the security certification and accreditation process and comprises the following three principal activities:

• Configuration management and control
• Security control monitoring and impact analyses of changes to the information system
• Status reporting and documentation

The objective of these tasks is to continuously observe and evaluate the information system security controls during the system life cycle to determine whether changes have occurred that will negatively impact the system security. This information is, then, reported to the authorizing official and the agency senior security officer. If necessary, reaccreditation is performed to ensure that the information system meets the requirements of the system security plan. NIST SP 800-37, “Guide for the Security Certification and Accreditation of Federal Information Systems,” provides details of the continuous monitoring process, and NIST SP 800-53A, “Guide for Assessing the Security Controls in Federal Information Systems,” offers guidance in evaluating information system security controls.

Continuous Monitoring

Continuous monitoring takes place after the initial system security accreditation and involves tracking changes to the information system that occur during its lifetime and determining the impact of those changes on system security. During the lifetime of an information system, necessary changes in hardware, software, and firmware will be implemented. These changes will affect the information system security posture; therefore, an evaluation of the results of these modifications has to be conducted to determine whether corresponding changes have to be made to security controls to return the system to the desired security state. Then, if necessary, appropriate upgrades are made to the security controls, the changes are documented, and the results are reported to the agency authorizing official and senior agency information security personnel. These documents can also be used to meet FISMA requirements for reporting modifications made to address security issues.

NIST SP 800-37 poses the following questions to be asked as part of the continuous monitoring process.

• Could any of the changes to the information system affect the current, identified vulnerabilities in the system or introduce new vulnerabilities into the system?
• If so, would the resulting risk to agency operations, agency assets, or individuals be unacceptable?
• When will the information system need to be reaccredited in accordance with federal or agency policy?

Overall, continuous monitoring involves the following detailed steps:

• Configuration management and control
  • Documentation of information system changes
  • Security impact analysis

• Security control monitoring
  • Security control selection
  • Selected security control assessment

• Status reporting and documentation
  • System security plan update
  • Plan of action and milestones update
  • Status reporting

Configuration management and control ensure the documentation of the proposed or actual changes to the information system. In addition, corresponding updates are made to the system security plan and plan of action. Recall that during the initial certification process, the plan of action and milestones are provided by the information system owner to the authorizing official for use in monitoring the correction of deficiencies discovered during certification. In continuous monitoring, SP 800-37 states that the plan of action and milestones should perform the following functions:

• “Report progress made on the current outstanding items listed in the plan
• Address vulnerabilities in the information system discovered during the security impact analysis or security control monitoring
• Describe how the information system owner intends to address those vulnerabilities (i.e., reduce, eliminate, or accept the identified vulnerabilities)”

This updating of the security plan and plan of action is critical because the information system owner, certification agent, authorizing official, and senior agency information security officer base subsequent security certification and accreditation activities on these plans. Reaccreditation is required when changes to the information system negatively impact the security of the system or when a period of time has elapsed as specified by agency or federal policy.

It is usually not feasible or possible to continuously monitor the entirety of security controls in an information system. Therefore, a recommended course of action is for the information system owner to choose a subset of the controls that can be monitored at intervals, the frequency of which would be a function of the criticality of the information system and its information to the agency and its operations. FIPS 199 security categorizations are useful in determining the importance of different types of information to an agency.

Thus, security controls would be partitioned into two categories: those that are monitored continuously and those that are monitored periodically.

Monitoring Security Controls

Security control monitoring requires choosing the security controls to be monitored and assessing these controls according to methods determined by the owner of the information system. The selection of controls to be monitored can be supported by using FIPS 199 to determine the security categories of the information and information systems and identify the elements that are most critical to the organization. This categorization can, in turn, identify the security controls that, if compromised, would result in the most harm to the agency. The security controls selected for monitoring and the frequency of monitoring should be subject to the approval of the information system owner and authorizing officer.

Once the security controls to be monitored are determined, the next step is to assess whether the controls are performing as required in the system security plan. This task is the responsibility of the information system owner and can be implemented through audits, self-assessments, and other evaluation methods. NIST SP 800-53A provides a standard approach to the assessment of NIST SP 800-53 security controls.

NIST SP 800-53A recommends the following criteria for selecting assessment procedures for an information system’s security controls:

• The specific security controls selected and employed by the organization to protect the information system
• The FIPS 199/Special Publication 800-53 impact level of the information system
• The assurance or level of confidence that the organization must have in determining the effectiveness of the security controls in the information system

Appendix D of NIST SP 800-53A describes three basic types of assessment methods: the interview, the examination, and testing. These approaches are intended to verify that the security control is operating as required, implemented properly, and fulfilling the desired security functions in protecting the information system. A summary of the three approaches is given in the following paragraphs.

The Interview

The interview consists of having focused meetings and interchanges with appropriate personnel in an agency to gain information and evidence relative the effectiveness of security controls. Examples of the individuals to be interviewed include:

• Authorizing officials
• Chief information officers
• Facilities managers
• Human resource managers
• Information owners
• Information system operators
• Information system owners
• Information system security managers
• Information system security officers
• Network and system administrators
• Personnel officers
• Physical security officers
• Senior agency information security officers
• Site managers
• Training officers
• Users

Depending on the level of assessment conducted, NIST SP 800-53A defines the following three types of interview:

• Abbreviated - Informal, ad hoc interviews that consist of generalized, high-level discussions with selected organizational personnel on particular topics relating to the specifications, mechanisms, or activities associated with the security control being assessed
• Substantial - Informal, structured interviews that consist of generalized, high-level discussions and specific discussions in targeted areas with selected organizational personnel on particular topics relating to the specifications, mechanisms, or activities associated with the security control being assessed
• Comprehensive - Formal, structured interviews that consist of generalized, high-level discussions and specific, in-depth discussions with selected organizational personnel on particular topics relating to the specifications, mechanisms, or activities associated with the security control being assessed

The Examination

The examination assessment method is used to review, inspect, and analyze assessment objects such as policies, plans, requirements, designs, hardware, firmware, and security activities to determine the effectiveness of information system security controls. This activity is effective for looking into the details of security policies, reviewing audit trails and logs, evaluating backup procedures, examining contingency plans and practice drills, and evaluating incident response procedures. If the results of previous security control assessments are available, they should also be reviewed as part of the examination process. As in the interview process, the depth of the examination can be abbreviated, substantial, or comprehensive. The characteristics of these three depth levels are defined in NIST SP 800-53A as follows:

• Abbreviated - Examinations that consist of brief, high-level reviews, observations, or inspections of selected specifications, mechanisms, or activities associated with the security control being assessed using a limited body of evidence or documentation. These types of examinations are typically conducted using only functional-level descriptions of specifications, mechanisms, or activities, and they employ checklists or other similar assessment techniques consistent with an abbreviated assessment period.
• Substantial - Examinations that consist of detailed analyses, observations, or studies of selected specifications, mechanisms, or activities associated with the security control being assessed using a body of evidence or documentation that is greater than that available during abbreviated examinations. These types of examinations are typically conducted using functional-level descriptions of specifications, mechanisms, or activities and, where appropriate, high-level design information. Substantial examinations employ a variety of analysis techniques and require a longer assessment period than abbreviated examinations do.
• Comprehensive - Examinations that consist of detailed and thorough analyses, observations, or studies of selected specifications, mechanisms, or activities associated with the security control being assessed using a body of evidence or documentation that is greater than that available during substantial examinations. These types of examinations are typically conducted using functional-level descriptions of specifications, mechanisms, or activities, and where appropriate, high-level design, low-level design, and implementation-related information (e.g., source code). Comprehensive examinations employ a variety of sophisticated analysis techniques and require a longer assessment period than substantial examinations do.

Testing

The testing form of assessment involves observing or conducting the operation of physical devices, hardware, software, and firmware and determining whether they exhibit the desired and expected behavior. Examples of testing assessment include tests of:

• Encryption devices
• Contingency plans
• Information system penetration
• Access control mechanisms
• Previous test and audit results
• System backups

The scope of a test is characterized by one of the following three definitions from NIST SP 800-53A:

• Functional testing (black-box testing) - Assumes knowledge of the functional specifications, high-level design, and operating specifications of the item under assessment.
• Structural testing (gray-box, white-box testing) - Assumes (some) explicit knowledge of the internal structure of the item under assessment (e.g., low-level design, source code implementation representation).
• Penetration testing - A test methodology in which assessors, using all available documentation (e.g., system design, source code, manuals) and working under no constraints, attempt to circumvent the security features of an information system.

Figure 15-1, from NIST SP 800-53A, summarizes the attributes of assessment methods based on the information system impact level.

Open table as spreadsheet

ASSESSMENT METHODS: Interview, Examine, Test		INFORMATION SYSTEM IMPACT LEVEL
ATTRIBUTE	VALUE	LOW	MODERATE	HIGH
Depth (Interview and examine methods only)	Abbreviated	√	- - -	- - -
	Substantial	- - -	√	- - -
	Comprehensive	- - -	- - -	√
Scope (Test method only)	Functional (black-box)	√	√	√
	Penetration	- - -	√	√
	Structural (gray-box, white-box)	- - -	- - -	√
Coverage (All methods)	Number and types of assessment objects determined by organizations in collaboration with assessors.21	√	√	√

Figure 15-1: Assessment method attributes.

If the assessment reveals that the security controls are not meeting the expected assurance requirements, the system security plan and plan of action have to be updated to indicate corrective actions required.

Configuration Management and Control

This task is concerned with documenting any proposed or actual changes to the agency information system and identifying the impact of those changes on the security of the affected information system and on its accreditation. The configuration management and control task is the responsibility of the information system owner.

The agency should apply standard configuration management methods and tools to track proposed or actual changes to the information system, including operating system patches, software upgrades, hardware and firmware changes, and other modifications to the computing environment. Configuration management methods are discussed in detail in Chapters 6 and 7 of this text.

Once the proposed or actual changes to information system are identified and placed under configuration management, the next step is to determine the impact of those changes on the security of the information system. This activity typically includes checking for weakening of existing controls, exposing new vulnerabilities, or identifying areas where additional security controls are required. If the impact analysis indicates that the security and accreditation posture of the information is or will be compromised by the information system changes, compensating controls should be initiated and the plan of action should be updated. Any changes should be coordinated with users and other relevant agency personnel.

NIST SP 800-37 defines security impact analysis as “The analysis conducted by an agency official, often during the continuous monitoring phase of the security certification and accreditation process, to determine the extent to which changes to the information system have affected the security posture of the system.”

Environment Monitoring

The information system owner is responsible for monitoring the information system environment for factors that can potentially negatively impact the security of the system and its accreditation. These factors can be the result of legal, political, weather-related, human-initiated, physical, and other types of events. Typical examples of such events are:

• Power failures
• Facility damage
• Floods
• Storms
• Earthquakes
• Sabotage
• Strikes
• Warfare
• Terrorist acts
• Legal actions
• Political actions
• Chemicals
• Water damage
• Pollution
• Hackers
• Viruses and other malware
• Attacks originating from the Internet
• Internal threats

If specific threats are applicable to a particular agency, then these threats should be used in the determination of security controls for the agency information systems. FIPS 199 security categories are useful in determining the impact level of a particular threat on the agency systems.

Documentation and Reporting

An important part of continuous monitoring is documenting the status of the information system and reporting this information to the authorizing official and agency information security officer. Documentation includes making any changes to the system security plan that delineate any changes made or proposed to be made to the information system and updating the plan of action and milestones. These reports are used to meet the FISMA reporting requirements and determining whether recertification is necessary.

The information system owner is responsible for updating the system security plan, which should include all changes made to the information system. This updating should be done at reasonable intervals to ensure that significant information system changes are reported.

Based on the changes to the information system described in the system security plan, the information system owner is also responsible for updating the plan of action and milestones document. The plan of action and milestones should include the handling of vulnerabilities identified by the security impact analysis and the status of outstanding issues listed in the plan. The authorizing official, senior agency information security officer, information system owner, and security assessor will be using the updated plans to guide future security assessment activities.

As with the system security plan, the frequency of generating the plan of action and milestones is at the discretion of the information system owner but should be done at reasonable intervals to ensure that significant changes to the security posture of the information system are reported. The continuous monitoring results should also be considered.

The plan of action and milestones are used by the senior agency information system security officer and the authorizing official to determine whether a security reaccreditation is required. If the decision is that reaccreditation is necessary, the authorizing official will inform the information system owner of the decision.

Reaccreditation should be initiated if one or more of the following events have occurred:

• Modifications to the information system have negatively impacted the system security controls.
• Modifications to the information system have introduced new vulnerabilities into the system.
• The risk to agency operations, agency assets, or individuals has been increased.
• A specified time period has elapsed, requiring the information system to be reauthorized in accordance with federal or agency policy (typically 3 years).

Assessment Questions

You can find the answers to the following questions in Appendix A.

1.	“Continuously observing and evaluating the information system security controls during the system life cycle to determine whether changes have occurred that will negatively impact the system security” best describes which process in the certification and accreditation methodology? Continuous monitoring Continuous improvement Continuous management Continuous development
2.	Which one of the following activities is not a component of the continuous monitoring process? Operation and maintenance Security control monitoring and impact analyses Status reporting and documentation Configuration management and control
3.	Which one of the following publications provides details of the continuous monitoring process? NIST SP 800-14 NIST SP 800-42 NIST SP 800-37 NIST SP 800-41
4.	Which one of the following best describes when continuous monitoring takes place? Before the initial system certification After the initial system security accreditation Before and after the initial system security accreditation During the system design phase
5.	Which one of the following questions is not asked as part of the continuous monitoring process? Could any of the changes to the information system affect the current, identified vulnerabilities in the system or introduce new vulnerabilities into the system? If new vulnerabilities are introduced into an information system, would the resulting risk to agency operations, agency assets, or individuals be unacceptable? What maintenance schedule should be followed during the operation/maintenance phase of the information system? When will the information system need to be reaccredited in accordance with federal or agency policy?
6.	In configuration management and control, if necessary, updates have to be made to which of the following documents? System security plan System security plan and plan of action and milestones Plan of action and milestones System deficiency report and plan of action and milestones
7.	Which one of the following documents should report progress made on the current outstanding items and address vulnerabilities in the information system discovered during the security impact analysis or security control monitoring? Plan of action and milestones System security plan System security plan and plan of action and milestones System deficiency plan
8.	What process should be initiated when changes to the information system negatively impact the security of the system or when a period of time has elapsed as specified by agency or federal policy? Incident response Systems engineering Reaccreditation Reclassification of data
9.	What course of action is recommended when it is not feasible or possible to continuously monitor the entirety of security controls in an information system? Begin the reaccreditation process Begin the recertification process Enter the system development life cycle (SDLC) Select subsets of controls and monitor them at intervals
10.	Selecting controls to be monitored can be best aided by what document? FIPS 199 NIST SP 800-37 FISMA NIST SP 800-18
11.	What document provides a standard approach to the assessment of NIST SP 800-53 security controls? FIPS 199 NIST SP 800-53A NIST SP 800-30 NIST SP 800-66
12.	Appendix D of NIST SP 800-53A describes what three basic types of assessment methods? The interview, the examination, and testing The interview, the validation, and testing The interview, the examination, and remediation The interview, the verification, and testing
13.	NIST SP 800-53A defines which of the following three types of interviews, depending on the level of assessment conducted? Initial, substantial, comprehensive Abbreviated, substantial, comprehensive Abbreviated, moderate, comprehensive Abbreviated, substantial, detailed
14.	What NIST SP 800-53A assessment method is used to review, inspect, and analyze assessment objects such as polices, plans, requirements, designs, hardware, firmware, and security activities to determine the effectiveness of information system security controls? Verification Interview Examination Validation
15.	Observing or conducting the operation of physical devices, hardware, software, and firmware and determining whether they exhibit the desired and expected behavior describes what type of SP 800-53A assessment method? Examination Testing Validation Remediation
16.	In continuous monitoring, tracking of proposed or actual changes to the information system, including operating system patches, hardware, software, and firmware is called: Systems engineering The system development life cycle (SDLC) Configuration management and controls Security categorization
17.	Determination of the effect of changes to the information system on the security of the information system is called: Validation analysis Verification Impact analysis Continuous improvement
18.	Who is responsible for monitoring the information system environment for factors that can potentially negatively impact the security of the system and its accreditation? The information system owner Chief information officer (CIO) The user Accrediting officer
19.	Which of the following items are types of factors that can potentially negatively impact the security of the system and its accreditation? Legal Human-initiated Weather-related All the above
20.	What guidance document is useful in determining the impact level of a particular threat on agency systems? FIPS 199 NIST SP 800-53 NIST SP 800-14 NIST SP 800-41
21.	Documentation is an important part of continuous monitoring. In this context, documentation comprises which of the following activities? Making changes to the security plan that address any changes or proposed changes to the information system Updating the plan of action and milestones Establishing the accreditation boundary a and b
22.	As part of the documentation process, reports are usually sent to which of the following personnel in the agency? Authorizing official Authorizing official and senior agency information security officer Senior agency information security officer User
23.	In continuous monitoring, what personnel will normally be using the updated plans in the documentation report to guide future assessment activities? The senior agency information security officer The authorizing official The information system owner and security assessor All the above
24.	The frequency of generating the system security plan and the plan of action and milestones is at the discretion of which of the following personnel? The authorizing official The information system owner The agency information system security officer All the above
25.	Generating the system security plan and plan of action and milestones should be done at what frequency? Every three months Reasonable intervals to ensure that significant changes to the security posture of the information system are reported At the discretion of the authorizing official Every three years
26.	Who determines whether a security reaccreditation is required after reviewing the plan of actions and milestones? The senior information system security officer The authorizing official The senior information security officer and the authorizing official The information system owner
27.	The following events are used to determine whether which activity has to be initiated? Modifications to the information system have negatively impacted the system security controls. Modifications to the information system have introduced new vulnerabilities into the system. A specified time period has elapsed, requiring the information system to be reauthorized in accordance with federal or agency policy (typically 3 years). The risk to agency operations, agency assets, or individuals has been increased. Reaccreditation Maintenance Peer review Security categorization
28.	Continuous monitoring documentation reports are also used to meet which one of the following reporting requirements? NIST FISMA HIPAA FBI
29.	Power failures, floods, earthquakes, and sabotage are examples of what types of events? Events that can potentially negatively impact the security of the system and its accreditation Events that cannot be taken into consideration during the impact analysis process Events that are out of one’s control and, therefore, cannot be accounted for in risk analysis Events for which the associated risk can be reduced to zero if proper precautions are taken
30.	NIST SP 800-53A defines a form of testing as one that “assumes (some) explicit knowledge of the internal structure of the item under assessment (e.g., low-level design, source code implementation representation).” Which one of the following items is that form of testing? Validation Black-box Structural Evaluation
31.	What are the types of assessment tests addressed in NIST SP 800-53A? Functional, structural, penetration Functional, evaluation, penetration Validation, structural, black-box Validation, structural, penetration
32.	A test methodology in which assessors, using all available documentation (e.g., system design, source code, manuals) and working under no constraints, attempt to circumvent the security features of an information system is defined in NIST SP 800-53A as what type of test? Validation test Functional test Structural test Penetration test
33.	In the continuous monitoring examination assessment method, three examination depth levels are defined in NIST SP 800-53A. The definition “examinations that consist of brief, high-level reviews, observations, or inspections of selected specifications, mechanisms, or activities associated with the security control being assessed using a limited body of evidence or documentation” refers to which one of the following examination assessment types? Functional Abbreviated Substantial Comprehensive

Answers

1.	Answer: a The answer a is correct. The other answers are distracters.
2.	Answer: a Operation/maintenance is a component of the system development life cycle (SDLC) and is not one of the elements of continuous monitoring.
3.	Answer: c Answer c, NIST SP 80-37 “Guide for the Security Certification and Accreditation of Federal Information Systems,” is correct. NIST 800-14, “Generally Accepted Principles and Practices for Securing Information Technology” (answer a) lists eight principles for securing information technology systems and 14 security practices. NIST SP 800-42 (answer b) is the “Guideline on Network Security Testing,” and NIST SP 800-41 (answer d) provides “Guidelines on Firewalls and Firewall Policy.”
4.	Answer: b Continuous monitoring is aimed at determining whether any changes have occurred to the information system security posture following the initial system certification.
5.	Answer: c Answers a, b, and d are the three questions asked in NIST SP 800-37.
6.	Answer: b The system security plan and the plan of action and milestones are the documents that may have to be updated. Answer d is a made up distracter.
7.	Answer: a
8.	Answer: c The information system should be reaccredited because new vulnerabilities have been found that are not adequately protected by existing security control mechanisms.
9.	Answer: d The answer d is correct. Answers a and b are incorrect because, at this stage, it has not yet been determined whether new vulnerabilities have been exposed. The controls have to be monitored first. Answer c is a made-up distracter.
10.	Answer: a FIPS 199 security categories can be used to identify elements that are most critical to the organization and the corresponding security controls that, if compromised, would result in the most damage to the system.
11.	Answer: b NIST SP 800-53A (answer b) is the “ Guide for Assessing the Security Controls in Federal Information Systems.” FIPS 199 (answer a) provides guidelines for security categorizations; SP 800-30 (answer c) delineates guidelines for risk management; and NIST SP 800-66 (answer d,) is the “Introductory Resource Guide for Implementing the HIPAA Security Rule.”
12.	Answer: a The answer a is correct. The other answers are made-up distracters.
13.	Answer: b The answer b is correct. The other answers are made-up distracters.
14.	Answer: c The correct answer is c, examination, by definition.
15.	Answer: b The answer b, testing, is correct. Answer a, examination, is another SP 800-53A assessment method, and answers c and d are made-up distracters.
16.	Answer: c The correct answer is c, by definition.
17.	Answer: c The correct answer is c, by definition.
18.	Answer: a The correct answer is a, the information system owner.
19.	Answer: d
20.	Answer: a FIPS 199, (answer is a) is the “Standard for Security Categorization of Federal Information Systems.” The categories of FIPS 199 provide the framework for determining the impact level of specific threats. NIST SP 800-53 (answer b) is the “Recommended Security Controls for Federal Information Systems; NIST 800-14 (answer c) is “Generally Accepted Principles and Practices for Securing Information Technology,” which lists eight principles for securing information technology systems and 14 security practices. NIST SP 800-41 (answer d) which provides “Guidelines on Firewalls and Firewall Policy.”
21.	Answer: d Documentation includes both making changes to the security plan that address any changes or proposed changes to the information system and updating the plan of action and milestones.
22.	Answer: b The documentation report should be sent to the authorizing official and senior agency information security officer on a regular basis.
23.	Answer: d All these personnel will be involved in planning future assessment activities.
24.	Answer: b The information system owner has discretion over how frequently these documents are generated.
25.	Answer: b The frequency of plan generation is at the discretion of the information system owner.
26.	Answer: c If the decision is that reaccreditation is necessary, the authorizing official will inform the information system owner of the decision.
27.	Answer: a Any of these events makes reaccredidation necessary.
28.	Answer: b
29.	Answer: a The answer a is correct. Relative to answers b and c, these types of events are taken into account during impact analysis and risk analysis. Answer d is incorrect because risk can never be completely eliminated.
30.	Answer: c
31.	Answer: a The answer a is correct. In the other answers, evaluation and validation types are made-up distracters. Black-box testing is another word for functional testing.
32.	Answer: d
33.	Answer: b