Assessment Logistics

Now that you understand how to develop your organization's theatre of war, attack profiles and plans, defensive posture and remediation techniques, you need to determine how often all of this should be completed. Conducting one vulnerability assessment and believing you will forever be secure is a naive concept. Network perimeters evolve continuously to provide new or upgraded services, retire services no longer in use, and satisfy other requirements of business units on an ongoing basis. Even upgrading the IT infrastructure equipment or installing vendor-supplied patches may cause perimeters to change.

Assessment Frequency

In addition to changes within your organization's perimeter, new digital threats are introduced daily through newly discovered software glitches and system and network reconfiguration. Better- educated attackers are also a factor. In order to provide confidence in your network operations with regard to integrity, confidentiality, and general availability, the perimeter of your network should be analyzed regularly. Organizations must determine the frequency of analysis based on cost, overall impact to the organization, and regulatory requirements they are facing . One thing is for certain: once is not enough, and there is most likely not a security professional out there who will say you can overassess an environment. Ongoing vulnerability assessments are key to ensuring an organization's perimeter is initially secured and, more importantly, stays secure as the organization evolves.

Assessments Internally

There always seems to be a question of who should conduct vulnerability assessments. Many administrators believe they are capable of conducting the assessments themselves . The question to ask is, if administrators are responsible for securing the IT infrastructure, should they also be responsible for checking their own work? The answer is not as simple as yes or no. Administrators should conduct vulnerability testing any time changes are made in the environment. Conceivably, they could even conduct ongoing assessments of the entire infrastructure. But at some point, an organization's management should consider outsourcing vulnerability assessments at some frequency to audit the progress the internal staff makes in securing the perimeter. Of course, larger organizations have an internal IT audit staff whose responsibility it is to perform these assessments organization-wide. They are (usually) managed outside of the IT organizational unit and therefore are unencumbered by potentially restrictive corporate politics.

Assessments Outsourced

When relying on internal resources to conduct assessments, experience and knowledge is limited to the internal team. Partnering with a security firm opens the knowledge base up to an entire team of professionals who specialize in security. The "partner approach" enables administrators to work closely with professionals from security organizations. The goals of this partnership should include more than just obtaining vulnerability assessments. They should also include identifying potential weaknesses, developing documentation of findings, and learning new techniques from the security professionals so that those techniques can be used in the future to help secure the organization's environment. Stated simply, hire a fox to assess the hen house's security, but keep him on a leash.

Before retaining any services from a security firm, there are many questions that should be running through your head. These questions should be discussed internally within your organization and eventually asked of your potential security partners in order to evaluate them. Ten important questions you should ask your potential security partners are listed below. These are not listed in order of importance, since what is important to one organization may mean very little to the next .

• Is a larger firm necessarily better? Larger firms will claim they have more experience because of sheer volume, but you may not be getting the "cream of the crop" in terms of personnel when you retain a larger firm's services.

• Should the firm be local? This may give you an increased sense of security from a business standpoint, but since the assessment should be conducted from outside your network, this may not technically be a concern. How important will it be to meet with the firm face-to-face to discuss findings? If the firm isn't local, will the actual engineers who conduct the assessment travel to meet with you as part of the engagement?

• How flexible should my partner be? If I have legitimate business concerns for delaying ongoing assessments (such as network upgrades, delays in remediation, and so on), will my partner be flexible in conducting assessments? Should I expect my partner to be flexible? If so, this should be a requirement.

• Will my security partner train my internal resources? Many security firms provide a report showing vulnerabilities and expect that if the organization cannot fix the problems itself, it should simply hire out the remediation. Other firms pride themselves on explaining the vulnerabilities in a manner that empowers the organization to complete remediation activities internally by arming them with the relevant knowledge.

• What tools will my partner use? This question is sometimes skirted by security firms. Many are concerned that by answering, they are providing customers with the necessary information to conduct their own assessments. Others are concerned by the lack of commercial tools in use (keep in mind that many of the best tools available are open source).

• Does my partner develop any customized tools? Most clueful security firms are actively involved in creating tools that are used during assessments. These tools may be released to the public or used internally, but the key is to see active development. This helps ensure the firm truly understands the vulnerabilities and that the firm's staff consists of seasoned professionals as opposed to a bunch of knuckleheads following someone else's process and using someone else's tool kits.

• How should my partner ensure my data remains private? The security partner that e- mails your vulnerability assessment report to you (in cleartext) is not looking out for your best interests. The vulnerability assessment will contain the weaknesses of your infrastructure. Sending these via an unencrypted and unsecure communication mechanism does not demonstrate good practices. If you cannot trust delivery of the report itself, how can you trust the work put into the report? Some type of secure data transfer should be used when communicating your organization's sensitive data. Ask how the report is customarily delivered to their clients . Ask if they retain it, how they store it, and so on.

• How does my partner account for ongoing assessments? Reporting should include methods to account for previous assessments, what has been remediated, what has been determined to be an acceptable risk, and what is new for that assessment period. The organization should not have to parse past reports to find differences between assessments.

• How comprehensive is the assessment? A partner's assessment must meet all the requirements of your organization. If it does not, ask if other areas can be added. If the security firm says they cannot comply , shop elsewhere. Refer to the earlier question regarding flexibility.

• How much will the ongoing assessments cost? You cannot base your decision on cost alone. The comprehensiveness of the assessment must be equivalent to the cost of the assessment. Remember, you get what you pay for!

Logistics Summary

The logistical needs regarding vulnerability assessments can vary greatly for each organization. While one organization may have a single department conducting all IT infrastructure security work, another organization may have an entire auditing team dedicated exclusively to conducting internal audits and assessments that have no operational responsibility whatsoever. Whatever the situation, an organization must determine when and how often vulnerability assessments should be conducted and who should be conducting them (whether this should be internal staff, external security vendors , or a mix of both).