Recovering Services After a Security Incident

Once the incident has been controlled and countermeasures are in place against that type of attack, you should begin looking at the restoration of normal operations. Services that have been closed down will be reopened, network connections that have been rerouted will be restored, and systems that have been compromised will be rebuilt and brought online. Of course, it might not be prudent to return to normal operations all at once. For example, if you have terminated all external access to your network as a countermeasure to an attack in progress, turning on every service at once might not be the best course of action.

If all services that have been shut down are brought online at once, it might not be possible to monitor them adequately and ensure that no additional compromise is attempted. The result can be as bad or even worse than the original incident if your countermeasures fail and the attacker regains her foothold on your network. In such a case, the stakes will be higher for the attacker: she might have concerns about covering her tracks to escape retribution for her actions, or her pride might be wounded, or both things might be case. In other words, you should consider an attacker in this position much more dangerous.

You should also be concerned with existing client sessions on any backup servers. If a secondary server is brought back online, either because it was taken offline for evidence or because it had been compromised and was later rebuilt, you need to be aware of the user experience of those accessing such a secondary system. Building a duplicate system with the same name can cause numerous network problems that can result in poor user experience (or worse) for that system s users. Instead, plan and implement a graceful transition that follows operational best practices, such as those described in Microsoft Operations Framework (MOF) under Release Management (http://www.microsoft.com/technet/itsolutions/msm/smf/SMFRELMG.asp). By applying a project management mindset to service restoration, you can avoid many potentially devastating problems.