Section 18.3. Security Administrators

18.3. Security Administrators

Over the last two years, we have conducted a number of ethnographic field studies looking at database management, web hosting, operating system administration, and security administration. We observed activities ranging from database backup and recovery to configuration of geographic load balancers, from patch management to computer security forensics. Most system administrator activities are either directly related to security or have security implications.

In this section, we profile two typical security administrators, Joe and Aaron. In the following section, we describe five security administration case studies from our observations. Finally, based upon our observations and interviews, we discuss the unique aspects of security administration work, and how well this work is supported by available tools.

18.3.1. Profile of a Security ManagerJoe

Joe is a senior security engineer at a computing center in a large public university. He has been in this position for over four years. Previously, he worked as a system administrator at the same center and in various software development positions at different companies. The computing facilities at the center provide services for about 300 employees and external collaborators throughout the U.S. Joe manages a team of three other security administrators. Together their responsibilities include setting up, configuring, and monitoring intrusion detection systems and responding to alerts reported. They work to proactively protect systems, detect attacks, and perform forensic analysis on compromised systems. As a manager, he is also responsible for defining policies and dealing with policy issues, and interacting with other concerned groups within the university as well as at other institutions. His motto is "Know Thy Network." Joe proudly claims that he learned most of the necessary skills primarily on the job. As shown in Figure 18-1, Joe's office has a bookcase full of various kinds of puzzles and games, and he considers this hobby as a nice metaphor for his day job.

Figure 18-1. Puzzles in Joe's office

Joe typically starts the day at 9:00 a.m. One of the first things he does is to start clients to read email and a MOO (An Object Oriented MUDMultiple User Dialog). MOO is essentially a persistent messaging system composed of a set of virtual rooms representing dedicated spaces for admins to communicate on pre-agreed topics. Joe likes the MOO system, as it provides him a way to catch up on things quickly in the morning using its history function that shows past messages. Joe frequently checks both the MOO and email throughout the day. While the MOO allows him to feel the pulse of administration activity at the university, email provides updates from automated monitoring tools as well as security news from the outside world. Joe also participates in regular meetings and phone conferences with his counterparts at other organizations to share information concerning recent and emerging threats. When he hears about new vulnerabilities through these channels, Joe often researches them further using a variety of web sites, some created by the security community, others by hackers.

Joe and others recently finished a five-month project to revamp the center's security policies. Because they work at a center within a larger university, they need to abide by the university's policies, yet they also need more restrictive policies in areas such as file sharing, wireless networks, connecting personal computers to the university network , external collaborator use, and so on. One of the biggest problems is to make the policies usable by making the policy document short enough so that all employees can read, remember, and follow it. Joe thinks a lot of education still needs to be done, as policy issues still do come up frequently. Users often read the policy document only once, when required to at the beginning of their employment, so continuing education and discussion are needed.

Joe typically leaves work around 5:00 p.m., but he quickly logs back in again from home so that he can be in touch with his colleagues. At 9:00 p.m., email reports from the center's change management and analysis tool are sent, and he usually likes to look at these before going to bed so that he can sleep with some peace of mind. Joe has been officially on call 24/7 for the last eight years, although he has received few off-hour calls in the last few years. He really enjoys the challenges of his work, but at times it can be a little too demanding on his personal life.

18.3.2. Profile of a Security EngineerAaron

Aaron is a security engineer who reports to Joe. He graduated with a master's degree in Computer Science last year, and has been working in this position ever since. He shares an office with Tom, another security administrator, and their office is full of computers and displays, as shown in Figure 18-2.

Aaron's schedule is similar to Joe's, but offset an hour or two later. The security group members deliberately stagger their hours to ensure better coverage at both ends of the workday. In fact, one of their team members works remotely from several time zones away, further increasing coverage. The rhythm of Aaron's day is centered on email, which he checks frequently (5 to 10 times per hour), looking for email notifications of security alerts . Through experience, he knows that certain alerts may be ignored on-sight, and others cause him to perform an immediate investigation such as looking through monitoring logs, checking network ports, searching for vulnerability/exploit data on the Web, and consulting with his co-workers via the MOO and face-to-face. He also checks the MOO regularly, but less frequently than email; he uses the MOO primarily for getting advice from more experienced admins.

When Aaron is not investigating a particular alert, he has assigned projects, such as scanning all the machines on the network for a newly discovered vulnerability. Any free time is occupied reading security mailing lists, though Aaron receives more mail from these lists than he has time to read. When he hears about new vulnerabilities through alerts or mailing lists, he usually performs web searches to learn more, and he occasionally downloads

Figure 18-2. Aaron's workspace

sample code for the attack to better understand how it works and which machines are vulnerable. Aaron has fewer meetings than Joe, and during meetings he always continues to monitor the network and research new potential attacks from his laptop. At the end of the day, Aaron heads home and, like his manager, connects in again after dinner to continue his tasks through the evening.

Aaron is the most junior member of the security team, but his educational background is strong. He spends much of his time educating himself to improve his knowledge of security. He is very motivated and really enjoys his work, and he seems to like the role of being a "good guy" helping fend off the "bad guys."