The Role of Education

The Role of Education

I mentioned that security education is a pet subject of mine or, more accurately, the lack of security education is a pet peeve of mine, and it really came to a head during the Windows Security Push in the first quarter of 2002. Let me explain. During the push, we trained about 8500 people in ten days. The number of people was substantial because we made it mandatory that anyone on a team that contributed to the Windows CD (about 70 groups) had to attend the training seminars, including vice presidents! We had three training tracks, and each was delivered five or six times. One track was for developers, one was for testers, and one was for program managers. (In this case, program managers own the overall design of the features of the product.) Documentation people went to the appropriate track dictated by their area of expertise. Some people were gluttons for punishment and attended all three tracks!

Where am I going with this? We trained all these people because we had to. We knew that if the Windows Security Push was going to be successful, we had to raise the level of security awareness for everybody. As my coauthor, David, often says, People want to do the right thing, but they often don't know what the right thing is, so you have to show them. Many software developers understand how to build security features into software, but many have never been taught how to build secure systems. Here's my assertion: we teach the wrong things in school or, at least, we don't always teach the right things. Don't get me wrong, industry has a large role to play in education, but it starts at school.

The best way to explain is by way of a story. In February 2002, I took time out from the Windows Security Push to participate in a panel discussion at the Network and Distributed System Security Symposium (NDSS) in San Diego on the security of Internet-hosted applications. I was asked a question by a professor that led me to detail an employment interview I had given some months earlier. The interview was for a position on the Secure Windows Initiative (SWI) team, which helps other product teams design and develop secure applications. I asked the candidate how he would mitigate a specific threat by using the RSA (Rivest-Shamir-Adleman) public-key encryption algorithm. He started by telling me, You take two very large prime numbers, P and Q. He was recounting the RSA algorithm to me, not how to apply it. I asked him the question again, explaining that I did not want to know how RSA works. (It's a black box created and analyzed by clever people, so I assume it works as advertised.) What I was interested in was the application of the technology to mitigate threats. The candidate admitted he did not know, and that's fine: he got a job elsewhere in the company.

By the way, the question I posed was how you would use RSA to prevent a person selling stock from reneging on the transaction if the stock price rose. One solution is to support digitally signed transactions using RSA and to use a third-party escrow company and timestamp service to countersign the request. When the seller sells the stock, the request is sent to the third party first. The company validates the user's signature and then timestamps and countersigns the sell order. When the brokerage house receives the request, it realizes it has been signed by both the seller and the timestamp service, which makes it much harder for the seller to deny having made the sell order.

The principle skill I was looking for in the interview was the ability, in response to a security problem, to apply techniques to mitigate the problem. The candidate was very technical and incredibly smart, but he did not understand how to solve security problems. He knew how security features work, but frankly, it really doesn't matter how some stuff works. When building secure systems, you have to know how to alleviate security threats. An analogy I like to draw goes like this: you go to a class to learn to defensive driving, but the instructor teaches you how an internal combustion engine works. Unless you're a mechanic, when was the last time you cared about the process of fuel and air entering a combustion chamber and being compressed and ignited to provide power? The same principle applies to building secure systems: understanding how features work, while interesting, will not help you build a secure system.

IMPORTANT
Make this your motto: Security Features != Secure Features.

Once the panel disbanded, five professors marched up to me to protest such a despicable interview question. I was stunned. They tried convincing me that understanding how RSA worked was extremely important. My contention was that explaining in an exam answer how RSA works is fairly easy and of interest to only a small number of people. Also, the exam taker's answer is either correct or incorrect; however, understanding threat mitigation is a little more complex, and it's harder to mark during an exam. After a lively debate, it was agreed by all parties that teaching students how to build secure systems should comprise learning about and mitigating threats and learning how RSA and other security features work. I was happy with the compromise!

Now back to the Windows Security Push. We realized we had to teach people about delivering secure systems because the chances were low that team members had been taught how to build secure systems in school. We realized that many understood how Kerberos, DES (Data Encryption Standard), and RSA worked but we also knew that that doesn't help much if you don't know what a buffer overrun looks like in C++! As I often say, You don't know what you don't know, and if you don't know what makes a secure design, you can never ship a secure product. Therefore, it fell on our group to raise the security awareness of 8500 people.

What Should We Teach Students?

We need more education regarding secure design, secure coding, and more thorough testing. A good, well-rounded, three-semester course on systems security would cover general security concepts and threat analysis in the first semester, understanding and applying threat mitigation techniques in the second, and practicing designing and building a real system in the third. The student would learn that systems should be built not only to serve the business or customer but also to serve the business or customer securely. The course should provide the student with balanced doses of security theory and security technology.

The net of this is that you have to train people about security issues and you have to train them often because the security landscape changes rapidly as new threat classes are found. The saying, What you don't know won't harm you is simply not true in the area of security. What you do not know can (and probably will) leave your clients open to serious attack. You should make it mandatory for people to attend security training classes (as Microsoft is doing). This is especially true for new employees. Do not assume new hires know anything about secure systems!

IMPORTANT
Education is critical to delivering secure systems. Do not expect people to understand how to design, build, test, document, and deploy secure systems; they may know how security features work, but that really doesn't help. Security is one area where What I don't know won't hurt me does not apply; what you don't know can have awful consequences.

Resistance to Mandatory Training

We were worried that mandatory training would have negative connotations and be poorly received. We were amazed to find we were completely wrong. Why were we wrong? Most software development organizations are full of geeks, and geeks like learning new things. If you give a geek an opportunity to learn about a hot topic such as security, she or he will actively embrace it. So provide your geeks with the education they need! They yearn for it.

NOTE
While we're on the subject of geeks, don't underestimate their ability to challenge one another. Most geeks are passionate about what they do and like to hold competitions to see who can write the fastest, tightest, and smallest code. You should encourage such behavior. One of my favorite examples was when a developer in the Internet Information Services (IIS) 6 team offered a plaster mold of his pinky finger to anyone who could find a security flaw in his code. Even though many people tried they all wanted the trophy no one found anything, and the developer's finger is safe to this day. Now think about what he did for a moment; do you think he cared about losing his trophy? No, he did not; all he wanted was as many knowledgeable eyes as possible to review his code for security defects. He did this because he doesn't want to be the guy who wrote the code that led to a well-publicized security defect. I call it clever!

Ongoing Training

It's unfortunate, but true, that each week we see new security threats or threat variations that could make seemingly secure products vulnerable to attack. Because of this, you must plan ongoing training for your development teams. For example, our group offers monthly training to make people aware of the latest security issues and the reasons for these issues and to teach how to mitigate the threats. We also invite guest speakers to discuss lessons learned in their area of security and to offer product expertise.

Advancing the Science of Security

It turns out security education has an interesting side effect. Once you communicate security knowledge to a number of domain experts for example, in the case of Windows, we have people who specialize in file systems, globalization, HTTP, XML, and much more they begin thinking about how their feature set can be used by malicious users. Figure 2-2 illustrates this concept.

Figure 2-2. The mind-set change that occurs when you teach security skills to formerly nonsecurity people.

This shift in perspective gave rise to a slogan, One person's feature is another's exploit, as domain experts used their skill and knowledge to come up with security threats in features that were once considered benign.

TIP
If you do not have security skills in-house, hire a security consulting company that offers quality, real-world training courses to upskill your employees.

IMPORTANT
There are two aspects to security training. The first is to teach people about security issues so that they can look over their current product and find and fix security bugs. However, the ultimate and by far the most important goal of security education is to teach people not to introduce security flaws into the product in the first place!

Education Proves the More Eyes Fallacy

I often hear that more eyes reviewing code equals more security flaws found and therefore more secure code. This is untrue. The people reviewing the code need to know and understand what security vulnerabilities look like before they can determine whether the code is flawed. Here's an analogy. While this book was being written, a number of accounting scandals came to light. In short, there's evidence that a number of companies used some imaginative accounting practices. So, now imagine if a company's CEO is called before the United States Congress to answer questions about the company's accounting policies and the conversation goes like this:

• Congressional representative: We believe your company tampered with the accounts.

• CEO: We did not.

• Congressional representative: How can you prove it?

• CEO: We had 10,000 people review our accounts, and no one found a single flaw.

• Congressional representative: But what are the credentials of the reviewers? What is their accounting background, and are they familiar with your business?

• CEO: Who cares? We had 10,000 people review the books that's 20,000 eyes!

It does not matter how many people review code or specifications for security flaws, not unless they understand and have experience building secure systems and understand common security mistakes. People must learn before they can perform a truly appropriate review. And once you teach intelligent people about security vulnerabilities and how to think like an attacker, it's amazing what they can achieve.

Now the Evidence!

In 2001, I performed a simple experiment with two friends to test my theories about security education. Both people were technical people, with solid programming backgrounds. I asked each of them to review 1000 lines of real public domain C code I found on the Internet for security flaws. The first developer found 10 flaws, and the second found 16. I then gave them an intense one-hour presentation about coding mistakes that lead to security vulnerabilities and how to question assumptions about the data coming into the code. Then I asked them to review the code again. I know this sounds incredible, but the first person found another 45 flaws, and the second person found 41. Incidentally, I had spotted only 54 flaws in the code. So the first person, who found a total of 55 flaws, had found one new flaw, and the second person, with 57 total flaws, had found the same new flaw as the first person plus two others!

If it seems obvious that teaching people to recognize security flaws means that they will find more flaws, why do people continue to believe that untrained eyes and brains can produce more secure software?

IMPORTANT
A handful of knowledgeable people is more effective than an army of fools.

An interesting side effect of raising the security awareness within a development organization is that developers now know where to go if they need help rather than plod along making the same mistakes. This is evident in the huge volume of questions asked on internal security newsgroups and e-mail distribution lists at Microsoft. People are asking questions about things they may not have normally asked about, because their awareness is greater. Also, there is a critical mass of people who truly understand what it takes to design, build, test, and document secure systems, and they continue to have a positive influence on those around them. This has the effect of reducing the chance that new security defects will be entered into the code.

People need security training! Security is no longer a skill attained only by elite developers; it must be part of everyone's daily skill set.