Chapter 11. Knowledge for Software Security

Chapter 11. Knowledge for Software Security^[1]

^[1] Parts of this chapter appeared in original form in IEEE Security & Privacy magazine as two articles, one coauthored with Sean Barnum [Barnum and McGraw 2005] and one with Nancy Mead [Mead and McGraw 2005].

Knowledge is power.

Francis Bacon

Knowledge management can play a central role in encapsulating and spreading the emerging discipline of software security more efficiently. This chapter is about the kinds of security knowledge that can be used to provide a solid foundation for software security practices.

Knowledge is more than simply a list of things we know or a collection of facts. Simply put, information and knowledge aren't the same thing, and it is important to understand the difference. Knowledge is information in contextinformation put to work using processes and procedures. A checklist of potential security bugs in C and C++ is information; the same information built into a static analysis tool is knowledge.

At this nascent stage of the game in software security, a number of early adopters have created various checklists for use when thinking about software security and application security. One of the problems with these lists is that they have a tendency to combine categories of information in hard-to-grok ways. For example, a "Top Ten Things to Know about Application Security" document that treats "Apply the Principle of Least Privilege" the same as "Avoid Buffer Overflows," "Monitor BugTraq," and "Use a Code Scanning Tool" combines lots of good ideas in an incoherent package. It is better to organize software security knowledge into coherent chunks.

The first hurdle along these lines to overcome is the propensity to think of software security as a coding issue. I like to refer to this kind of approach as the "bug parade." Sure, there are hundreds of bugs that can lead to security problems (especially in languages like C and C++). But simply developing a checklist of coding issues to avoid in C and having your developers read it will not solve the software security problem.^[2] If instead of making a static list, we build a database of coding issues and create a tool to help us uncover these problems, then we're getting somewhere. This is precisely what is happening with the static analysis space.

^[2] If you want to experience firsthand why reading rules is tedious, check out Appendix B.

Of course, by now we know that we must address bugs (of the sort that a tool can easily find) and flaws (which require a smart human to find). We ignore flaws by declaring them "too hard to deal with at this time" at our peril.

The second hurdle is the incorrect belief that software security is really about adopting various security features and/or conventions. One place where this is going particularly wrong is in the creation of generic classes for filtering input. We all know by now that a black-list solution to the input-filtering problem (trying to identify all possible malicious input) is inferior to a white-list solution (ensuring that only input that plays by certain rules is allowed). The problem is that black lists are potentially infinite every timethere is no way to anticipate future malicious input. Consider for a moment the encoding problem and various Unicode attacks discussed in Exploiting Software and you'll see what I mean [Hoglund and McGraw 2004].

Given a thorough understanding of a program (say, when you're building it), you are in a perfect position to create a correct input-filtering approach since you know precisely what kind of input you are expecting. Wrongheaded thinking has led to the idea of "security classes" that you can buy and link into your code. In the case of generic filtering capability, this idea is unlikely to work. Of course, there is nothing wrong with adopting great coding practices and even borrowing solid code to use. In any event, as this book demonstrates, software security is more about assurance than it is about features. Some people call the feature-based approach to software security a "cookbook" approach. Cookbooks can certainly help you with recipes, but just reading cookbooks without ever turning on your stove and actually tasting stuff won't make you a good cook. Experience is the most powerful teacher.

The third and final major hurdle is overuse of the checklist. Checklists are great in the hands of an expert. They serve as reminders of things to think about. However, checklists are by their very definition incomplete. Consider the STRIDE model from Writing Secure Code [Howard and LeBlanc 2003]. The activity of thinking carefully about Spoofing, Tampering, Repudiation, Information disclosure, Denial of service, and Elevation of privilege while you ponder system security is a great idea. The problem is that there are definitely more than six categories of attack. If you limit your thinking to a checklist, you will likely overlook interesting risks that lead to new attacks. Attackers know this well, and they will go out of their way to game this problem. For example, no virus writer worth his salt will release a new virus without first running every available commercial antivirus checker against it as an acceptance test. (Not to imply that there aren't plenty of really dumb virus writers out there.)

The way around these hurdles is to organize and apply software security knowledge with care.

This chapter may be too academic or research oriented for some. Software security practitioners and software security scientists will certainly want to develop the catalogs we cite (or participate in group exercises to develop a common set of open catalogs for all). But large organizations worried about adopting software security programs (as described in Chapter 10) will be better served with the information architecture covered there. This chapter is more about the intellectual exercise of organizing and cataloging knowledge than it is about making that knowledge actionable in an enterprise.