4.3 Content Validity

The act of judging the adequacy of content for a potential measure is called content validation. The notion of content validity is particularly critical in the evaluation of people. The typical computer science student will perform very well on the mathematical and reasoning sections of the typical standardized tests used in universities (e.g., the SAT or the GRE). These same students do not do very well on the verbal sections of these same standardized tests. Typically, the verbal scores are not good predictors of a student's success as an undergraduate or graduate student, yet they are part of the composite score of the student's test. In this case, the verbal sections of these tests do not have good content validity, in that for most purposes, it is quite OK for programmers and computer science students to be functionally illiterate.

Return, for a moment, to the construction of a test of programming ability. Content validity will be very difficult to achieve in a general instrument of programming ability. The reason for this is quite simple. There are many different types of programming languages and environments. A programmer who really likes to code in C++ or Java would probably not like to code in COBOL. A programmer who likes to write compilers would probably die if asked to work on an accounts receivable package. The act of writing code to perform complex numerical analysis requires deep and abiding knowledge of the underlying mathematics. There is no universal programming task. There can be no universal test of programming aptitude. No matter what his score on a test of aptitude, if we ask a programmer to do a job that he does not understand in an unfamiliar language environment, this person will not do well at his new task.