6.3. Quality Criteria Authentication mechanisms often have deficiencies in one or more areas. To support meaningful comparison of authentication mechanisms , we propose a set of evaluation criteria that can be used to assign relative deficiency values to each authentication mechanism. Once these values have been assigned, the characteristics of the environment within which the authentication mechanism will be used are identified and used to group the criteria into four different groupscritical, vital, significant, and incidentalindicating their relative importance to the developer. The assigned deficiency values in each group can then be used to support an informed decision as to which authentication mechanism to use in a particular situation. Three fundamental authentication mechanism deficiency categories can be isolated from the previous discussion: accessibility , memorability, and security. A cost category will also be included, because a cost-benefit analysis is part of any decision process. Each of these categories has several dimensions, which are discussed in the following sections. In our discussion, we describe how these criteria can be applied generally to several types of authentication mechanisms. The deficiency values for specific instances of each of these mechanisms may be somewhat different. However, after reading this section, you should be able to use these criteria to evaluate any authentication mechanism yourself. 6.3.1. Accessibility Figure 6-8 depicts the various aspects of this dimension, which reflects how easy it is for users to use a particular authentication mechanism. The aspects are described in the list that follows. Figure 6-8. Accessibility 
Special hardware and software requirements This aspect refers to the minimum hardware, software, or technical expertise required to support the authentication mechanism. If only one of these is required at the user's machine, only a minor deficit can be applied; if two are required, it is obviously more serious; and if all are required, it can be considered to be a major deficit. Many authentication systems can have special requirements. Most obvious are biometrics such as fingerprint- or iris-based identification, which naturally requires a fingerprint or iris scanner. On the other hand, biometrics systems such as voice recognition or identification by keystroke dynamics can often be performed using standard PC hardware. And biometrics or smart card readers may be built into the terminal used by the target populationfor example, there are mobile phones with integrated fingerprint or smart card readersand thus the special requirements may not pose any problems at all. Even a token-based authentication system that displays a code may require that the server be equipped with special software or hardware to verify the tokens. Smart cards further require that the user have access to a smart card reader, and USB tokens require that the user have a computer that has an available USB connector. Special requirements must always be carefully considered for authentication mechanisms that are likely to be accessed from small, handheld devices .
Convenience There are three aspects of convenience to be considered: enrollment time, authentication time, and key replacement time. Authentication time is the most relevant (it will mount up), so a large deficiency can be applied if this is time consuming. A smaller deficit can be applied if the mechanism is time consuming only at either enrollment or replacement. A large deficit results if all stages are time consuming. Only random passwords are fast and convenient both at enrollment and at authentication , with replacement being potentially time consuming depending on how it is handled. Graphical passwords, both recognition-based and position-based, are considerably more time consuming at all stages, but less so than cultural passwords where users have to answer a succession of questions. The most time consuming are biometrics, where the user has to potentially spend a substantial amount of time enrolling and being authenticated (depending on the biometrics mechanism and the level of control applied).
Inclusivity This aspect addresses the issue of the exclusion of users. Three kinds of disability are considered herecognitive, physical, and sensoryand the deficit should be assigned based on whether users in the disability categories are excluded. The deficit gets larger as more categories of disabled users are excluded. If users in all categories are excluded, that constitutes a maximal deficit.
Cultural passwords do not exclude users with any type of disabilityassuming that an accommodation has already been made so that the user can enter a responsewhereas random passwords affect users with cognitive disabilities, such as users with memory difficulties. Recognition-based graphical authentication systems affect the same disabilities, but also exclude people affected with cognitive disabilities such as dyspraxia. Biometrics possibly affects users with physical disabilities, such as amputees in the case of fingerprint devices. Physiological changes, such as those that occur in retinas during pregnancy, may affect users' use of retinal screening devices. Position-based devices may exclude users with both sensory and physical disabilitiesor even people who are simply too short or too tall, depending on how the biometrics system is mounted. In general, this deficit depends on the actual type of authentication strategy being used and should be tailored accordingly. 6.3.2. Memorability Figure 6-9 depicts the various aspects of this dimension, which reflects the importance of the memorability of authentication mechanisms. Most authentication mechanisms are knowledge-based, so this dimension is especially important. Biometrics mechanisms have minimal deficits with respect to any of the memorability criteria because the only burden they place on the user is that the user must remember how to use the biometrics device. Other mechanisms are essentially knowledge-based so that they impact on the users' memory load to a lesser or greater extent. The memorability aspects are:
Retrieval strategy Users find it easier to recognize than to recall. Hence, a system that requires only recognition or a system that does not require the user to remember at all has no deficit. A mechanism that relies on recall has a maximal deficit in terms of retrieval, and a mechanism that provides cues has a smaller deficit. Figure 6-9. Memorability 
The retrieval strategy used by recognition- and position-based graphical passwords is recognition, so there is no deficit in the retrieval strategy criterion. Cultural passwords provide a cued recall situation that is better than the uncued recall required by random passwords, which is assigned a maximal deficit.
Meaningfulness Humans remember things best if they are deducible and very well if they are meaningful. Hence, if the authenticator is self-assigned and deducible by means of a special scheme, no deficit is assigned. If it is self-assigned and meaningful to the user, there is only a small deficit. If it is self-assigned but not necessarily meaningful or deducible, the deficit is significantly larger. If it is assigned arbitrarily by the system, it has a maximal deficit in terms of meaningfulness. Cultural passwords are very meaningful, so they have no deficit for meaningfulness. Random passwords are usually meaningful to the user, and a position chosen in a position-based graphical password is also usually meaningful. Graphical passwords, if assigned by the system, may not be meaningful at all. If a recognition-based graphical password is chosen by the user, it will have the same deficit as a position-based graphical password, if the images are chosen by the system. If the images are provided by the user, they become very meaningful and no deficit is assigned here.
Depth of processing Humans remember things better if, at the encoding stage, there is some cognitive activity associated with the process.[24] The cognitive activity involved in the encoding of an authenticator based on something the user knows will determine how well the user can retrieve the authenticator later. An authentication system that does not require effort to remember (such as biometrics) has no deficit. The deficit increases as less and less cognitive activity is involved at enrollment time. A maximal deficit can be assigned if only cursory and shallow cognitive activity, such as rehearsal, is involved at enrollment. [24] V. H. Gregg, Introduction to Human Memory (London: Routledge & Kegan Paul, 1988).
Cultural passwords link the "password" to things the user already knows so that at enrollment, there is a substantial processing of information. Position-based passwords require users to choose a particular position, which also requires some cognitive activity. Graphical passwords, using images, allow dual encoding of the "password," which will encode better than a simple random password (which generally requires very little cognitive activity at enrollment).
6.3.3. Security Figure 6-10 depicts the various aspects of the security dimension, which are related and interdependent but do need to be considered separately because of their different characteristics. Figure 6-10. Security 
Note that the predictability of a key is affected by the abundance of the key for knowledge-based keys, but not for biometrics keys, which are neither abundant nor predictable. Abundance also affects breakability for knowledge-based keys, but once again not for biometrics keys. They are described in the following list.
Predictability Predictability of an authentication key is a big issue: the plethora of password choice recommendations on the Web is a bleak testimony to the tendency of people to choose weak authentication keys. There is no deficit if the authentication key is completely unpredictable, as is the case for a public encryption key. A varying deficit can be assigned depending on how many people find the key predictable. Random and position-based graphical passwords are very predictable and can be assigned a maximal deficit value. Biometrics are unpredictable, and so are system-assigned recognition-based graphical passwords. If a graphical password is chosen by the user, it becomes as predictable as random passwords. Cultural passwords are assigned a deficit halfway between maximum and minimum because although they can tap into a user's childhood memories, which hold facts generally known to few others, they can be uncovered by means of a research-based attack.
Abundance The user should be able to either choose from, or be assigned, one of a wide number of possible authenticators. Abundance has two aspectsone is the number of keys that are available for usage if the key needs to be replaced, and the other is the number commonly used in practice. So, for example, there is potentially an extremely large number of passwords if one includes all possible combinations of letters, both upper- and lowercase, and digits and special characters. In practice, however, people use very few of the available passwords because they usually restrict themselves to recognized words, reducing the number of passwords in practice to a number close to 106 (for English words, and fewer for other languages). Graphical passwords have a potentially infinite domain from which to draw imagesbut only if they are assigned by the system. Cultural passwords can also use an unlimited variety of questions if used correctly, but the keys cannot be replaced if they become knownother questions have to be formulated. Random passwords also theoretically have a limitless supply of possibilities, but the literature reports that in practice, users tend to choose passwords from a relatively small subset of words, so they too have a large deficit in the area of abundance. Position-based passwords do not offer users a large enough range of choice[25] and are also deficient in this respect. [25] Ibid.
Biometrics is not abundanthumans have only two retinas, ten fingerprints, two hand geometries, etc.so as a result, biometrics mechanisms have a maximal deficit. If the biometrics device is used in a controlled environment that is not susceptible to replay attacks, abundance will not be an issue because it is unlikely that the biometrics will ever have to be replaced, but in an uncontrolled environment, this lack of abundance may be serious.
Disclosure An authenticator should not be disclosed to another user; otherwise, authentication fails. Hence, the mechanism is clearly deficient if a user can easily record his authenticator, and if it can be purposely disclosed, observed by, or stolen by another person. There is no deficit if is impossible for the user to do this, and a varying deficit can be applied between these extremes to denote the ease with which disclosure can occur. Cultural passwords and both types of graphical passwords can be observed at key entry time, so there is a deficit in this area, but it is not the maximal deficit, which is assigned to random passwords based on the literature reporting widespread recording of passwords in insecure locations. Biometrics need to be disclosed in order to be verified, so they have a maximal deficit in this respect. Private keys do not need to be disclosed in order to be used, so they have no disclosure deficit. Token-based systems that do not use public key technology are typically based on a shared secret that does not need to be disclosed for the user to be authenticated, but the shared secret does need to reside on the system verifying the token. Thus, they are subject to disclosure.
Confidentiality Authentication requires the user and the system to exchange a pre-agreed key. If the user has to supply the full key at authentication, it is possible for a transmission sniffer to observe the key, or for some other person to observe it and reuse it. If there is another way for the user to demonstrate knowledge of the key without revealing it, that makes the authentication mechanism less vulnerable. Hence, an authenticator that relies on the full authentication key being revealed is assigned a maximal deficiency value. If the user doesn't have to reveal the key at all, or if the revealed key cannot be reused, there is no deficit. Values in between can be assigned depending on how much of the key has to be revealed. Random passwords generally reveal the entire key at authentication, which assigns a maximal deficit to confidentiality . Cultural passwords may capture the answers to many questions at enrollment time, and then only use a few at authentication time, so that the deficit is not as large. If a password mechanism can be tailored to only ask for particular letters or digits, the confidentiality deficiency decreases. Authentication systems that involve responding to a random challenge with a response that is signed with a private key may not have any confidentiality deficit.
Privacy An authenticator may record many details about the user to support authentication or key replacement, which, if not stored securely, could compromise other systems for which the person is required to use the same details, because a key reused is a key weakened.[26] It can also violate the person's privacy.[27] If an authenticator requires the user to reveal personal details, a maximal deficiency value is assigned. No deficiency is assigned if no personal details are required. An approach that allows users to decide which personal details to reveal will earn a deficiency value depending on the type and number of details that have to be revealed. [26] B. Ives, K. R. Walsh, and H. Shneider, "The Domino Effect of Password Reuse," Communications of the ACM 47 (2004), 7578.
[27] H. Berghel, "Identity Theft, Social Security Numbers and the Web," Communications of the ACM 43 (2000), 1721.
Assigned random and graphical passwords generally do not reveal any personal details, so they have no deficit in the privacy area. This changes if users are permitted to specify their own "random" passwords or provide their own images for graphical-based authentication. User-specified passwords may also compromise the security of other systems, if the same password is used in multiple locations. Cultural passwords and biometrics both reveal very personal details, so they have a maximal deficit. Because of a century-long association with law enforcement, many users feel that fingerprints represent private or confidential information. As a result, these users may feel that the use of biometrics systems based on fingerprints constitutes an inherent privacy violation. Although users do not seem to have similar associations with iris biometrics at this time, such associations could emerge in the future.
Breakability and crackability The values will be assigned depending on the time an attacker would have to spend to attack the authenticator. The higher the price an attacker must pay in terms of time and effort, the less vulnerable an authentication mechanism is. A research-based attack is inevitably time consuming for anyone other than close friends and family. Thus, if the authentication mechanism is vulnerable to a research-based attack, a relatively small deficiency value can be assigned. If the authentication key is vulnerable to dictionary or brute force attacks, which are more common, a larger deficit can be assigned. If the authenticator is vulnerable to a key logger,[28] a maximal deficit can be assigned. Many users do not have virus software or firewalls running on their home computers, and this cracking mechanism is particularly cheap from an attacker's point of view. [28] A key logger is a program that can be surreptitiously loaded onto your computer to record all of your key presses on the keyboard. In this way, many passwords can be obtained and reported back to the attacker.
Cultural passwords are vulnerable to research-based attacks , which are very time consuming, so they have a small deficit for breakability and crackability. Biometrics can be subject to replay[29] and reverse engineering[30] attacks, depending on how the biometrics is implemented. The entropy or information content of the biometrics also needs to be considered: some biometrics, such as handprint geometry, do not have enough "randomness" that they can successfully distinguish between all members of a nation-size population. Other biometrics, such as iris recognition, have so much entropy and variance that it is unlikely that two people will ever be found who have matching biometrics values. [29] A replay attack records the authentication key as the user provides it or as it is transmitted; then, when an attacker wants to get into the site, he simply provides the system with the previously recorded key.
[30] Reverse engineering is a technique that obtains the stored description of a person's biometrics, such as his voiceprint or fingerprint description. The attacker uses software to derive a description of the voice or fingerprint that can be entered into the biometrics device as an authentication key.
Recognition-based and password-based graphical passwords are both subject to brute force attacks, and random passwords are subject to dictionary attacks, so they have the maximal deficit because such attacks can be largely automated.
6.3.4. Cost No set of criteria can be complete without considering cost , in terms of both time and money. Cost has various dimensions: Random passwords appear to be a relatively cheap option, until you consider the salaries paid to help-desk teams to replace forgotten passwords in an online banking environment and the risk that these individuals might fall prey to a social engineering attack. It may be worth spending a bit more on software and having a time-consuming enrollment to reduce the replacement and maintenance costs of the system by automating the key replacement stage. In order to apply a deficit to cost, all the different cost factors should be considered and a cumulative cost assessment performed to come up with a deficit value. Biometrics has the greatest costs because you need extra hardware and software, and you have to assist users being enrolled and authenticated. Graphical passwords need special software, but enrollment, authentication, and replacement can be automated. Random passwords need no special hardware or software and have almost no cost at enrollment and authentication, but secure replacement can involve humans and can be expensive. Cultural passwords are time consuming at enrollment, but this helps make replacement a rare event, and even if it is required, it can be easily automated. |
