Intentions, Methods, and Outcomes: Three Areas Worthy of Inquiry

Methods of Persuasion

Examining the methods an interactive technology uses to persuade is another means of establishing intent and assessing ethics. Some methods are clearly unethical, with the most questionable strategies falling outside a strict definition of persuasion. These strategies include making threats, providing skewed information, and backing people into a corner. In contrast, other influence strategies, such as highlighting cause-and-effect relationships, can be ethically sound if they are factual and empower individuals to make good decisions for themselves .

How can you determine if a computer’s influence methods are ethical? The first step is to take technology out of the picture to get a clearer view. Simply ask yourself, “If a human were using this strategy to persuade me, would it be ethical?”

Recall CodeWarriorU.com, a Web site discussed in Chapter 1. While the goals of the online learning site include customer acquisition and retention, the influence methods include offering testimonials, repeatedly asking potential students to sign up, putting students on a schedule for completing their work in each course, and tracking student progress. Most people would agree that these methods would be acceptable ways to influence if they were used by a person. So when it comes to this first step of examining ethical methods of influence by interactive technology, CodeWarriorU.com earns a passing grade.

Now consider another example: a Web banner ad promises information, but after clicking on it you are swept away to someplace completely unexpected. A similar bait-and-switch tactic in the brick-and-mortar world would be misleading and unethical. The cyber version, too, is unethical. (Not only is the approach unethical, it’s also likely to backfire as Web surfers become more familiar with the trickery .) ^[16]

Using Emotions to Persuade

Making the technology disappear is a good first step in examining the ethics of persuasion strategies. However, it doesn’t reveal one ethical gray area that is unique to human-computer interactions: the expression of emotions.

Because humans respond so readily to emotions, it’s likely that computers that express “emotions” can influence people. When a computer expresses sentiments such as “You’re my best friend,” or “I’m happy to see you,” it is posturing to have human emotions. Both of these statements are uttered by ActiMates Barney, the interactive plush toy by Microsoft that I described in Chapter 5.

The ethical nature of Barney has been the subject of debate. ^[17] When I monitored a panel discussion of the ethics of the product, I found that panelists were divided into two camps. Some viewed the product as ethically questionable because it lies to kids, saying things that imply emotions and motives, and presenting statements that are not true or accurate, such as “I’m happy to see you.” Others argued that kids know it is only a toy without emotions or motives, part of a fantasy that kids understand.

The social dynamics leveraged by ActiMates characters can make for engaging play, which is probably harmless and may be helpful in teaching children social rules and behaviors. ^[18] But social dynamics could be used in interactive toys to influence in a negative or exploitative way what children think and do, and this raises ethical questions.

My own view is that the use of emotions in persuasive technology is unethical or ethically questionable only when its intent is to exploit users or when it preys on people’s naturally strong reactions to negative emotions or threatening information expressed by others. ^[19] For instance, if you play at theWeb site TreeLoot.com, discussed earlier in this chapter, you might encounter a character who says he is angry with you for not visiting the site’s sponsors (Figure 9.3).

Figure 9.3: A TreeLoot.com character expresses negative emotions to motivate users.

Because the TreeLoot site is so simple and the ruse is so apparent, you may think this use of emotion is hardly cause for concern. And it’s probably not. But what if the TreeLoot system were much more sophisticated, to the point where users couldn’t tell if the message came from a human or a computer, as in the case of a sophisticated chat bot? Or what if the users believed the computer system that expressed anger had the power to punish them? The ethics of that approach would be more questionable.

The point is that the use of emotions to persuade has unique ethical implications when computers rather than humans are expressing emotions. In addition to the potential ethical problems with products such as ActiMates Barney and TreeLoot.com, there is the problem discussed earlier in this chapter: while computers may convey emotions, they cannot react to emotions, giving them an unfair advantage in persuasion.

Methods That Always Are Unethical

Whether used by a person or a computer system, some methods for changing attitudes and behaviors are almost always unethical. Although they do not fall into the category of persuasion per se, two methods deserve mention here because they are easy to incorporate into computing products: deception and coercion.

Web ads are perhaps the most common example of computer-based deception. Some banner ads (Figure 9.4) seem to do whatever it takes to get you to click on them. They may offer money, sound false alarms about computer problems, or, as noted earlier, promise information that never gets delivered. The unethical nature of these ads is clear. If the Web were not so new, it’s unlikely we’d tolerate these deceptive methods. ^[20]

Figure 9.4: This banner ad claims it’s checking qualifications—a deception (when you click on the ad, you are simply sent to a gambling site).

Besides deception, computers can use coercion to change people’s behaviors. Software installation programs provide one example. Some installation programs require you to install additional software you may not need but that is bundled as part of the overall product. In other situations, the new software may change your default settings to preferences that benefit the manufacturer rather than the user , affecting how you work in the future (some media players are set up to do this when installed). In many cases, users may feel they are at the mercy of the installation program. This raises ethical questions because the computer product may be intentionally designed to limit user choice for the benefit of the manufacturer.

Methods That Raise Red Flags

While it’s clear that deception and coercion are unethical in technology products, two behavior change strategies that fit into a broad definition of persuasion— operant conditioning and surveillance—are not as clearly ethical or unethical, depending on how the strategies are applied.

Operant Conditioning

Operant conditioning, described in Chapter 3, consists mainly of using reinforcement or punishment to promote certain behavior. Although few technology products outside of games have used operant conditioning to any great extent, one could imagine a future where operant conditioning is commonly used to change people’s behavior, sometimes without their direct consent or without them realizing what’s going on—and here is where the ethical concerns arise.

For instance, a company could create a Web browser that uses operant conditioning to change people’s Web surfing behavior without their awareness. If the browser were programmed to give faster page downloads to certain Web sites—say, those affiliated with the company’s strategic partners —and delay the download of other sites, users would be subtly rewarded for accessing certain sites and punished for visiting others. In my view, this strategy would be unethical.

Less commonly, operant conditioning uses punishment to reduce the instances of a behavior. As I noted in Chapter 3, I believe this approach is generally fraught with ethical problems and is not an appropriate use of conditioning technology.

Having said that, operant conditioning that incorporates punishment could be ethical, if the user is informed and the punishment is innocuous. For instance, after a trial period, some downloaded software is designed to take progressively longer to launch. If users do not register the software, they are informed that they will have to wait longer and longer for the program to become functional. This innocuous form of punishment (or negative reinforcement, depending on your perspective) is ethical, as long as the user is informed. Another form of innocuous and ethical punishment: shareware programs that bring up screens, often called “nag screens,” to remind users they should register and pay for the product.

Now, suppose a system were created with a stronger form of punishment for failure to register: crashing the computer on the subsequent startup, locking up frequently used documents and holding them for ransom, sending email to the person’s contact list pointing out that they are using software they have not paid for. Such technology clearly would be unethical.

In general, operant conditioning can be an ethical strategy when incorporated into a persuasive technology if it is overt and harmless. If it violates either of those constraints, however, it must be considered unethical.

Another area of concern is when technologies use punishment—or threats of punishment—to shape behaviors. Technically speaking, punishment is a negative consequence that leads people to perform a behavior less often. A typical example is spanking a child. Punishment is an effective way to change outward behaviors in the short term , ^[21] but punishment has limited outcomes beyond changing observable behavior.

Surveillance

Surveillance is another method of persuasion that can raise a red flag. Think back to Hygiene Guard, the surveillance system to monitor employees ’ hand washing, described in Chapter 3. Is this an ethical system? Is it unethical? Both sides could be argued. At first glance, Hygiene Guard may seem intrusive , a violation of personal privacy. But its purpose is a positive one: to protect public health. Many institutions that install Hygiene Guard belong to the healthcare and food service industries. They use the system to protect their patients and patrons.

So is Hygiene Guard ethical or unethical? In my view, it depends on how it is used. As the system monitors users, it could give gentle reminders if they try to leave the restroom without washing their hands. Or it could be set up mainly to identify infractions and punish people. I view the former use of the technology as ethical and the latter application as unethical.

The Hygiene Guard example brings up an important point about the ethics of surveillance technology in general: it makes a huge difference how a system works—the nature and tone of the human-machine interaction. In general, if surveillance is intended to be supportive or helpful rather than punitive, it may be ethical. However, if it is intended mainly to punish, I believe it is unethical.

Whether or not a surveillance technology is ethical also depends on the context in which it is applied. Think back to AutoWatch, the system described in Chapter 3 that enables parents to track how their teenagers are driving. ^[22] This surveillance may be a “no confidence” vote in a teenager, but it’s not unethical, since parents are ultimately responsible for their teens’ driving, and the product helps them to fulfill this responsibility.

The same could be said for employers that implement such a system in their company cars. They have the responsibility ( financially and legally, if not morally) to see that their employees drive safely while on company time. I believe this is an acceptable use of the technology (although it is not one that I endorse). However, if the company were to install a system to monitor employees’ driving or other activities while they were not on company time, this would be an invasion of privacy and clearly an unethical use of technology.

Figure 9.5: The ethical nature of a persuasive technology can hinge on whether or not the outcome was intended.

Outcomes: Intended and Unintended

In addition to examining intentions and methods, you can also investigate the outcomes of persuasive technology systems to assess the ethics of a given system, as shown in Figure 9.5. (This line of thinking originated with two of my former students: Eric Neuenschwander and Daniel Berdichevsky.)

If the intended outcome of a persuasive technology is benign , generally there is no significant ethical concern. Many technologies designed for selling legitimate products and strengthening brand loyalty fall into this category.

The intended outcomes of other technologies may raise ethical concerns. Think back to Banana-Rama, the high-tech slot machine described in Chapter 5. This device uses onscreen characters, an ape and a monkey , to motivate players to continue gambling. When you win, the characters celebrate. When you hesitate to drop in more of your money to continue playing, the characters’ expressions change from supportive to impatient.

Some people would find this product ethically objectionable because its intended outcome is to increase gambling, an activity that conflicts with the values of some individuals and cultures. Other people would not consider this intended outcome a cause for ethical alarm; gambling is an accepted part of many cultures and is often promoted by government groups. However, if Banana-Rama were wildly popular, with Las Vegas tourists lining up to lose their fortunes, the outcome may be significant enough to make it a major ethical issue.

Hewlett-Packard’s MOPy (Multiple Original Printouts) is a digital pet screen saver that rewards users for printing on an HP printer (Figure 9.6). The point of the MOPy system is to motivate people to print out multiple originals rather than using a copy machine. As you make original prints, you earn points that can be redeemed for virtual plants and virtual toys for your virtual fish. In this way, people use up HP ink cartridges and will have to buy more sooner.

Figure 9.6: The MOPy screen saver (no longer promoted by Hewlett-Packard) motivates people to make original prints, consuming disposable ink cartridges.

Some might argue that MOPy is unethical because its intended outcome is one that results in higher printing costs and environmental degradation. (To HP’s credit, the company no longer promotes MOPy.) ^[23] Others could argue that there is no cause for ethical alarm because the personal or environmental impact of using the product is insignificant.

But suppose that Banana-Rama and MOPy were highly successful in achieving their intended outcomes: increasing gambling and the consumption of ink cartridges. If these products produced significant negative impacts—social, personal, and environmental—where would the ethical fault reside? Who should shoulder the blame?

In my view, three parties could be at fault when the outcome of a persuasive technology is ethically unsound: those who create, distribute, or use the product. I believe the balance of culpability shifts on a case-by-case basis. ^[24] The creators have responsibility because, in the case of MOPy, their work benefited a private company at the expense of individuals and the global environment. Likewise, distributors must also shoulder the ethical responsibility of making unethical technologies widely available.

Finally, users of ethically questionable persuasive technologies must bear at least some responsibility. In the cases of Banana-Rama and MOPy, despite the persuasive strategies in these products, individual users are typically voluntarily choosing to use the products, thus contributing to the outcomes that may be ethically questionable.

Responsibility for Unintended Outcomes

Persuasive technologies can produce unintended outcomes. Although captology focuses on intended outcomes, creators of persuasive technology must take responsibility for unintended unethical outcomes that can reasonably be foreseen.

To act ethically, the creators should carefully anticipate how their product might be used for an unplanned persuasive end, how it might be overused , or how it might be adopted by unintended users. Even if the unintended outcomes are not readily predictable, once the creators become aware of harmful outcomes, they should take action to mitigate them.

Designed to reduce speeding, the Speed Monitoring Awareness Radar Trailer, discussed in Chapter 3, seems to have unintended outcomes that may not have been easy to predict. Often when I discuss this technology with groups of college students, at least one male student will say that for him the SMART trailer has the opposite effect of what was intended: he speeds up to see how fast he can go.

As far as I can tell, law enforcement agencies have not addressed the possibility that people might actually speed up rather than slow down when these trailers are present. It may be the unintended outcome has not been recognized or is considered to apply to a relatively small number of people—mostly younger male drivers who seek challenges. In any case, if this unintended outcome were to result in a significant number of accidents and injuries, I believe the developers of the SMART trailer would have to take responsibility for removing or altering the system.

Some companies turn a blind eye to the unintentional, though reasonably predictable, outcomes of using their products. Consider the video game Mortal Kombat, which rewards players for virtual killing. In this game, players interact with other players through virtual hand-to-hand combat. This entertainment product can be highly compelling for some people.

Unfortunately, Mortal Kombat and other violent video games not only motivate people to keep playing, they also may have a negative effect on players’ attitudes and behaviors in the real world. Social learning theory ^[25] suggests that practicing violent acts in a virtual world can lead to performing violent acts in the real world. ^[26] The effect of video game violence has been much debated for over a decade . After reviewing results of previous studies and presenting results of their own recent work, psychologists Craig Anderson and Karen Dill conclude:

When the choice and action components of video games . . . is coupled with the games’ reinforcing properties, a strong learning experience results. In a sense, violent video games provide a complete learning environment for aggression, with simultaneous exposure to modeling, reinforcement, and rehearsal of behaviors. This combination of learning strategies has been shown to be more powerful than any of these methods used singly . ^[27]

Although violent real-world behavior is not the intended outcome of the creators of video games such as Mortal Kombat, it is a reasonably predictable outcome of rewarding people for rehearsing violence, creating an ethical responsibility for the makers , distributors, and users of such violent games.

^[16] See “Do they need a “trick” to make us click?,” a pilot study that examines a new technique used to boost click-through, by David R. Thompson, Ph.D., Columbia Daily Tribune, and Birgit Wassmuth, Ph.D., University of Missouri. Study conducted September 1998. Paper presented at the annual Association for Education in Journalism and Mass Communication Convention, August 4–7, 1999, New Orleans, Louisiana.

^[17] At the 1999 ACM SIGCHI Conference, I organized and moderated a panel discussion on the ethical issues related to high-tech children’s plush toys, including Barney. This panel included the person who led the development of the Microsoft ActiMates products (including Barney) and other specialists in children’s technology. The panelists were Allen Cypher, Stagecast Software; AllisonDruin, University of Maryland; Batya Friedman, Colby College; and Erik Strommen, Microsoft Corporation.

You can find a newspaper story of the event at http://www.postgazette.com/businessnews/19990521barney1.asp.

^[18] E. Strommen and K. Alexander, Emotional interfaces for interactive aardvarks: Designing affect into social interfaces for children, Proceeding of the CHI 99 Conference on Human Factors in Computing Systems, 528–535 (1999).

^[19] In an article reviewing various studies on self-affirmation, Claude Steele discusses his research that showed higher compliance rates from people who were insulted than from people who were flattered. In both cases, the compliance rates were high, but the people receiving the negative assessments about themselves before the request for compliance had significantly higher rates of compliance. See C. M. Steele, The psychology of self affirmation: Sustaining the integrity of the self, in L. Berkowitz (ed.), Advances in Experimental Social Psychology, 21: 261–302 (1988).

For a more recent exploration of compliance after threat, see Amy Kaplan and Joachim Krueger, Compliance after threat: Self-affirmation or self-presentation? Current Research in Social Psychology, 2:15–22 (1999). http://www.uiowa.edu/~grpproc. (This is an online journal. The article is available at http://www.uiowa.edu/~grpproc/crisp/crisp.4.7.htm.)

Also, Pamela Shoemaker makes a compelling argument that humans are naturally geared to pay more attention to negative, threatening information than positive, affirming information. See Pamela Shoemaker, Hardwired for news: Using biological and cultural evolution to explain the surveillance function, Journal of Communication, 46(2), Spring (1996).

^[20] For a statement about the “Wild West” nature of the Web in 1998, see R. Kilgore, Publishers must set rules to preserve credibility, Advertising Age, 69 (48): 31 (1998).

^[21] For book-length and readable discussions about how discipline works (or doesn’t work) with children in changing behavior, see

a. I. Hyman, The Case Against Spanking: How to Discipline Your Child without Hitting (San Francisco: Jossey-Bass Psychology Series, 1997).

b. J. Maag, Parenting without Punishment: Making Problem Behavior Work for You ( Philadelphia, PA: The Charles Press, 1996).

^[22] To read about the suggested rationale for AutoWatch, see the archived version at http:// web.archive.org/web/19990221041908/
http://www.easesim.com/autowatchparents.htm.

^[23] While Hewlett-Packard no longer supports MOPy, you can still find information online at the following sites:

http://formen.ign.com/news/16154.html

http://cna.mediacorpnews.com/technology/bytesites/virtualpet2.htm

^[24] Others suggest that all parties involved are equally at fault. For example, see K. Andersen, Persuasion Theory and Practice (Boston: Allyn and Bacon, 1971).

^[25] A. Bandura, Self-Efficacy: The Exercise of Control (New York: Freeman, 1997).

^[26] C. A. Anderson and K. E. Dill, Video games and aggressive thoughts, feelings, and behavior in the laboratory and in life, Journal of Personality and Social Psychology , 78 (4): 772–790 (2000). This study, which includes an excellent bibliography, can be found at http://www.apa.org/journals/psp/psp784772.html .