Section 23.4. Trolls, Anonymous Cowards, and Insensitive Clods

23.4. Trolls, Anonymous Cowards, and Insensitive Clods

Many discussion forums, be they mailing lists, web-based discussions, or IRC channels, produce high-quality discussions among a few participants. The challenge is to scale this "few-to-few" communication all the way to "many to many." Adding a moderator helps to a point. Many moderated discussions, though, improve the "signal-to-noise ratio" at a cost: over time, the signal takes on more and more of the moderator's particular viewpoint. Other viewpoints, intentionally or not, are stifled with a hostile reaction. Rarely can a discussion forum have enough commonality in interest to draw a critical mass of audience, have enough variety in viewpoints to keep the discussion interesting, and scale to ever-larger audience sizes without losing the signal of the discussion in the noise of the chatter.

Discussion systems can avoid the tyranny of a single moderator by letting all users vote on moderation. However, letting every user vote equally on moderation assumes that all users are equally informed, equally concerned, and equally motivated about what they are moderating. These are unrealistic assumptions that lead to a tyranny of the masses as damning as the tyranny of a single moderator.

In the spring and summer of 1998, Slashdot faced all these challenges. What emerged was a mix of software engineering and social engineering to cope with the growth of the site. Not everything that was tried worked, and some solutions are still works in progress.

User accounts were created. Registered users could customize which slashboxes (sets of links to changing external content; a precursor to today's RSS) appeared on their Slashdot page, and where. Today we would call this portalizing the site. To the Slashdot team it was a way of minimizing complaints and encouraging visitors to register and log in. Users could filter content to see only stories on certain topics, or to exclude stories on certain topics. Having a low user account number also became a point of pride with the regular visitors.

An interesting and intended side effect of user accounts was to reduce the number of off-topic or deliberately inflammatory comments. People are, by nature, less inhibited when communicating anonymously. The simple step of encouraging people to identify themselves helped restore some order to discussions.

An issue continuously debated behind the scenes was whether anonymous posting should be allowed at all. In the end, free-speech considerations have trumped all other considerations in this debate. Anonymous posting allows an employee to speak out candidly about his employer without fear of retribution. Anonymous posting allows someone to express an opinion or political view without being chastised by their peers. Ultimately, anonymous posting contributed significantly to key events that have left an indelible mark on Slashdot.

One step that also occurred was the slowly diminishing value of anonymous commentsover time, their base score became set to 0, which was lower than the 1 threshold that the non-logged-in user reads at by default. This means that most of the people only read anonymous posts that have been moderated up.

As the audience grew, so did the number of submissions. It became impractical to get every worthy submission posted to the front page; stories simply would have scrolled by too fast. In May of 1998, sections were added to Slashdot, creating a separate front page for those interested in a specific topic. Today there are 14 sections on Slashdot, ranging from Book Reviews, to Science, to Politics.

The most controversial modification to Slashdot came in October 1998, with the introduction of moderation. Each comment is classified on two dimensions: one for the type of comment, and the other for the quality of the comment. Types of comments include: Overated, Underated, Troll, Insightful, Informative, Redundant, Offtopic, and Flamebait. Quality is numeric, from -1 to 5. All comments initially started at mod level 1, but could be moderated up or down for as long as the discussion on that story was open.

One of the important settings available to registered users is the moderation level. Users can select what level of comments they see by default, and can change that setting on a story-by-story basis. Thus, if the number of comments at a certain level is too many to read through, or the relevance drops off too much, users can filter out comments below a certain level. The default moderation level also enables a form of filtering without censoring. The most irrelevant or inflammatory comments routinely get moderated down to a level of 0 or -1. An unregistered user visiting the site has his default moderation level set to 1. Since 95% of all visitors to the site never change their moderation level, the vast majority of visitors never see the lowest moderated comments. Moderation also provided an additional motivation to get users to register. After a time, the policy was changed so that an anonymous user's posts generally started with a moderation level of 0, but a regsitered user's comments started with a moderation level of 1.

Many discussion forums have tried various forms of moderation. The challenge is to find a system that is scalableSlashdot routinely generates tens of thousands of comments per dayand heterogeneous, namely reflecting more than a single point of view about how comments should be moderated.

Slashdot met these challenges by borrowing from the principles of collaboration in its open source roots. The audience is essentially self-moderating, and indeed the more people participate in the system, the better the moderation gets. This is the enormous differentiator for Slashdot. Where most discussion forums crumble under a deteriorating signal-to-noise ratio as their size increases, Slashdot actually benefits from network effects: more is better.

The key is that Slashdot tracks a wide range of information about its users: how many comments a user has posted, how many stories a user has submitted, how many submissions have been accepted, what moderation level a user's comments tend to settle on, what type of comments a user typically makes. All of this data is combined to produce a number that roughly quantifies the value of a user to the site. This number is referred to in the Slashdot system as karma.

Users with high karma are periodically selected to moderate. The system is automated, requiring little intervention from the Slashdot staff. Once selected, a user has his moderation authority turned on for a period of time, enabling him to moderate up or down, or classify comments he reads. After a period of time, moderation is turned off, and passes to another user. At any given time there are roughly 1,850 users moderating comments.^[1]

^[1] If one thinks of the task of a moderator as similar to that of a copyeditor, it is possible to put an approximate monetary value on the work performed by users while moderating. If the typical moderator spends even an hour a day on moderationand many spend much longerthis amounts to roughly $50,000 worth of work being done for the site for free each day. The key, though, is to see it in terms of value provided rather than money saved. Thanks to Slashdot's tiered moderation system, the site continues to scale and discussion continues to be valuable. There simple isn't anything one could spend $50,000 a day on to provide comparable value.

Originally, a user's karma number was viewable. This policy led to problems. Users viewed their karma rating as a score, and raising their karma as a game. Once people tried to deliberately game the system, the whole system no longer functioned as well.

The Slashdot staff was also inundated with email complaints about karmafor instance, "My latest comment was modded up to 5 but my karma went down; I think your system is broken." Of course, in this context, karma is just a technical term for the sum of a formula used in the Slashdot system; as such, it could not possibly be "broken." Furthermore, the code for Slashdot has always been available as open source, meaning that those who really wanted to understand their karma rating could have done so. Human nature being what it is, however, people quickly slipped into thinking that there was some real thing to which karma corresponded and which the system was trying to approximate. In the end, the only workable solution has been to keep karma ratings private and give users only a vague approximation of their karma ratings.

Slashdot also evolved to have a number of "social engineering" elements that did as much to channel users' behavior as any of the technical features. Some of these social engineering elements are blatant, like referring to not logged-in posters as "anonymous cowards." Similarly, wildly off-topic or deliberately inflammatory posters are referred to as "trolls."^[2]

^[2] The term troll does not originate with Slashdot, but in fact dates back to the early Usenet discussion forums on the Internet. Presumably it's a reference to the children's story "The Three Billy Goats Gruff," in which a troll lurks under the bridge waiting to ambush hapless passers by.

Some of these elements are subtler. Watch a first-time visitor try to find the "submit" link, for example. Visitors aren't actively encouraged to submit; you have to really want to get your submission in. Creating that barrier to entry means that the overall submission is of higher quality. Because some effort is required to learn how to submit, people who put some thought into their submissions are more likely to submit.

The site also has a distinct personality, one that does not take itself too seriously. This is apparent from the self-deprecating tag line ("News for nerds; stuff that matters") to the obvious humor in many of the weekly polls. Staff on the site are referred to by their nicknames, usually originating as nicks on an IRC network or handles on a BBS. These nicknames have deliberate cultural references meant to be understood by an audience with the right cultural background. "Cowboy Neal" derives from a character in Kerouac. "CmdrTaco" is a reference to a Dave Barry column. If you haven't watched "The Simpsons" or "South Park" regularly, much of the humor on Slashdot will pass you by.

In its own way, this too is part of the social engineering. The site personality and the cultural references are a subtle test for like-mindedness with the audience, a way of encouraging participation from those who "get it" and distancing those who don't.

Slashdot grew up a lot as a site in the summer of 1998. That spring the site began running banner advertising as a means of generating revenue. Banner sales were originally outsourced to a third party, but out of frustration with the ad sales company's inability to manage sales or understand the Slashdot audience (who still vociferously complain at the use of Flash animations in ads), advertising sales were brought back in-house in July of 1998 and were managed by Jeff Bates.

With little fanfare, Rob Malda quit his job in August 1998 and became Slashdot's first full-time employee. While other staff continued to work on a part-time or volunteer basis, this was a significant milestone. Less than a year after the registration of the Slashdot domain name, the Blockstackers had gone from running a web site to running a business.