The Free User-Generated Content Myth The name millions of Web users know me by is "Roblimo," my nom-de-net on Slashdot (www.slashdot.org), a site started in 1997 by Rob "CmdrTaco" Malda, then a student at Hope College in Holland, Michigan. The site was a hobby at first. Rob and his friend Jeff "Hemos" Bates and a few others posted little stories and observations about technology, science, and computer software (especially Linux), and readers posted responses. While Rob was busily starting Slashdot, I was freelancing for Netly News (part of Time Life's now-dead Pathfinder site) and assorted print publications, and operating a small but profitable limousine service. I enjoyed Slashdot thoroughly; not only were Taco's and Hemos' and their friends' articles fun to read, but they often made excellent points, and readers' posts added to those articles often made even better ones. There was a pleasant, clubby feel to Slashdot in the early days. You could say almost anything, and almost anyone could say almost anything back. There was a little vituperation at times, but it was generally a good-hearted group. Slashdot has drawn criticism for its poor English usage from the beginning, but I never minded Rob's loose I prefer to call it "unique" approach to spelling and grammar. If anything, I felt, it encouraged posts by others, especially programmers and technologists who were experts in their fields but not confident of their writing skills. The site's motto was (and still is) "News for Nerds. Stuff that matters," not "Stories and links posted only after care-ful proofreading by professional editors." Fast-forward to early 1999. I'm writing and editing full-time for a small online publisher called Andover News Network that is running on venture capital and dreaming of an IPO. Taco and Hemos have graduated from college and are trying to make a full-time go of Slashdot. They're up to half a million pageviews a day, and they are overwhelmed by the sheer amount of work it takes to run the site. Linux use has exploded, and Linux news is now a large proportion of Slashdot's content. Linux companies are offering to buy Slashdot for ever more foolish amounts of cash and stock as both the general dot-com investment frenzy and the "Get rich from Linux" investment fads near their peaks. For various reasons, including a contract that guarantees them nearly absolute editorial freedom, Rob and Jeff take Andover's offer, even though it is not as high as others they have gotten. Andover selects me as the editorial overseer of Slashdot and the Linux software index site freshmeat (www.freshmeat.net) which they also bought in mid-1999. I was already more familiar with Slashdot than most; I had User ID #357 out of over 100,000 registered users (a number that grew to over 600,000 by the end of 2001, and is still growing). I was a regular, well-known participant in Slashdot discussions, and many stories I had written elsewhere had been linked to by Slashdot and dissected in some of those discussions. But I was not fully prepared for the overwhelming amount of work it took to make Slashdot run, seemingly without effort, day after day. Evaluating Reader Submissions Takes Time The first morning I actively worked on Slashdot, I found that almost 100 stories had been submitted overnight. By the time I had sorted through all of them, looking for the five or ten we might want to post to start the day, I easily could have written a full day's worth of Slashdot stories (generally about 15) on my own, and had the rest of the day to myself. I knew that this would have destroyed Slashdot, which is what it is because it is almost entirely reader-driven. But that first Slashdot submissions bin experience drove home something every Web publisher who wants to save money by replacing professionally-written material with reader-generated content must realize: The amount of time and effort it takes to find usable submissions in what amounts to a huge, ever-changing daily slush pile easily exceeds the amount of time it takes an experienced writer to produce the same amount of material from scratch. Then there are the Slashdot reader comments. They are "free content" in an immediate sense, but in the long run they are not. A reader types his or her thoughts into an online form (or pastes copy written in a text editor into the form), touches a "submit"button, and the comment appears almost instantly. Content from heaven! Except that some of this "free content" may be full of obscenities; or contain other people's copyrighted material; or contain libelous statements or material that can otherwise cause trouble, including the kind of trouble that leads to high legal bills. Plus, in a reworking of Gresham's Law, "Bad money drives out good," about the debasement of currencies, a site full of dreck comments drives off readers who might have posted level-headed, intelligent ones. At the same time, keeping posting fast and easy, and allowing anonymous posts, often gives Slashdot important reader-generated material found nowhere else. The balance between free speech and repelling readers (and attracting lawyers) can be thin and hard to define. Rob Malda's solution was to devise a system that would allow readers to moderate each other's comments, but would not allow a reader to both post and moderate in the same discussion. This worked up to a point, but around the time Slashdot started to get an average of one million daily pageviews, the system started breaking down because some readers decided to post comments that had no other purpose than to gain them moderation points, or "Slashdot karma," which defeated the original purpose of moderation. Posts were rated on a scale of 1 to +5, and the default "moderation level" at which readers saw Slashdot discussions was +1. This made the worst posts invisible to someone who casually looked at Slashdot for the first time, although registered readers were free to change their normal viewing level either upwards so they saw only posts rated at +2 or +3 or higher, or downwards, right down to 1, so they could read everything, no matter how bad. And then came moderation system abusers who set up numerous Slashdot "identities" so they could post with one identity and moderate with another, and controls had to be instituted to keep them in check. Other Slashdot abusers wrote little software routines whose sole purpose was submitting obscene or useless material over and over, several times a minute, and more software had to be written to protect against them. It became a constant battle, one that has not ended and shows no sign that it is going to end as long as Slashdot exists. We simply accept the fact that there are thousands of Slashdot users who seem to think of the site as an online game rather than as a news and discussion system. Some of them even have sites and email groups set up where they talk about nothing but "trolling" Slashdot in one way or another. "Thousands of Slashdot trolls" sounds like a lot, but Slashdot gets around one million page-views on its slowest days, and over two million on its busiest ones. The abusers are only a tiny percentage of the total Slashdot population. The problem with this whole game is that Slashdot's programmers are outnumbered and are constantly playing catch up. No sooner do they eliminate one loophole, such as writing a script that prevents some of the more prurient users from posting pornographic "art," than another user comes up with a new way to subvert the scheme. That problem gets solved, then another one comes up. This race has been going on, at an increasing pace, since Slashdot broke out of its original little club-like niche and started attracting a mass audience. More readers will only make the problem worse. Slashdot's comment handling, even without defenses against unruly posters, takes hardware and software far more complex than you need to put up professionally-generated content on a fixed schedule. Slashdot's database has new entries written to it many thousands of times each day, and at the same time, thousands of users are accessing the site simply to read it. Comments don't come in, and pages aren't sent out, at an even hourly rate. There are peak load times when up to 100,000 users may try to access the site at almost exactly the same moment, while 1000 or more are simultaneously posting comments. This takes dozens of server computers to handle, and the load balancing that makes them work together smoothly (at least most of the time) takes nearly constant human babysitting. None of this would happen on a small discussion site that got only a few thousand visitors and a dozen posts per day, but that small site couldn't survive on ad revenue. You need a minimum of 50,000 daily pageviews to make any kind of profit from a discussion site, and from there on up you are going to discover that discussion sites don't scale as profitably as straight-up news sites. A story you write (or pay to have written), once posted, needs only to be served and sent out. The more people who read it the better, because the additional technical cost for each additional reader, once the story is written and saved to your server, is negligible, while the technical cost of hosting discussions goes up as your audience increases, not quite in a linear fashion but close enough to it that you will not see profit margins increase with additional readership nearly as fast as you will with content which you generate yourself. Don't Forget the Lawyers Professional journalists are usually keenly aware of legal issues. They are supposed to know how to "source" a story correctly; to take careful notes or make tape recordings of interviews so subjects can't later claim they were misquoted; to avoid copyright infringement or plagiarism, and generally avoid getting sued for petty reasons. Volunteer posters have no such constraints. If you allow open posting on your site, you will inevitably attract comments that irritate someone who has an attorney on retainer, and when that happens, you had better have one of your own. All meaningful legal complaints we have had at OSDN have been over reader contributions, and most of them originate on Slashdot. The most common problem is copyright infringement. We have had readers post everything from articles that belong to other online publishers, to pieces of the Church of Scientology's fiercely-defended "scriptures," to program code claimed by Microsoft Corporation. I assure you, from personal experience, that hosting open discussions online can create more legal headaches than almost anything else you can do on the Internet. Do you still want to run a discussion site, or have I talked you out of it? |
