The Register’s Andrew Orlowski finds “encouraging signs from the Wikipedia project, where co-founder and überpedian Jimmy Wales has acknowledged there are real quality problems with the online work.” Noting that up to now “criticism from outside the Wikipedia camp has been rebuffed with a ferocious blend of irrationality and vigor,” Orlowski quotes from an October 6 post that Wales made in response to my criticism of Wikipedia’s quality: “I don’t agree with much of this critique … but the two examples he puts forward are, quite frankly, a horrific embarassment. [[Bill Gates]] and [[Jane Fonda]] are nearly unreadable crap.” Although the Internet encyclopedia “appears ill-equipped to respond to the new challenge,” writes Orlowski, “at least Wikipedia is officially out of QD, or the ‘Quality Denial’ stage.”
Other responses to my critique from wikipedians (as I write that word, I am overwhelmed by a vision of very tiny, very cheerful people dressed in green velvet dancing in a dell) point out that some of the encyclopedia’s entries, particularly those on technical or otherwise esoteric subjects, are quite good. That’s true, and it’s worth thinking about. Why are entries on arcane topics generally better than those on more broadly understood topics? I would guess that the reason’s pretty simple: Fewer people contribute to the arcane entries, and the contributors all tend to have some degree of specialized expertise in the field. For general-interest subjects, on the other hand, it’s a free-for-all. Everyone feels qualified to add his two cents, and the entries decay toward, as Wales puts it, “unreadable crap.”
In other words, the example of Wikipedia actually undercuts, rather than supports, the Web 2.0 tenet of “collective intelligence.” It reveals that collectivism and intelligence are actually inversely correlated. Here, then, is what I’ll propose as the Law of the Wiki: Output quality declines as the number of contributors increases. Making matters worse, the best contributors will tend to become more and more alienated as they watch their work get mucked up by the knuckleheads, and they’ll eventually stop contributing altogether, leading to a further fall in quality. As Dave Winer puts it, “No matter how good something is, there are always more idiots and morons to take it down.”
Now, there’s a way around this “collective mediocrity” trap. You can abandon democracy and impose centralized control over the output. That’s one of the things that separates open-source software projects from wikis; they incorporate a rigorous quality-control filter to weed out the crap before it pollutes the product. If Wikipedia wants to achieve it’s goal of being “authoritative,” I think it will have to abandon its current structure, admit that “collective intelligence” makes a pretty buzzphrase but a poor organizational model, and define and impose some kind of hierarchical power structure. But that, of course, would raise a whole other dilemma: Is a wiki still a wiki if it isn’t a pure democracy? Can some wikipedians be more equal than others?
I don’t think it’s necessarily true that the number of participants decreases the quality of the output. It depends on the subject and the people involved. Two physicists can write an entry on Quantum Mechanics, but twenty of them can fill it out will all kinds of specialized information that only two wouldn’t have to offer. But that’s because it’s a circumscribed topic relevant to a somewhat narrow community of practice (which is actually what wiki’s were sort of created to support to begin with).
Once you start opening things up to *anybody* about *anything* — i.e. “vulgar” interests — you will end up with mediocre writing and information about topics that appeal to the lowest common denominator. That’s just how human systems structure themselves.
So yes, once you take the “pure wiki” out of the rarified environment of specialists or small groups, you definitely have to impose some top-down structure and peer review. I don’t think anyone but the most absurdly wide-eyed idealists would say differently.
The trade-off on Mediocrity
What you can do in order to save projects like the wikipedia from turning into moronic proclamations? One option, as Carr suggested, is to impose some form of centralized control. This is also what is done with open source projects, wherein a lot of …
Good point, Andrew. I was talking about truly democratic wikis. Specialized wikis, with expert participants, would certainly get better as the membership grows. I wonder, though, whether even these would reach a breaking point – beyond which additional members begin to erode quality rather than bolster it.
Your Law of the Wiki that output quality declines as the number of contributors increases is an interesting observation and is more than likely true. However there is one more related observation of an information source – as the level of general knowledge on a particular topic decreases, the value of an article of that topic increases for the majority. For example, most people would claim reasonable knowledge of what tap water is, where it comes from and it’s general properties. As you might expect from your “law”, the article isn’t of a hugely high standard. However, the value to be gained by most from such an article is limited as there is less new information. In contrast, a lot of people might not know so much about heavy water and therefore the (higher-quality) article is of more value to them. Orlowski misses this point in the original article: The value of wikipedia’s content lies in the thousands of well-written articles of low knowledge base, rather than in those of common knowledge to which everyone feels they can contribute.
Does Wikipedia need editors?
Nicholas Carr’s skeptical posts about Wikipedia raise questions about media run by amateurs
In other words, wikipedia has a long tail (of sorts).
In his book, The Wisdom of Crowds, James Surowiecki noted 3 requirements that needed to be met for a crowd to be smarter than the individuals in it. One of the requirements was the ability to aggregate results, which is the one that Wikipedia needs to address.
In an open source project there are two ways to aggregate contributions: (1) by the owner of a current module incorporating the “good” code contributions & discarding the “bad”, and (2) by a new contributor setting up a new module if his contribution is rejected (i.e. forking). The users and other developers also vote on which code base they use.
For Wikipedia an alternative to the “hierarchical power structure” proposed by this blog might be to give ownership of an article to whoever wrote it. People who want to improve it need to submit their changes to be incorporated by the page owner. Like open source, some of these contributors will be granted the right to edit the page directly & to approve other suggested changes. If a person’s suggestions are rejected then he or she has the option of creating a new page on the same topic.
Anyone trying to visit that topic would then see a special page with pointers to all the different articles various people had written—hopefully ranked in order of usefullness by reader votes. Such a process rewards quality without the need for a centralized authority structure.
how to become a wiki critic and get rich quickly?:) not unlike in academia, ownership of an article in wikipedia could lead to what is known from domain grabbing, buzzwordism, and folksonomy-sloganomics, finding words and terms which are formerly unused and claim them related to an author name or brand. certainly the desire to become a “term-owner” can lead to more diversity, but not neccesarily to more substance. on the other hand people would become more inventive of establishing and extending existing terminologies. however wikipedia due to an architect (jimbo wales) it’s rather a cathedral than a bazaar, thank god :)
There’s another dynamic at work here, too, and simply giving “ownership” of a subject to an individual may amplify Wikipedia’s quality problems.
Former contributors to the project, such as Jason Scott (here and here) point out that the project rewards administration rather than content. This deters expert contributors, who simply give up fighting. The winner of a dispute is the last man standing.
It’s the “Look busy, here’s the boss” syndrome. I hear this so often it can’t be attributed to sour grapes from someone who’s lost a vote.
The main difference between this and other similar academic environments is the pure speed at which stuff can happen; you can literately have 30-40 little editing nibbles on a page within a single day. If people are feeling frisky, it can take place in a few hours. This means that you get all the politics and turf war of Ivory Tower Academia without the mitigating barrier of time to cool down or consider. That is, you get a nice big mess.
and …
Wikipedia has a large contingency of users who play the Wikipedia Rules of Etiquette and Procedure like they were Role Playing Games and function within them causing havok and personal gratification at the expense of moving the project forward.
The result is that even less premium is placed on quality. Look how highly the issue figured in the candidates’ election literature. One wrote of “… our core principles of neutrality, civility, freedom and openness“. Creating a great reference work isn’t even a goal.
Institutions can and do change, but only when they realize they must reform to stay relevant. Such is the determinism exhibited by Wikipedia’s supporters – it reminds me of Marxists – this may not happen any time soon. But it will.
If we’re honest, the only reason Wikipedia has generated such attention is through the absence of free online reference material, because of copyright holders’ compensation issues.
But these are fixable: the quality of reference works available to me instantly online here is amazing – have a look. This is Wikipedia’s real competition.
Libraries are already experts at collectively negotiating with rights holders. If I had to place a bet on the future of reference works, I’d bet on this.
Del.icio.us is another example where the quality declined as the number of contributors increased. In its initial run the social bookmarkers of del.icio.us were the technical and web-savy elite and inevitably they converged on interesting things. Now the bookmarks and quality of annotation (‘tags’) are mostly noise/
After a similar discussion with students, I posted an entry to Wikipedia which, if not corrected, would be perfectly innocent but which should not have been allowed. In fact it was picked up. So some form of auto-regulation is in place. However, many of the articles are very incomplete and one-sided. If the specialised pieces seem better it may be because they are harder to understand. Maybe all entries should remain but readers should assign them ratings, which would automatically force the dross to the bottom.
Another relevant point is that if you make a fool of yourself on wikipedia, you don’t do it in front of an audience you care about. We’re all tourists there. That’s much less true of specialist subjects.
Another relevant point is that if you make a fool of yourself on wikipedia, you don’t do it in front of an audience you care about. We’re all tourists there. That’s much less true of specialist subjects.
Though I don’t mean to act as a hype-mouthpiece for an unfinished project, I did come across a project on Seth Godin’s website (http://sethgodin.typepad.com/) yesterday that perhaps offers an approach to this problem. Read down for “Squidoo”. From what I can gather (and I might have it wrong), they are seeking to keep individuals’ inputs separate. Seems like this would go a long way towards solving this problem of mixing red, green and blue and only ending up with grey.
Nick’s focus on QC at Wikipedia is laudable and necessary. The examples he cited were atrociously bad, indeed.
It’s good to note also the high-quality entries there; they include “This is Spinal Tap” as well as OpenDocument.
I do not believe the Gates & Fonda entries — or any of the possibly thousands which are incomplete or distorted — are enough to take Wikipedia down, nor to quell my own enthusiasm for some of the pricipals of Web 2.0 we’ve been talking about a lot lately.
The educated human can suss information quality for herself — and Bill Gates or Jane Fonda could with little trouble have their maids emend the text.
You have some valid points but I have to disagree. The concept of Collective Intelligence promises a potential for great knowledge but ONLY if ALL participate and offer what they know (see Pierre Levy’s book Collective Intelligence for more). The real problem here isn’t that there are immature yahoos posting inaccurate information because those posts are rectified quickly enough. The problem is that not enough qualified, intelligent people are willing to share what they know for the greater good.
What’s the last thing YOU added to Wikipedia? Why critique it when we can improve it?
Wikipedia, and many other Web 2.0 applications, function on an evolving knowledge model that encourages valid information (don’t read that as accurate because I think that biased information can just as important) will be sustained and invalid information will be cut out. It’s a “Survival of the Fittest” model for knowledge collection and it can work.
Actually, your “law of the wiki” has been formulated more generally in the past, and is accompanied by an equal and opposite formulation of why problems occur with critical but esoteric issues about which few have expertise:
http://www.freebsd.org/doc/en_US.ISO8859-1/books/faq/misc.html#BIKESHED-PAINTING
I had just posted this link in a comment on your October 3rd blog entry; it seems as if it may be even more relevant to this one.
I’m not sure how a discussion of “collective intelligence” is at all relevant to Wikipedia.
By definition, Wikipedians are supposed to regurgitate unbiased articles based on existing knowledge and universally accepted meanings. Just the facts, with no room for independent thinking, creativity, knowledge creation, etc. Indeed “original research” is actively discouraged, and removed from the site.
This is not collective intelligence we’re talking about, this is more just a collective unpaid internship.
As to your point that “Output quality declines as the number of contributors increases” – I’m not sure if I follow your jump. As long as there is sufficient fragmentation – experts in Britney Spears are not posting in the Quantum Physics articles, and vice versa – I’m not sure if I buy that simply increasing the number of unspecialized participants will infect the specialized articles.
Wiki bad or wiki good?
One problem with the participation model is that so much of it is fueled by idealists. Well, it’s not totally a problem, because we need idealists. But it makes the “movement” behind the model seem naive to the more realistic and/or …
@ Sarah:
If anything, *over* contribution seems to be the problem, as much incomplete contribution. Large chunks of our culture emphasises blagging, the act of doing your best to seem and act like you know what you’re talking about and in most cases that’s a useful skill. Armchair professionals and all that. Rarely are we taught to confess that “I don’t know anything about that” and back off and wait for someone who knows better to come along. So you get a lot of people chipping in, feeling like they should add their 2 cents to a particular article, but actually they are only serving to dilute the quality of information rather than enhance it.
And um … “What’s the last thing YOU added to Wikipedia? Why critique it when we can improve it?” … by critiquing it and raising this debate, he’s helping to improve it. He doesn’t have to act directly to assert influence.
Those that can, do. Those that can’t, teach. Those that have nothing better to do, maintain Wiki. The competent ones are too busy “doing” to babysit the Wikers’ work.
Let’s face reality. The ones who are most competent aren’t going to be uber-contributers because they are busy doing, and wouldn’t get paid to “contribute.” Their contribution is focused elsewhere. Traditionally, their teaching comes through mentoring, leadership in management, or formal teaching at a school after they have established themselves. Their time would be wasted performing maintenance on a Wiki where their contribution could get overwritten by an eager beaver who did enough cleanup to become an administrator.
That’s not to insult all Wiki contributers. But I’ve been exposed to quite a few situations (without being personally involved) in which competent people have their competent entries replaced by incompetent, biased, or politically-polerized fluff. It happens. Although Wiki’s are useful, they are not a haven for dependably trustworthy information.
To Jud-
“By definition, Wikipedians are supposed to regurgitate unbiased articles based on existing knowledge and universally accepted meanings. Just the facts, with no room for independent thinking, creativity, knowledge creation, etc.”
Supposed to…yet some of the better-quality entries will always reflect special knowledge.
Taking my example of the OpenDocument entry, there is a lot of knowledge imbedded in David Wheeler’s recent updates, in the organization and selectivity as to what is relevant.
The facts aren’t the half of it.
Sam –
That was actually Lawrence Coburn’s comment you were replying to (comments precede rather than follow sigs).
Nick Carr’s Amorality
Cast aside the anti-hype rhetoric, and keep in mind it is an argument not of fact or policy, but value, and you will find Nicolas Carr’s post on the amorality of Web 2.0 has a salient point — that…
Web2.0 repairs needed
In this article, the founder of Wikipedia admits to serious quality problems to one of the much evangelicasized (is that word??) services coming out of Web2.0. I’ve questioned Wikipedia for a while now. Sure it is great to get a gist of a part…