Wikipedia

Why is Wikipedia as good as it is? While the debate about precisely what level of goodness that entails has been heated, the free online encyclopaedia offers a better standard of information than many would have expected from a resource that absolutely anyone can write and edit.

Three groups of researchers claim to have untangled the process by which many Wikipedia entries achieve their impressive accuracy^1,2,3. They say that the best articles are those that are highly edited by many different contributors.

Listening to lots of voices rather than a few doesn't always guarantee the success that Wikipedia enjoys — just think of all those rotten movies written by committee. Collaborative product design in commerce and industry also often generates indifferent results. So why does Wikipedia work where others have failed?

Dennis Wilkinson and Bernardo Huberman of Hewlett Packard's research laboratories in Palo Alto, California, addressed the specific question of how Wikipedia's better articles get to be that way. Specifically, do edits by a (largely) unregulated crowd really make an entry better?

Right now there are around 6.4 million articles on Wikipedia, generated by over 250 million edits from 5.8 million contributors. Wilkinson and Huberman have studied the editing statistics and found that they don't simply follow the statistical pattern expected from a random process in which each edit is made independently of the others¹.

Highly edited

Instead, there is an abnormally high number of very highly edited entries. The researchers say this is just what is expected if the number of new edits to an article is proportional to the number of previous edits. In other words, edits attract more edits. The disproportionately highly edited articles, the researchers say, are those that deal with very topical issues.

And does this increased attention make them better? So it seems. Although the quality of an entry is not easy to assess automatically, Wilkinson and Huberman assume that those articles selected as the 'best' by the Wikipedia user community are indeed in some sense superior. These, they say, are more highly edited, and by a greater number of users, than the less visible entries.

Who is making these edits, though? Some have claimed that Wikipedia articles don't truly draw on the collective wisdom of its users, but are put together mostly by a small, select élite, including the system's administrators. Wikipedia co-founder Jimmy Wales has admitted that he spends "a lot of time listening to four or five hundred" top users.

Aniket Kittur of the University of California, Los Angeles, and co-workers have set out to discover who really does the editing². They have looked at 4.7 million pages from the English-language Wikipedia, subjected to a total of about 58 million revisions, to see who was making the changes, and how.

The results were striking. In effect, the Wiki community has mutated since 2001 from an oligarchy to a democracy. The percentage of edits made by the Wikipedia 'élite' of administrators increased steadily up to 2004, when it reached around 50%. But since then it has steadily declined, and is now just 10% (and falling).

Weight of numbers

Even though the edits made by this élite are generally more substantial than those made by the masses, their overall influence has clearly waned. Wikipedia is now dominated by users who are much more numerous than the elite but individually less active. Kittur and colleagues compare this to the rise of a powerful bourgeoisie within an oligarchic society.

This diversification of contributors is beneficial, Ofer Arazy and colleagues at the University of Alberta in Canada have found³. In 2005, when Nature 's news team arranged for expert comparisons between articles in Wikipedia and Encyclopaedia Britannica online, the experts found only a moderate excess of errors in the Wikipedia articles⁴. (The idea that the two sources were broadly similar was vigorously challenged by the Encyclopaedia Britannica; see http://www.nature.com/nature/journal/v438/n7070/full/438900a.html and http://www.nature.com/news/2006/060327/full/440582b.html) Arazy's team says that of the 42 Wikipedia entries assessed in the article, the number of errors decreased as the number of different editors increased.

The main lesson for tapping effectively into the 'wisdom of crowds', then, is that the crowd should be diverse. In fact, in 2004 Lu Hong and Scott Page of the University of Michigan in Ann Arbor showed that a problem-solving team selected at random from a diverse collection of individuals will usually perform better than a team made up of those who individually perform best — because the latter tend to be too similar, and so draw on too narrow a range of options⁵. For crowds, wisdom depends on variety.

Visit our morethe_wikier.html">newsblog to read and post comments about this story.  

• References

  1. Wilkinson D. M. & Huberman B. A. preprint http://xxx.arxiv.org/abs/cs.DL/0702140 (2007).
  2. Kittur A., et al. alt.chi, preprint (2007).
  3. Name A., et al. Arazy, Paper presented at 16th Workshop on Information Technologies and Systems, Milwaukee, 9-10 December 2006.
  4. Giles J., et al. Nature, 438 . 900 - 901 (2005). | Article |
  5. Hong L. & Page S. E. Proc. Natl Acad. Sci., 101 . 16385 - 16389 (2004).