The more voices, the better the result in Wiki world
Here's the pre-edited version of my latest article for news@nature…

The secret to the quality of Wikipedia entries is lots of edits by lots of people

Why is Wikipedia so good? While the debate about just how good it is has been heated, the free online encyclopaedia offers a better standard of information than we might have any right to expect from a resource that absolutely anyone can write and edit.

Three groups of researchers claim now to have untangled the process by which many Wikipedia entries achieve an impressive accuracy [1-3]. They say that the best articles are those that are highly edited by many different contributors.

Listening to lots of voices rather than a few doesn't always guarantee the success that Wikipedia enjoys – just think of all those rotten movies written by committee. Collaborative product design in commerce and industry also often generates indifferent results. So why does Wiki work where others have failed?

Wikipedia was created by Jimmy Wales in January 2001, since when it has grown exponentially both in terms of the number of users and the information content. In 2005, a study of its content by Nature [4] concluded that the entries were of a comparable standing to those generated by experts for the Encyclopaedia Britannica (a claim that the EB quickly challenged).

The idea behind Wikipedia is encapsulated in writer James Surowiecki's influential book is The Wisdom of Crowds[5]: the aggregate knowledge of a wide enough group of people will always be superior to that of any single expert. In this sense, Wikipedia challenges the traditional notion that an elite of experts knows best. This democratic, open-access philosophy has been widely imitated, particularly in online resources.

At face value, it might seem obvious that the wider the community you consult, the better your information will be – that simply increases your chances of finding a real expert on Mozart or mud wrestling. But how do you know that the real experts will be motivated to contribute, and that their voices will not be drowned out or edited over by other less-informed ones?

The crucial question, say Dennis Wilkinson and Bernardo Huberman of Hewlett Packard's research laboratories in Palo Alto, California, is: how do the really good articles get to be that way? The idea behind Wikipedia is that entries are iterated to near-perfection by a succession of edits. But do edits by a (largely) unregulated crowd really make an entry better?

Right now there are around 6.4 million articles on Wikipedia, generated by over 250 million edits from 5.77 million contributors. Wilkinson and Huberman is have studied the editing statistics, and say that they don't simply follow the statistical pattern expected from a random process in which each edit is made independently of the others [1].

Instead, there are an abnormally high number of very highly edited entries. The researchers say this is just what is expected if the number of new edits to an article is proportional to the number of previous edits. In other words, edits attract more edits. The disproportionately highly edited articles, the researchers say, are those that deal with very topical issues.

And does this increased attention make them better? Yes, it does. Although the quality of an entry is not easy to assess automatically, Wilkinson and Huberman assume that those articles selected as the 'best' by the Wikipedia user community are indeed in some sense superior. These, they say, are more highly edited, and by a greater number of users, than less visible entries.

Who is making these edits, though? Some have claimed that Wikipedia articles don't truly draw on the collective wisdom of its users, but are put together mostly by a small, select elite, including the system's administrators. Wales himself has admitted that he spends "a lot of time listening to four or five hundred" top users.

Aniket Kittur of the University of California at Los Angeles and coworkers have set out to discover who really does the editing [2]. They have looked at 4.7 million pages from the English-language Wikipedia, subjected to a total of about 58 million revisions, to see who was making the changes, and how.

The results were striking. In effect, the Wiki community has mutated since 2001 from an oligarchy to a democracy. The percentage of edits made by the Wikipedia 'elite' of administrators increased steadily up to 2004, when it reached around 50 per cent. But since then it has steadily declined, and is now just 10 per cent (and falling).

Even though the edits made by this elite are generally more substantial than those made by the 'masses', their overall influence has clearly waned. Wikipedia is now dominated by users who are much more numerous than the elite but individually less active. Kittur and colleagues compare this to the rise of a powerful bourgeoisie within an oligarchic society.

This diversification of contributors is beneficial, Ofer Arazy and coworkers at the University of Alberta in Canada have found [3]. They say that, of the 42 Wikipedia entries assessed in the 2005 Nature study, the number of errors decreased as the number of different editors increased.

The main lesson for tapping effectively into the 'wisdom of the crowd', then, is that the crowd should be diverse: represented by many different views and interests. In fact, in 2004 Lu Hong and Scott Page of the University of Michigan showed that a problem-solving team selected at random from a diverse collection of individuals will usually perform better than a team made up of those who individually perform best – because the latter tend to be too similar, and so draw on too narrow a range of options [6]. For crowds, wisdom depends on variety.

Reference
1. Wilkinson, D. M. & Huberman, B. A. preprint http://xxx.arxiv.org/abs/cs.DL/0702140 (2007).
2. Kittur, A. et al. preprint (2007).
3. Arazy, O. et al. Paper presented at 16th Workshop on Information Technologies and Systems, Milwaukee, 9-10 December 2006.
4. Giles, J. Nature 438, 900-901 (2005).
5. Surowiecki, J. The Wisdom of Crowds (Random House, 2004).
6. Hong, L. & Page, S. E. Proc. Natl Acad. Sci. USA 101, 16385-16389 (2004).