ASCII by Jason Scott

Jason Scott's Weblog

The Great Failure of Wikipedia —

I have now tried extended interaction with Wikipedia. I consider it a failure. In doing so, I will describe why, instead of just slinking off into the night on my projects. Maybe it will do some good. Maybe it will not. I’m sure, at the end of the day, there must be hundreds like me at this point. Burned, slapped, ejected from the mothership for not following the rules, no matter how intricate and foolish. Let me at least go with some smoke.

The concept of Wikipedia is a very engaging and exciting one, especially to someone like myself who spends an awful lot of time collecting information and then presenting it to people. Normally, the work I do is the work that’s done. That is, if I don’t give much attention to a specific section of my sites, that section will stay static, even if it’s in need of improvement. This is not very enjoyable. In collaboration, you will put your tools down for the night, and when you wake up the next morning, more work is done. This is very exciting, very enjoyable. It’s why people work in teams in the first place.

Within the social spectrum of information specialists, I am best classifyable as a moody loner. I don’t work well with others, at least, that is, people who I don’t like. Which is a lot of people. As a result, the vast majority of my sites are one-man operations, meaning that the firehose of my concentration goes from one site to another, giving it some sort of monsoon season of attention and update before firing in another direction. This means that some are truly ghost towns for months at a time. With the additional strike of my documentary, my time is almost completely eaten up, and so the other sites have been suffering.

The idea of Wikipedia, on its face, is really neat. A bunch of people can work on an entry in a huge, growing encyclopedia, its subject matter gaining far and wide, and deep, into the whole of human knowledge. Pretty cool stuff to hear about, as you browse the outside of it. You click on some of the more complete entries, and really, you just say to yourself “wow, such a great thing is man” and so on. Maybe you wave a little flag with a logo on it. Wikipedia’s watchword is entirely collaboration. With a few exceptions, anyone can modify anything in any way, and anyone can undo their modifications, with a full tracking of the history of edits and methods included to keep track of these changes. Exciting stuff.

I had run into people who spoke of Wikipedia in a near-fanatical aspect, of how all these hands were forming these great and beautiful entries, and that it was just a matter of running along and having a great time making the whole project better. Naturally, I regarded this with suspicion and hopeful interest. I’m always interested in people doing stuff tangental to the work I do, but I always wonder if it will turn out my work has been for naught, or if in fact I’m still doing something unique and the efforts being expended on the other project are unrelated.

On the other hand, it is an awful lot of work tracking down the history of America Online or John Paul Aleshe or any of the other subjects I am always researching, and a bunch more hands kicking in would be fantastic. So I bought in, for a little while. Signed myself up and started some work.

I should pause for a moment, before I continue further. If you work on Wikipedia, I’m just going to make you angry. What I am doing is trying to stop people from working on Wikipedia with the idea that they’re accomplishing good. I am quite convinced, from the outside, over here, that no amount of suggestions from a lone voice will have any effect. Mobs don’t listen. So please, traipse happily back to Wikipedia and get cracking; someone is not eschewing a NPOV, even as you read this. Go! Go!

Note, also, that in what I’m writing about, I’m not speaking about the concept of a “Wiki”, or indicating that a collaborative tool such as Wiki software is a failure. Confined with a number of limitations on who does what in the context of undoing work, it can certainly work. It’s just software, after all. it’s the implementation that’s the sticking point.

To understand Wikipedia, it helps to understand the Usenet FAQ and its unique place in history.

The Usenet FAQ was (and is) to me, one of the true great advantages and creations of the Internet age. Previously, it had always been the case that the same 5 or 10 questions plagued a subgroup, cultural icon, hobby or occupation. These questions, while quite valid, quite reasonable, would be asked so many times that it would eventually be the case that no-one was willing to step up for the thousandth time and explain them. This led, inevitably, to speculation, wrong information and misquoting that would come back to bite everyone later. For no good reason! But the Frequently Asked Questions list fixed that. It allowed all the common questions to be answered, and even for the common controversies to be addressed even if no firm conclusion had been reached. These things grew like crazy in the 1980s and there’s massive collections of them out there, still with good information (as long as its a general subject, and not a pop star or the like), and the work of many people coming together to build something good. Like Wikipedia is supposed to be.

I would attest that the reason for the success of the FAQ was a lot of collaborators but a short list of people maintaining it. A very large amount of maintainers leads to infighting, procedural foolishness, and ultimately a very slow advancement schedule. There’s an interesting book called The Mythical Man-Month that goes into this in some detail, but the basic idea is: the more people you slap into a project that’s behind, the more the project will fall behind. Unintuitive, but true. Even in the case of raw horsepower, this becomes the case; you would think that if the basic job (photocopy this paper) was simple enough, the job would go faster with more people, but it doesn’t. You end up with people photocopying stuff wrong, collating wrong, bending pages badly, skipping pages… and the errors increase as you smack more people on. And you fall behind.

Now, at the risk of sounding a tad elitist and exclusivist, a low barrier to entry leads to crap. Maybe not initially, but with any amount of quality attached to a project, once it gains some respectability and perhaps fame or infamy, it is then beset upon by crap. By making it really, really easy to change, fundamentally, the nature of a project, you run the risk of the project becoming a battleground. A really, really crappy battleground.

For exhibit A, one merely has to traipse over to the Internet Movie Database (IMDB), which is now a sub-company of Amazon. For better or worse, Amazon now defines that body of information’s future, and along the way they ended up adding a few nice features (like a very fast search engine, and links to buy the movie if you so choose). But they also added user comments.

Have you seen the user comments there? Choose a movie; I won’t bias you. Go find the threads under a $100 million picture, an Oscar-winner or an independent production. Go browse them and bring to me a thread that isn’t a garbage pile of one-line off-the-cuff nothings, a handful of withered one-sentence dandelions. Devoid of insight, meaningless to anyone truly trying to find insight into the film at hand. In many cases, these films represent years of work in someone’s life, but because tourist3398 can just log right in and ask a completely stupid question, or make an inane comment, they get equal time on that front page for a while. Ultimately, it’s insulting to the work done and it adds nothing.

The reviews that accompany each movie are slightly better, because it appears there is a moderation system in place. Reviews that are fundamentally wrong are in place, but they’re wrong in a matter of opinion, not often in a matter of being unreadable. You disagree with what’s being said, not the brickheaded way it’s being said.

The simple fact is, a low barrier to entry and an easy access to an audience tends to lead to problems. A web-based comic named Penny Arcade captured this succinctly, but I want to add that the issue is not that damage will occur immediately, but will occur over time. And eventually, given enough time, damage and low quality will win over high quality, because high quality requires effort and low quality does not.

So we come to Wikipedia.

I often get myself into trouble and lose my audience with my metaphors, but so be it. Think of Wikipedia as a massive garage where you can build any car you want to. Great tools are provided, a lot of shop manuals are there, and you get your own lift and away you go. Fantastic. But every one else, and I mean everyone else in the garage can work on your car with you. There’s no “lead mechanics”, no “shop floor managers”, no anything. In fact, the people who are allowed to work on your car can completely disregard what you were doing with it. They could have flown in from Boola-Boola Island 2 hours ago, not know the language, can’t read the manuals, and just go in and paint your car pink. And drive it. And leave it somewhere. Now, since tools are free and paint is free and you can easily go and retrieve your nice car and get it back to something resembling sanity, a lot of the people in the garage see there’s no problems. But in fact, the fifth, or the hundredth time you’re traipsing down the lane to find your messed-up, polka-dotted, covered-in-chrome-pussycats car, you’re kind of inclined to drive it into the lake and leave it upside down, wheels spinning.

This is what the inherent failure of wikipedia is. It’s that there’s a small set of content generators, a massive amount of wonks and twiddlers, and then a heaping amount of procedural whackjobs. And the mass of twiddlers and procedural whackjobs means that the content generators stop being so and have to become content defenders. Woe be that your take on things is off from the majority. Even if you can prove something, you’re now in the situation that anybody can change it. And while that’s all great in a happy-go-lucky flower shower sort of way, it’s when you realize that the people who are going to change it could have absolutely no experience with the subject whatsoever, then you see where we are.

If you’ve ever worked in a large company, one where not everyone’s name is known by everyone else, you’ve bumped into these people, who don’t know the thing the company makes very well, don’t keep on top of new ideas beyond buzzwords, yet wield the kind of power where they can stop and start innovation and positive growth because they simply feel like it. It’s pretty heartbreaking stuff and I hope a bunch of you never have to deal with it.

But thanks to Wikipedia, you can experience this on a daily basis! College students with too much free time deciding your subject matter is not worth reporting. Bizzare insight from strange lands telling you they didn’t think your paragraph was relevant. And ever the bizzare need for a Neutral Point of View. Neutral Point of View is a doctrine about how Wikipedia articles should be written. Like wikipedia itself, it is a great idea in theory. In application, of course, it turns into yet another hammer for wonks and whackjobs to beat each other and innocent bystanders.

Wikipedia is a relatively new creation, but it already quite beset with the same problems that inhabit any self-styled intellectual collaboration. People make little empires, have their agendas, push through ideas and themes they want, and disregard and delete things they do not. The main difference between this and other similar academic environments is the pure speed at which stuff can happen; you can literally have 30-40 little editing nibbles on a page within a single day. If people are feeling frisky, it can take place in a few hours. This means that you get all the politics and turf war of Ivory Tower Academia without the mitigating barrier of time to cool down or consider. That is, you get a nice big mess.

One of the stated concerns of nanotechnology (wherein a bunch of tiny things make a lot of small changes very quickly, which should sound forever) is the ‘grey goo’ problem. The concern with grey goo is poorly programmed nanobots will make a bunch of wrong incremental changes to the world and reduce everything to a grey, goo-like substance of all creation. It is not hard, browsing over historical edits to majorly contended Wikipedia articles, to see the slow erosion of facts and ideas according to people trying to implement their own idea of neutrality or meaning on top of it. Even if the person who originally wrote a section was well-informed, any huckleberry can wander along and scrawl crayon on it. This does not eventually lead to an improved final entry.

Let’s put it another way. In the motion picture industry, there’s a term called “on the screen”. It’s phrased in this way: “is this money going to end up on the screen?”. And what is meant by that is that if you pay a bunch of money to rent a spectacular shooting location, then it’s going to end up on the screen, that is, people will see the spectacular location and go “wow” and they’ll feel the movie is giving them their money’s worth. If you pay for the rights to use a location, and then there’s a hurricane and your location is wiped out and you didn’t get insurance, then you just spent a lot of money, and did no shooting. Your money is not on the screen. Other than on a bunch of reciepts, there will be no trace of the $2 million you spent on that location rental. So there’s wasted money, energy and time, and that can add up. This is what plagued movies like Cleopatra, Waterworld and Heaven’s Gate, which contained huge behind-the-scenes costs that did not result in footage, meaning the movies were now expensive blockbuster budgets with medium-level footage to show for it.

I would contend to you that the Great Failure of Wikipedia is how little truly ends up “on the screen”.

As I said, you are no longer in the role of content generator soon after your works are exposed to the wonks and twiddlers and procedural whackjobs. You are a content defender and that means that time you could be spending finding new and interesting facts or finding original sources or otherwise making the world a better place (or at least an entry or two) is being spent explaining for the hundredth time that no, this really happened and yes, I got clearance for that photograph, and yes, I believe this shows a neutral point of view, and so on. It’s like you get to play one note of your trumpet and then you spend 20 minutes defending it. To anybody who walked in. Just now.

I’m sorry, but content creators are relatively rare in this world. Content commentators less so. Content critics are a dime a hundred, and content vandals lurk in every doorway. Wikipedia lets the vandals run lose on the creators, while the commentators fill the void with chatter. It is, a failure.

Naturally, the question that arises is what solutions would I suggest to fix this situation. Well, I continue to run my own private collection of data and research and will continue to do so. You know, generating content. I made the mistake of gifting over a photograph to Wikipedia back when I thought it would be a success and not a failure, but I will not make that mistake in the future. I may offer the works I have collected and generated essentially for free to the public, but I will not give permission to Wikipedia to use them. This actually breaks the Wikipedia way, because they need explicit permission to function. It’s another fatal flaw; they will not mirror content. They are terrified of copyright violation. They are scared of what might happen if they were to copy over some text in a fair use situation and they were to be sued. So they do nothing. This is why so many links from Wikipedia are dead.

I’ve seen people point to Linux/Open Source as an example of a Wikipedia-like entity accomplishing things, but the fact is that this is a false comparision. The vast, vast, vast majority of open-source projects have a small amount of maintainers and an audience of users who, upon being able to see the code, suggest changes. If maintainers suck, they fork, but more often than not, the maintainers are told of their bugs, of feature requests, and so on, and implement them, sometimes slowly and sometimes not. Incremental improvements, not waking up one day and finding the I/O libraries gone or switched to a neutral point of view. Maybe this is a natural maturation of collaborative projects that Wikipedia hasn’t gone through yet. Time will tell on that level.

Already, there are many Wikis out there, sites using Wikipedia-like software (but interestingly, not often the exact software, choosing instead to implement their own version of the heirarchy and approach) and then collecting knowledge. Wikipedia calls them “Knowledge Bases”. I call them “Wikipedias”. I think over time, people will want to get away from the grey goo of Wikipedia’s mess and move towards specialized or properly-run Wikis, which contain, not a cabal, but at least a slight barrier to entry that will ensure that the person who is going to undo your hours of work with a few mouse clicks is at least, from some relatively objective standpoint, vaguely entitled to do so.

Meanwhile, I will aim my energy in my own direction, knowing that while my tools will be where I left them the previous morning, they’ll also still be recognizable as my tools.


Categorised as: Uncategorized

Comments are disabled on this post


13 Comments

  1. Andy Baio says:

    Wow, that’s quite a rant! I’ve always been impressed with the scope and quality of articles on Wikipedia, even if the arbitration and dispute process is somewhat arcane and bizarre. Regardless, even though I’m not a contributor, I’ve been a big proponent of the site.

    I’m guessing that your big complaint stems from the Textfiles.com deletion ordeal. I say, don’t let a couple ignorant people spoil your opinion of the site. The fact that they soundly rejected the Vote for Deletion 28-2 shows me that the site seems to work pretty well.

  2. Jason Scott says:

    I don’t want to go into it too much, just mostly move on, but I will clarify and say that the textfiles.com arbitration process happened well after I’d finished this article; it had been sitting in my queue for some time, and then there was this whole thing with Bardex Enemas….. The documentary is taking precedence over everything else, and there are essays sitting around in my queue, looking to be researched or improved further before going live. As it stands, this happened to hit the wire just as the final ranges of my experimentation with Wikipedia were finishing up. I’m quite satisfied with my decision, and I’m secure in my prediction.

  3. peterb says:

    Well, I guess we’ll check back in a few years and see where things stand. Part of the problem, it seems to me, is that your definition of Great Failure doesn’t really mean very much. I’ve avoided doing any “work” for wikipedia because, well, I have a life and a job and my own weblog, and much prefer to focus my energies there. But as a -user-, I’ve found it to be an invaluable resource. So as a -user-, I call it a success. I could care less whether there was lots of “lost energy” in creating the articles I used, as long as in the end they’re accurate, and as long as they keep coming.

    Judged by that standard, the project seems to be doing fine — unless there’s some example you weren’t citing…?

    I understand that you’ve been traumatized by your experience with wikipedia, but to a fair approximation what I hear you saying is “I was horrified to see how that sausage was made.”

    I suspect that most people, in the end, won’t really care, as long as the sausage tastes good.

  4. Jason Scott says:

    Brilliant. You know, you’re the first person to give me feedback (others have read thing and talked to me in other channels) to nail, exactly on the head, where my problems would not be obvious or would be insignificant to the audience. You are correct; from the outside, to someone who is looking for basic information, a lot of Wikipedia will be ‘good enough’. Mistakes made will not be any more intensely different than anywhere else, that is, shallow take on the topic, common misperceptions (that are in a lot of sources) and so on.

    To a toolmaker and tooluser, that is, an information gatherer and disseminator, a researcher who is specifically trying to better the world by showing their findings, I find it very, very problematic, as I spend a lot of time in the sausage factory. And therefore my problems are completely on a different plane from yours and likely many readers of this weblog, or users of Wikipedia.

    While a newspaper might report on a race car driver’s work in a way the race car driver wouldn’t, that doesn’t mean that the final stuff doesn’t have some use, even though it omits details the race car driver wouldn’t in describing his work. And the race car driver would be miserable working for the newspaper.

    That’s really interesting!

  5. mediageek says:

    Criticizing Wikipedia; the Problem with Open Editing

    Jason Scott has an interesting critique of Wikipedia, from the standpoint of a contributor. He isn’t criticizing the , from the standpoint of a contributor. He isn’t criticizing the

  6. peterb says:

    Thanks for clarifying; that makes a lot more sense.

    Given that I work for a living with inferior development tools (gdb, anyone?) that are in widespread use because, let’s face it, worse is better, I feel your pain.

  7. Kevin says:

    This is tangentially related to semi-developed theories I have about why small community BBSes back in the day had a much higher signal to noise ratio than places like newsgroups and slashdot do now.

    Part of my theory is a smaller number of participants, but also that there were often hierarchies of access, often even to find the bbs number or initial password to gain access to the bbs before you could even ASK for access (which you’d get from other people you knew or from BBS lists from other similar boards).

    Then once you were on, there’d be general boards that you would be able to post on, but there were often other boards that you wouldn’t even know about until you were approved by a sysop/cosysop or some similar higher trusted user. You had to gain acceptance by ‘proving your worth’ in some sense, varying depending on the board. (large numbers of post, providing content, uploading, etc).

    Now, one hard part in all this is how you balance general wide access and appropriate limited use that seems to generate a high quality user community and experience.

    I think one start is mostly unlimited viewing access for most general content while having a hierarchy of users which limit modification of content or at the very least have a moderated approval process of content. I think there’s still a place for limited access to viewing and editing certain content before it’s released for general consumption.

  8. milli@kaewth says:

    My comments are aimed at questioning throwing out the proverbial baby with the dirty water. I realize you’ve made your choice, and you are entitled to it. I’m hoping you’d like to help others out who are dealing with these problems even though you’ve chosen to wash your hands of the baby with the dirty water, or something like that.

    I really liked Kevin’s idea:
    “I think one start is mostly unlimited viewing access for most general content while having a hierarchy of users which limit modification of content or at the very least have a moderated approval process of content. I think there’s still a place for limited access to viewing and editing certain content before it’s released for general consumption.”

    That would help. And maybe a more broadly-conceived and regulated page-locking policy, even though page-locking is not a good habit and kind of goes against the whole principle.

    Can you think of other improvements that would benefit, if not Wikipedia, other collaborative projects that are using Wikipedia as a guide?

    Obviously the NPOV doesn’t effectively attend to “people [who] make little empires, have their agendas, push through ideas and themes they want, and disregard and delete things they do not” or to vandals. As you said, it’s more like a weapon than a guideline. Maybe a NPOV commission or system instead of delete-first-blame-later?

    What about something that can accommodate individualist researchers, like a project manager who has demonstrated a level of knowledge receives help and can opt to refuse help on their locked pages? I don’t really see why it has to be “Collaborate or Die” in Wiki land. Can’t we have the best of both worlds?

    The copyright problem is probably something that Wikipedia is experiencing now more acutely, since copyright is just this big, big problem with any cultural creation now.

    Some forums have “newbie” or some ranking system which could locate people who don’t have anything invested in the project. I wonder if that could be implemented in the software somehow to (passively or silently) deal with the problems you’re bringing up.

    I’m also wondering if these problems are limited to Wikipedia’s controversial pages or if it is a problem system-wide.

    Sorry if I’m blasting questions here, just trying to learn more about collaborative projects in general.

    Thanks to author and authors of this post/comments.

  9. Wibble says:

    Worse is easier

    The ever brilliant Jason Scott has a ather long rant up on his site about collaborative writing, and the effect it has on quality of content. This is something I’ve come up against so many times on the KDE docs….

  10. Jason Scott says:

    Well, there’s something you’re fighting against here, and that’s where the real problems are, if an “alternative solution” is proposed.

    Wikipedia is, fundamentally, founded on anarchist/communist principles, which propose that people working as a collective will be stronger than whacko individualists locking out the people and controlling all the toys for themselves. Any divergence from these principles immediately lose the kind of people who are deeply into Wikipedia.

    That is, once you start adding these barriers to entry, then come the natural concerns about elitism, the belief that “they” will pull the power in for “themselves”, and in most ways, that people will be shut out of “the club”. And since this openness, this very thing I claim turns the endeavor to crap, is what the whole project is founded on, I just don’t see reconcilliation.

    As it is, my pages are actually sort of a collaboration. If you look at my BBS Software page, it is the result of dozens of people mailing me software and suggestions and stories, and I implement them with attribution. But this wouldn’t be acceptable by Wiki standards, nor should it.

    I should mention that I’ve actually spent several years doing work for an organization, using software that is, basically, a Wiki. However, there’s only about 12 of us with access, and of the 12 maybe 6 are frequent contributors… And I thought this is how they all were. We just didn’t get in each others’ way. It was quite a shock to be on Wikipedia.

    So I don’t know if I’m helping here. I would say that Wikipedia is definitely advancing on one side: the software that allows people to work on the same bits of data without stepping on each other in more mundane hey-where-the-hell-is-my-paragraph version control issues. And guess what… that software is maintained by a very small amount of people compared to the users, who submit their changes to them.

  11. Á½¸öÅúÆÀÒâ¼û

    Ëæ×Åά»ù°Ù¿ÆÔÚ2004ÄêµÄÓ°Ïì³ÖÐøÀ©´ó£¬¹«ÖÚ¿ªÊ¼¸ü¶àµØ¹Ø×¢Õâ¸ö¼Æ»®¡£ÔÚÐí¶àÈËÃǸøÓèÈÏͬµÄͬʱ£¬Ò²ÓÐһЩÈËÌá³öÁËËûÃÇÅúÆÀµÄÒâ¼û¡£Î¬»ù°Ù¿ÆÊÇÒ»¸öÐÂÉúµÄÊÂÎ¶Ì¶ÌµÄÈý¡¢ËÄÄ꣬ÕâÖÖȺÌå´´×÷µÄ·½Ê½Õ¹ÏÖ³öÁËËüµÄÉúÃüÁ¦¡£µ«Î¬»ù°Ù¿ÆÏÂÒ»²½¾¿¾¹ÔõôҪ³É³¤ÎªÒ»¸öÔõÑùµÄÊÂÎ

  12. Wikipedia’s Reality Check

    The responses from some corners of the Wikipedia community were predictable when Larry Stanger, part of the team that developed Wikipedia, published a long article on kuro5shin arguing that “anti-elitism” and “trolls” were undermining this user-contrib…

  13. EVALS TEN says:

    草根、立場、烏托邦

    Yahoo知識+剛上線測試時,Schee曾說”blogger們可以趁beta階段,一舉大幅攻克奇摩知識頻道”。 過了一個半月,不知道有沒有哪位blogger成為知識名人?或是有哪位知識名人有在寫blog? 我覺得除了Ri…