Free the Encyclopedia!

As I work through the list of ten things that will be free, the order that I go in has no special meaning. Even so, it should not be surprising that the first thing I’ll discuss is the encyclopedia, since I’m best known as “the Wikipedia guy”.

“Imagine a world in which every single person is given free access to the sum of all human knowledge. That’s what we’re doing.” This is the Wikipedia mission statement.

The goal of Wikipedia (and the core goal of the Wikimedia Foundation) is to create and provide a freely licensed and high quality encyclopedia to every single person on the planet in his or her own language.

What does this mean, operationally? How can we define success?

“Freely licensed” is easy: free means that people can copy our work, redistribute it, modify it, and redistribute modified versions, commercially or non-commercially.

“High quality” means Britannica or better quality. Contrary to what some might suppose, Wikipedia is not intentionally or primarily an anarchy, despite our radical production methods. Wikipedia is a community, and a community, which is at the core, committed to _getting it right_. While many criticisms of Wikipedia’s current quality may be valid, others are clearly absurd (“public toilet” or “anti-elitist” in particular are not serious objections).

“Every single person on the planet” means exactly that — and so implies that we are involved in something that reaches beyond just wealthy western nations with broadband Internet access, but rather an effort that reaches languages used primarily by people who have no decent access to knowledge at all currently.

So, how are we doing? What are the odds of this goal being accomplished in the next 20 years?

First, it can be argued that although much work remains to be done in many areas, if you speak English, German, French, or Japanese, and have broadband Internet access, you have your encyclopedia. Each of those 4 languages has more than 100,000 articles and provides a reasonably comprehensive resource. Several other languages will pass the 100,000 threshold soon enough, and in 5 years time, all of these and many more will be larger than 250,000 articles.

Second, clearly there is a lot of work to be done in finding ways to actually distribute the work we have done already into areas where people can use it. Many people would be able to make positive use of English, French, or Spanish Wikipedia (for example) if only they had access to it.

Third, while it is important to provide our work in important global or “colonial” languages, we also think it is extremely important to provide our work in languages that people speak natively, at home. (Swahili, Hindi, etc.)

I will define a reasonable degree of success as follows, while recognizing that it does leave out a handful of people around the world who only speak rare languages: this problem will be solved when Wikipedia versions with at least 250,000 articles exists in every language which has at least 1,000,000 speakers and significant efforts exist for even very small languages. There are many local languages which are spoken by people who also speak a more common international language — both facts are relevant.

I predict this will be completed in 15 years. With a 250,000-article cutoff, English and German are both past the threshold. Japanese and French will be there in a year. Several other languages will be there in two years.

The encyclopedia will be free.

This entry was posted in guest post. Bookmark the permalink.

24 Responses to Free the Encyclopedia!

  1. Jason Scott says:

    Hello, my snuggly Wikiperson. Here’s my two essays on your project:

    The Great Failure of Wikipedia


    Much love from the dustbin,
    Jason Scott

  2. three blind mice says:

    from Jason Scott’s first link (am interesting read btw – despite the self-link):

    The story of the swastika’s entry continues after this, for over 1,200 edits. Dozens of people are involved, lots of facts are lost, many are gained… and you would be hard, hard-pressed to show why many of these folks should be editing the Swastika entry in the first place. Calling this “open source” and comparing it to programming projects is borderline insane: open-source programming projects have a core team with goals in mind that they state clearly, who then decide what gets in and what does not get in. Sometimes this works, and sometimes it does not, but people with anonymous IPs can’t just come in and fundamentally redo the graphics code on the program and then disappear, never to be seen again.

    well, jason scott, as napoleon complained “l’histoire est une suite de mensonges sur lesquels on est d’accord.”

    one valuable contribution that the “open source” editing process produces – that which is not found in traditional sources of fact or history – is the soft information of disagreement. that there have been 1200 edits to the entry for swastika – and a heated debate about what it really means – is itself an important fact. it means that the entry must be read with suspicion and a critical eye. it means that the definition is NOT accepted fact.

    in contrast, if you go to the entry for “triangle,” you will find about 30 edits – many of which complement (and compliment) the original. thus one can accept this entry with a high level of confidence.

  3. Rob Myers says:

    It is useful to contrast the success of the Free, Open, community-based Wikipedia with the failure of the proprietary, walled, commercial (I used to work upstairs from those guys, for the same company).

    h2g2 actively fought community organisation. Wikipedia is based on it.

    (And I love the mice’s soft information argument. It turns a key criticism of Wikipedia into a key strength.)

  4. Cyrus says:

    Wikipedia is not intentionally or primarily an anarchy, despite our radical production methods. Wikipedia is a community […]

    I’m always confused by when people say this. Last I checked there is no conflict between anarchy and community. You should read wikipedia’s articles on the subject. I think you’ll find you are more anarchist than you’d ever thought.

    This, in fact, seems to be Jason Scott’s main problem with wikipedia: the conflicting views of the large and primarily democratic group of wikiheads leads to article thrashing. Three blind mice (above) appropriatly addresses this criticism. It is true that some of people who participate in cut and run edits are malicous or “non-experts,” but they are the minority in almost all cases. Instead of complaining about such editors we need to learn to take our information with a grain of salt, if it comes from wikipedia or msnbc.

  5. Diana Hsieh says:

    Jimmy, I was hoping that you’d explain *why* a free encyclopedia is a worthwhile goal at all, given the countless hours of manpower required to create and maintain it.

    The mission is: “Imagine a world in which every single person is given free access to the sum of all human knowledge. That’s what we’re doing.”

    (Actually, that’s impossible. Wikipedia will never include what I ate for breakfast this morning, although I know it. Nor should it. An encyclopedia should be selective. But nevermind that…)

    Here’s my longstanding question: Why is free knowledge good? Is there something wrong with having to pay for information that’s of value to you? If so, what? If not, why is Wikipedia so great? Frankly, I think that to treat free-knowledge-for-all as self-evidently good is to presuppose an altruistic or collectivist standard of the good, particularly given the editing possible in a wiki. And surely that’s not something that you ought to do.

    In short: What’s the egoistic defense of free knowledge?


  6. three blind mice says:

    And I love the mice’s soft information argument. It turns a key criticism of Wikipedia into a key strength.

    Rob Meyers! we are in agreement!

    *mice suffer from temporary shock.*

    *then get warm and fuzzy feeling.*

    in digital communications a soft decoder keeps a track of the estimated received signal (one or minus one) and a soft metric indicating the level of confidence in the estimation. perhaps wikipedia entries might include a visual indicator of soft information – a measure of confidence in the definition.

    it is a valid criticism that the wikipedia is not authoritative, but certainly many of the entries are. making it more authorative is not inconsistent with the basic objectives and it seems (to us) like a reasonable goal.

    a measure of soft information could be done by an editorial staff, automatically by applying AI algorithms to the edit history, by asking users to vote, or some combination thereof.

    in addition to increasing the value to the user, soft information might also be used as feedback into the edit process. edits made to entries having a high confidence might be subjected to further scrutiny before being added.

  7. 데이빛 says:

    One other thing Wikipedia has accomplished has been to give constructed languages a shot of adrenalin. The smaller ones almost died in the early 90s when the internet was 90% English. It’s quite interesting watching the Esperanto, Ido and Interlingua encyclopedias develop.

    There are also some places like Korea that have a high level of internet penetration (70% or so, all broadband) but a small Wikipedia. There are some similar sites on the internet here in Korea that aren’t quite like Wikipedia but somewhat similar – one is an archived question and answer forum that has a few million questions to search through. Kind of like a cross between a bulletin board and Wikipedia.

  8. Noel Bush says:

    Just a note: All posts from the Lessig Blog appear to be from “Lessig Blog” in an RSS reader (I am using Thunderbird). And since your name doesn’t appear anywhere in the text, I have no way of knowing who is being referred to in the first person, unless I open the web page for the post.

    Sorry to distract from the content of the actual discussion.

  9. Dan Miller says:

    “Why is free knowledge good?”
    –Well, for the same reason that free anything is good. When something is available for a lower cost, all else being equal, society’s well-being is improved (at least from an economic perspective). Essentially, what Wikipedia does is approach every person on the planet (eventually) and offer them a free encyclopedia. Why not take it? I’d rather have something than nothing, as long as I don’t have to pay for it.

  10. Bill Korner says:

    Wikipedia is one of the top five greatest inventions that the internet (and its creator Al Gore) have produced.

  11. chad says:

    If I could play a little devil’s advocate here…

    “I’d rather have something than nothing, as long as I don’t have to pay for it.”

    Here is my cold, it is freely given to you. You do not currently have a cold, so you are clearly in need of one. My point being, Dan, that you have not address why knowledge is good. Disregarding the subjectivity of ‘good’, knowledge is neither good nor bad, it’s what you do with it that counts.

  12. nate says:

    Diana Hsieh asks “What’s the egoistic defense of free knowledge?”. I’m not sure why ‘egoistic’ is there, but I think this is a great question. What exactly would be the benefits of a world where every inhabitant had access to “the sum of all human knowledge”?

    Personally, it is something I believe with an almost religious fervor, and because of that fervor I’m always astounded when anyone expresses doubt. But when I try to justify my belief, I end up in a referential circle: free knowledge is good, because more people will have access to that knowledge, and thus will more efficiently create knowledge that can be passed on to others. It’s one of the few true faiths I hold that knowledge is good and ignorance bad. For me, it is a purpose of life question, and I have found no other answer.

    Is there something wrong with having to pay for information that’s of value to you? Perhaps not if you can and will, but yes, there is something deeply wrong about being in need of knowledge that one can not afford to buy, knowledge that would better your life, increase your productivity, prevent your children from dying from some trivial disease. Or even if the knowledge isn’t strictly necessary, I still feel there is something tawdry about making a decision that remaining in ignorance is the rational economic choice.

    But as I said, I’m clouded by religous fervor. So yes, Mr. Wales, I too would love to see your answer to this question. Why is universal access to a high-quality free encyclopedia a worthwhile goal?

  13. Dan Miller says:

    I should have been more clear in my terminology, then. First of all, we should be clear–Wikipedia’s goal is not to distribute “knowledge”, as such; it is to distribute an encyclopedia (a subset of knowledge). An encyclopedia is a good, in the economic sense–that is, it’s something that people are willing to pay for. This is what differentiates it from your cold, which people are willing to pay to avoid.

    Therefore, the practical impact of Wikipedia is to lower the price of an economic good (close to zero, though there are still time costs). Something that previously, you would have paid for, is now free. In essence, the reason that knowledge is good (as opposed to colds) is that people are willing to pay for it, and now they don’t have to.

  14. three blind mice says:

    Here’s my longstanding question: Why is free knowledge good? Is there something wrong with having to pay for information that’s of value to you? If so, what? If not, why is Wikipedia so great? Frankly, I think that to treat free-knowledge-for-all as self-evidently good is to presuppose an altruistic or collectivist standard of the good, particularly given the editing possible in a wiki. And surely that’s not something that you ought to do.

    Diana Hsieh the three blind mice are as wary of “teh free” almost as much as we are wary of the farmer’s wife and her carving knife.

    from our rodent perspective, the wikipedia is not about replacing the “paid for” encyclopaedia – it is something completely different.

    for example, we were writing a column for a magazine the other day and we wanted to use the phrase “all your base are belong to us” (don’t ask why). now no self-respecting dictionary would have a definition of this, and no encyclopedia, but sure enough, there was an exhaustive entry in the wikipedia.

    that’s one reason why the wikipedia is so great. it is full of the crazy, off-the-wall trivia of popular culture that never seems to find its way into “serious” publications. check out the entries for “cornholio” or “dead parrot”… can you imagine finding these bits of silliness in encyclopaedia brittannica?

    “free” as in no cost or “free” as in libre seem irrelevant. some people love ivory tower proselytizing about economic costs of distribution, blah, blah, blah, but the fact of the matter is that these seem to be far more operational imperatives than grand statements of social or economic principle. from the pedestrian point of view managing the copyrights, paying the contributors, and billing for use would make it an unwieldy business – and take all the fun out of it.
    like open source programming as long as it is fun and people are motivated by that you can do away with the rest – and it makes some sense to do so.

    whaddya say mr. wales? free as in libre: social perogative or operational imperative?

  15. Marc Perkel says:

    Wikipedia has a group of people who have a strong bias against Atheism. Any time an Atheist posts there they do everything they can to block the losting.

  16. rsailer says:

    I think the next interesting (and probably difficult) project (as if the Wikipedia isn’t interesting enough) will be the integration of the different language versions via some form of translation.

    For example, though I haven’t tested it, I suspect the Japanese entry about WWII and the US version of the same have got to be pretty different (although both reasonably accurate from their own point of view). The combo would present a more comprehensive entry than either of the individual entries.

  17. Jimbo Wales says:

    Quoting me:
    >The mission is: “Imagine a world in which every >single person is given free access to the sum of >all human knowledge. That’s what we’re doing.”

    Diana wrote:
    >(Actually, that’s impossible. Wikipedia
    >will never include what I ate for
    >breakfast this morning, although I know
    >it. Nor should it. An encyclopedia
    >should be selective. But nevermind that…)

    Absolutely right! An encyclopedia absolutely should not include what you ate for breakfast. In my mission statement this is the work of the word ‘sum’… not “all human knowledge” but the “sum”. I use the word in the sense used by Rand in VS, 21: “expanding one’s knowledge into an ever-growing sum.” One’s knowledge, like the knowledge placed into an encyclopedia, is not a random collection of unintegrated facts, but a sum.

    An encyclopedia should be and must be selective. And Wikipedia, of course, is.

    Now that we have that out of the way, of course the much more interesting question is: “What’s the egoistic defense of free knowledge?”

    The short answer is that I benefit from other people having access to information. I expect to benefit from everyone in the world, no matter where they are, having the tools at hand to educate themselves out of whatever false beliefs they would otherwise be unable to overcome due to lack of information. I don’t think terrorists, for example, are born that way, but made that way by comprachicos of the mind. If we can bathe the world in free knowledge, free neutral, clear, high quality knowledge, make it ubiquitous, then we undermine those who seek control through brainwashing.

    The longer answer involves understanding the intellectual values to be gained from the sort of discipline that writing for Wikipedia imposes on the author, from an interaction with other people who are passionately committed to getting it right. Most people who work on Wikipedia find it in no way a self-sacrifice or duty, but rather a passionate expression of the joy of working with knowledge.

    So while I don’t think very many things are self-evidently of value, I think that a free encyclopedia comes pretty close.

  18. Brian Mingus says:

    Free – as in ubiquitous – may be feasible within the next 15 years. Our pals Larry and Sergey think it will take 300 years to index the world’s information. To me that sounds much more realistic. I like to think that right now we are doing all the hard work. The hard work is getting started. Getting that “Hello World!” program to compile. Absolutely, we need a vision for the future. But the sum of human knowledge is, among other things, non-teleological. You have no idea where it’s going, what form it will take, or even, aside from being a compendium itself, what it will actually be. So I focus on the hard work. This is the most important time in the life cycle of the encyclopedia. Research done by IBM has shown that even after years of editing, the theme set by the person who started an article shines through. Years from now the work we are doing will shine through. And at that point the manipulation of information will be easy. We just need to get it all down first. We’ll figure out the rest when that time comes. See you at Wikimania!

  19. Jim McCoy says:

    One other point that seems to be missed here is that the wikipedia is a collection of facts, not knowledge. A more accurate statement would be that the wikipedia will contain a large collection of facts (that are not too controversial) and summations about a large collection of categories, some topical, some trivial, and some that are interesting to the participants. No one will learn particle physics from the wikipedia, and no one’s worldview will be fundementally altered by the wikipedia (hubris and bluster from Mr. Wales notwithstanding.) [To Mr. Wales, I would note that studies have shown that even if someone committed to a particular cause or worldview is provided with what appears to a neutral observer as overwhelming evidence against their committed position the “zealot” will treat the contrarian data source as noise and discount it in favor of unsupported facts that support his/her worldview. This is how Rush Limbaugh and Al Franken keep their jobs…]

    There is more to knowledge than just having access to the facts. I have had some education in legal theory and history and have access to Lexis/Nexis, but I doubt Prof. Lessig would give up his tenured position to me since I can “obviously” do his job at a lower cost to his employer (making more money available to spend on the students and increasing the utility of this investment in a “knowledge worker”) given my limited experience but unbounded access to collections of facts.

  20. Anon says:

    Thankfully there is already an effort to create ‘high quality’ articles by fact checking and referencing them. Right now there is one reference per fact but eventually I envision each fact being verified with multiple sources.

    Check out some past articles and how they were formatted and apply it other articles.

  21. Jason Scott says:

    Three blind mice, thanks for reading the essays before commenting. That happens so rarely these days…..

    ..your conclusion, however, that the soft information of massive edits being a guidepost to knowing the accuracy or contentiousness of an article falls short because of one aspect of wikipedia: complete and utter lack of accountability.

    The fact is, in the swastika article I link, the massive amounts of edits are in many cases totally vacuous; people with no research or knowledge on the subject jumping in to make procedural changes or to remove information because they don’t think it’s right. Then others try to put it back, and then a fight ensues. Those dozens of edits over a single fact or set of facts don’t provide soft information at all; they just reveal that there were a lot of edits.

    Subsequently, an article with 30 edits that all are minor don’t reveal anything on Wikipedia other than the battle not being waged there, so far.

    Minor administrative changes, including accountability, would improve wikipedia’s efficiency greatly. But as it is, it is not worth the time of a lot of people (working on the inside).

    One of the comments made to me are that my concerns are all from the point of view of content CREATORS and EDITORS. USERS don’t care if it took Wikipedia 1000 people and 3 years to achieve what 10 people could in 2 days. But I care, hence I do not contribute work to it any longer.

  22. Rob Myers says:

    free as in libre: social perogative or operational imperative?

    “Not Stallman or Torvalds, but Stallman and Torvalds!” 😉

    Lots of hard, boring or unpleasant work is involved in the mice’s “fun” or jimbo’s “joy”. Edit wars in wikis or regressions in free software can be depressing. Why do more people not just give up and rent a movie instead?

    In my opinion, people believe in the goal of free/open projects. Not the ideological direction of their founders, but in the practical aim. They are motivated by the prospect of the satisfaction of having completed the project and by the prospect of the utility of the finished work at least as much as by any fleeting enjoyment that may come from working on it.

    But even this “fun” would not be possible without its supporting ideological framework. In my opinion, a project just designed for people to have fun would not look the same as Wikipedia or Linux.

    There’s an interesting example of a non-“Free” but open development process that nonetheless satisfied its community here:

    As for why a liberated encyclopedia is good; it supports all human endeavour equally. This is a greater good (in terms of social or economic value, or of moral coherency) than providing a subset of universal knowledge to a few, however worthy that goal may have been in the past.

  23. Joseph Pietro Riolo says:

    To Three Blind Mice:

    Wikipedia is not something different. If you enter
    “encyclopedia” in the title in’s advance
    search, you will see that there are more than
    25,000 books that have encyclopedia in their title.
    There are so many different kinds of encyclopedia
    out there. Given your staunch support for expansive
    intellectual property and non-property rights,
    why don’t you buy these books to support the authors
    so that they can include the “all your base are
    belong to us” phrase in their encyclopedia? Your
    embracing Wikipedia surprises me. It seems that you
    unknowingly and blindly are drawn to the dark side
    of Stallman that seems more deadly than the carving

    Joseph Pietro Riolo
    <[email protected]>

    Public domain notice: I put all of my expressions in this
    comment in the public domain.

  24. anon says:

    Who will free the allegedly free encyclopedia, propaganda wise?

Leave a Reply