Comments on: Are Crowds Wise?

By: Karl

Karl — Wed, 20 Jul 2005 07:51:16 +0000

Similarily like in Who Wants to be a Millionaire you can bias the audience into picking the completely wrong answer. If one were to announce to the audience that they have some knoweldge on the question, and that they go as far as to label the correct answer as being incorrect, then the audience is more than likely to vote for the wrong answers. Case in point, there was a question that asked which toy translates to english as "play well" from its native language. The contestant said "It couldn't be Lego." So when he polled the audience everyone chose the the wrong answers, and a very small percentage of individuals still voted for Lego.

So the minority that still voted for Lego can either fit into two categories:
A.) They knew the answer.
B.) They didn't know it was the answer, but they voted for Lego anyway because they couldn't trust the contestant said.

It is obvious though, that any question can be loaded to provide a majority consent, even if it were a false one. It's just part of the rhetorical schemantics in our language.

By: Fred

Fred — Wed, 20 Jul 2005 01:49:11 +0000

This kind of information is useful in playing Who Wants To Be A Millionaire. You can ask for a poll of the audience in their multiple-choice questions. If you are completely clueless, but know it to be a common category and relatively easy question, the winner in the poll is almost invariably right. If you have some knowledge in a trickier category, but are from from sure of the right answer, reading the poll requires interpretation: A clear winner is less likely, and you might see two choices equally popular--in this case, you must see if you can figure the "trick" answer and discount those who are fooled by it.

By: sennoma

sennoma — Wed, 20 Jul 2005 01:29:44 +0000

Carrick may have pointed out the fly in the ointment (better mathematical minds than mine will have to weigh in on that), but the theorem as stated by Cass suggests a built-in way to measure how good the average guess is: simply take two groups, one significantly larger than the other, and compare their answers. If the larger group has the better answer, the average agent is more than 50% likely to be right.

By: Mike Huben

Mike Huben — Tue, 19 Jul 2005 22:52:39 +0000

I haven't looked carefully to see if I'm not repeating something already mentioned, but google is an aggregator of aggregators, complete with feedback loops.

In some ways it's like a stock market measuring prices, except that the measure is based on link weighting rather than money.

Creation of new links is also biased by this measurement: for example, I'll often search for new links for my Critiques Of Libertarianism web site with google, and those that come up first are more likely to be evaluated and eventually linked. So there are positive feedback loops involved.

By: Carrick

Carrick — Tue, 19 Jul 2005 19:52:22 +0000

Cass states: Here's a problem, though. If group members are less than 50% likely to be right, the likelihood that the average will be right approaches ZERO as the size of the group increases.

Actually, this isn't a good way to describe the jelly bean scenario, though it may be applicable to other types of decision making.

The correct way to statistically describe the jelly bean scenario is to say that each person has an uncertainty "S" in their estimate of the number of jelly beans. If they don't talk to each other (and thereby influence other people's decisions), then assuming no optical illusions or other effects that could generate a systematic bias, the uncertainty of the average of the guesses of everybody in the crowd is S/sqrt(N), where N is the number of persons making independent guesses and sqrt(N) means the square root of N.

If their ability to estimate gets worse, S gets larger, and you just need more people (large N) to get an exact count of the number of jelly beans. Or put another way, if your population size is kept constant, as the individuals ability to estimate gets worse, then so does their corresponding answer.

There are other types of decision making, e.g., categorical choices like "is this color chartreuse or a lime green"? If enough subjects don't know the difference between the two colors, you can make N as large as you want and still never get the correct answer.

There is another example of collective intelligence.... social insects such as ants. These insects as a group show some pretty cool abilities, such as "remembering" where different types os food are, regulating how much brood goes to the the workers versus the alates, etc.

This all ties in, by the way, to the importance of an informed populace. As the data coming from the media gets worse, the ability of the democracy to make appropriate choices becomes more and more impaired. The emergence of a blogosphere which is divided into a left and right hemisphere, with very different social characteristics, also suggests that there will be an asymmetric advantage to the hemisphere which is able to arrive at truthful and accurate conclusions. The hemisphere that spends most of its time spinning, on the other hand, may actually further screw up the functioning of its associated party.