http://www.mysterypollster.com/main/2004/11/faq_questions_a.html

My understanding:
1. Random precinct selection within each state with each precinct’s chance of selection weighted by turnout history

2. Each state is polled independently, but 1,469 precincts selected in all 50 states total. So on average 30 precints/state but Ohio and the other important states got more to increase accuracy in calling those states.

3. A separate National exit poll is conducted using a different set of 250 precincts selected randomly with weighting by turnout history.

Basic Limitations:
The margin of error of exit polls is in the range of 3% and less, but there are emerging limitations to exit polls such as a declining response rate, which is now at around 50%. Also, it is being investigated whether GOP voter non-response is greater than Democratic voter non-response and why.

Hope that helps.

To get correct exit polls, you need to get a really good sample normalized on voter population and previous voting patterns.

If you put more exit polls in the cities, then you’ll get a biased result.

-reza

Thanks.

Cranky

Thank you for the dialogue. Don’t forget that there are two other factors to the exit poll discrepancy:

3. Provisional votes and spoiled votes after a recount is completed might lower the discrepancy into the MOE.

4. Systematic errors in the widely deployed electronic tabulation machines used throughout the country. A hand recount would again ameliorate that problem.

Thanks!

The most intelligent frauds are perpetrated in ways that leave plausible explanations. For instance, instead of stealing money straight from the company’s bank account, a crooked accountant would be smarter to structure some kind of fake service agreement. The accountant creates documents agreeing to pay a consulting firm for some service, and the consulting firm doesn’t exist (and the service is never provided, but nobody ever complains), although the check is written and the accountant gets his money. There are ways to catch this kind of thing, so the accountant works on making the fake transaction seem plausible and less likely to be looked at closely.

So, I’m going to concede that if I were to commit voter fraud, I wouldn’t stuff ballot boxes with millions of votes, because I would expect that people would get suspicious when exit polls showing a different result. I would, instead, create a system meant to exploit weaknesses in exit polls so I could fly under the radar. For instance, I would create a lot of fake votes for Bush, and fewer fake votes for third-party candidates (who couldn’t win), since the election goes to whoever gets most votes, not necessarily a mojority.

OTOH, when a strange occurence can be explained plausibly or with a vast conspiracy theory, I put my money on the plausible explanation.

The only way to determine without reasonable doubt if fraud occured would be a release of the exit poll data, and access to the ballots (and voting machines). Since access to the Florida 2000 ballots was granted to several news organizations, I know it’s possible. I don’t expect to find anything, but an audit is always a good idea.

You misunderstand the categories of exit poll data. Freeman was wrong when he claims the data he used was not meant to be released. The Slate article he reference refers to the afternoon poll numbers. The numbers CNN.com posted on its website election NIGHT were released to the public, via cnn.com, and were weighted. What is true is that those election night exit polls are no longer available to the public as they were replace by corrected polls that match the election results. But don’t take my word for it. Read mysterypollster.com

The Ohio margin of error at the 95% confidence level was +/-2.87%. Kerry received 48.5% but the exit poll stated 52.1% for a difference of 3.6%. But you make a seemingly valid point that the margin of error is “double” if Bush’s gains are taken into account. It does seem an absurd oversight if true. Interesting. I’ll write back what I find out.

Thanks,

Alex

(1) After taking another look at footnote 22 (page 11), I have to concede this point.

(2) Freeman said the numbers he used “were reportedly not meant to be released directly to the public.” Why not? There are several reasons that the information might have not been meant to be released to the public. One is that it hadn’t been weighted.

He later added a section that he believes the numbers are properly weighted, based solely on his belief that NEP wouldn’t intentionally post unweighted numbers. Then again, the numbers weren’t supposed to be posted publicly at all. That isn’t proof that the numbers were properly weighted, nor is it proof that the numbers aren’t weighted. To keep this post almost short, I’ll concede this point because Freeman’s paper still has a fatal flaw in it (next item).

(3) The paper only gives the margin of error for Ohio (which was 2.2% — pg 12, footnote 23), and Bush’s 2.1% increase there falls squarely within it. I would expect Florida’s and Pennsylvania’s MOEs to be similar, because the sample size for a poll is usually chosen to get a particular margin of error. If they were, Bush’s 2.2% increase would be identical to the MOE.

Freeman’s paper complains that Kerry’s overall drop is outside the margin of error. The mistake the paper makes is that it ignores the simple fact that Bush, Nader, and Badnarick all received statistically-defensible gains (Nader getting 1% of the vote compared to less than 1% of the vote, for instance), and those gains add up to Kerry’s corresponding loss. Remember all those pre-election polls with MOEs of 3% that were “statistically tied” if the results were within 6 points? Why? Because a 3% increase for one candidate would mean a 3% drop for the other — a total difference of 6%.

I think we’ve found the culprit. Sorry, nothing to see here. Moral of the story: don’t put your money on vast conspiracy theories.

What about Pennsylvania? In Pennsylvania, the gain was 3.2 points (as was Kerry’s drop), and that is fishy. Then again, the MOE isn’t always accurate. Page 13 says that the chance the election results would be that far outside the MOE is slightly more than one-in-one-hundred. Let’s see, chance one poll is off by that much is one-in-one-hundred, and we conduct fifty polls, what is the chance that one of those polls is off by that much? The way I see it, 50%. Statistically speaking, I think we can expect this once every other election (each time in a different state).

For the record, I would like to have the raw data released as well. Then I could determine what the appropriate margins of error should be, and other people could determine if there is any evidence of true fraud. It’s a lot like honest companies submitting to audits.

I think you significantly mischaracterize exit poll methodology.
1. Precincts are chosen randomly with weighting by expected turnout.
2. Freeman does not use unweighted data. Your originally wondered if he used unweighted data and the answer is no.
3. The 2% discrepancy you cite is significantly outside the margin of error for this exit poll as Freeman’s study explains.

I agree in general with your skepticism of the accuracy of exit polls, but not with your arguments. The criticisms you offer mischaracterize exit poll data. And some of your criticism of Freeman is off the mark, because of this mischaracterization.

BTW, professionals think the answer is GOP voter non-response bias, i.e. response rate for exit polls is around 50% and GOP voters were less likely to respond than Democratic voters.

You would want to read this from Mysterypollster.com to better understand exit polling:
http://www.typepad.com/t/trackback/1471163

In fact Mysterypollster.com conducted an extensive review of the Freeman paper here:
http://www.typepad.com/t/trackback/1438364