Thoughts Aloud

Here is another beaut from a blog called Iowa Hawk (that is begging to be explored further by me). Entitled, “Balls and Urns,” it boils down statistics and calls polling for what it is:

Statisticians love balls and urns. A typical Stats 101 midterm, for example, usually includes a question along these lines:

“You take a simple random sample of 1000 balls from an urn containing 120,000,000 red and blue balls, and your sample shows 450 red balls and 550 blue balls. Construct a 95% confidence interval for the true proportion of blue balls in the urn.”

After choking back a giggle about “blue balls,” you whip out your calculator and text your frat brother who has a copy of last semester’s midterm. He instantly recognizes the correct formula is

95% confidence interval for P = p +/- 1.96 * sqrt( p*(1-p) / n) * FPC…

Then he goes into the geeky math that I used to thrive on as a kid, but just haven’t found much use for in the real world for the past 45 years, before making his case:

But what if the thing you are studying doesn’t quite fit the balls & urns template?

• What if 40% of the balls have personally chosen to live in an urn that you legally can’t stick your hand into?
• What if 50% of the balls who live in the legal urn explicitly refuse to let you select them?
• What if the balls inside the urn are constantly interacting and
  talking and arguing with each other, and can decide to change their
  color on a whim?
• What if you have to rely on the balls to report their own color, and some unknown number are probably lying to you?
• What if you’ve been hired to count balls by a company who has endorsed blue as their favorite color? 
• What if you have outsourced the urn-ball counting to part-time temp balls, most of whom happen to be blue?
• What if the balls inside the urn are listening to you counting out
  there, and it affects whether they want to be counted, and/or which
  color they want to be?

If one or more of the above statements are true, then the formula for margin of error simplifies to

Margin of Error = Who the hell knows?

Very well said indeed! He concludes with:

Because, in this case, so-called scientific “sampling error” is
completely meaningless, because it is utterly overwhelmed by
unmeasurable non-sampling error. Under these circumstances
“margin of error” is a fantasy, a numeric fiction masquerading as a
pseudo-scientific fact. If a poll reports it — even if it’s collected
“scientifically” — the pollster is guilty of aggravated bullshit in
the first degree.

The moral of this midterm for all would-be
pollsters: if you are really interested in how many of us red and blue
balls there are in this great big urn, sit back and relax until
Tuesday, and let us show our true colors.

Until then, fondle your own balls.

I couldn’t agree more! Mary would have loved this one. 😉 â—„Daveâ–º