Gallup Polls: biased?

Friday :: Sep 17, 2004
Why You Should Ignore The Gallup Poll This Morning - And Maybe Other Gallup Polls As Well
This morning we awoke to the startling news that despite a flurry of different polls this week all showing a tied race, the venerable Gallup Poll, as reported widely in the media (USA Today and CNN) today, showed George W. Bush with a huge 55%-42% lead over John Kerry amongst likely voters. The same Gallup Poll showed an 8-point lead for Bush amongst registered voters (52%-44%). Before you get discouraged by these results, you should be more upset that Gallup gets major media outlets to tout these polls and present a false, disappointing account of the actual state of the race. Why?

Because the Gallup Poll, despite its reputation, assumes that this November 40% of those turning out to vote will be Republicans, and only 33% will be Democrat. You read that correctly. I asked Gallup, who have been very courteous to my requests, to send me this morning their sample breakdowns by party identification for both their likely and registered voter samples they use in these national and I suspect their state polls. This is what I got back this morning:

Likely Voter Sample Party IDs – Poll of September 13-15
Reflected Bush Winning by 55%-42%

Total Sample: 767
GOP: 305 (40%)
Dem: 253 (33%)
Ind: 208 (28%)

Registered Voter Sample Party IDs – Same Poll
Reflected Bush Winning by 52%-44%

Total Sample: 1022
GOP: 381 (38%)
Dem: 336 (33%)
Ind: 298 (30%)

In both polls, Gallup oversamples greatly for the GOP, and undersamples for the Democrats. Worse yet, Gallup just confirmed for me that this is the same sampling methodology they have been using this whole election season, for all their national and state polls. Gallup says that "This (the breakdown between Reeps and Dems) was not a constant. It can differ slightly between surveys" in response to my latest email. Slightly? Does that mean that in all of these national and state polls we have seen from Gallup that they have "slightly" varied between 36%-40% GOP and 32%-36% Democrat? I already know from an email I got from Gallup earlier in the week that in their suspicious Wisconsin and Minnesota polls they seemingly oversampled for the GOP and undersampled for the Dems. For example in Wisconsin, in which they show Bush now with a healthy lead, Gallup used a sample comprised of 38% GOP and 32% Democratic likely voters. In Minnesota where Gallup shows Bush gaining a small lead, their sample reflects a composition of 36% GOP and 34% Democrat likely voters. How realistic is either breakdown in those states on Election Day?

According to John Zogby himself:

If we look at the three last Presidential elections, the spread was 34% Democrats, 34% Republicans and 33% Independents (in 1992 with Ross Perot in the race); 39% Democrats, 34% Republicans, and 27% Independents in 1996; and 39% Democrats, 35% Republicans and 26% Independents in 2000.

So the Democrats have been 39% of the voting populace in both 1996 and 2000, and the GOP has not been higher than 35% in either of those elections. Yet Gallup trumpets a poll that used a sample that shows a GOP bias of 40% amongst likely voters and 38% amongst registered voters, with a Democratic portion of the sample down to levels they haven’t been at since a strong three-way race in 1992?

Folks, unless Karl Rove can discourage the Democratic base into staying home in droves and gets the GOP to come out of the woodwork, there is no way in hell that these or any other Gallup Poll are to be taken seriously.

How likely is it that the Democrats will suffer a seven-point difference against the GOP this November or that the GOP will ever hit 40%?

Not very likely.

The real problem here is that Gallup is spreading a false impression of this race. Through its 1992 partnership with two international media outlets (CNN and USA Today), Gallup is telling voters and other media by using badly-sampled polls that the GOP and its candidates are more popular than they really are. Given that Gallup’s CEO is a GOP donor, this should not be a surprise. But it does require us to remind the media, like Susan Page of USA Today, who wrote the lead story on the poll in the morning paper, and other members of the media who cite this poll today, that it is based on a faulty sample composition of 40% GOP and 33% Democrats.

Steve Soto :: 10:46 AM :: Comments (128) :: TrackBack (49)
Other blogs commenting on this post