Quinnipiac and sample volatility

Last month's Quinnipiac poll of Ohio included some Party ID numbers in their sample that seemed a little fishy.

NRO highlighted the situation this way:

...today's sample was 24 percent Republican, 33 percent Democrat, and 33 percent independent or no party, and the remaining 9 percent were other or refused to answer.

In February, Quinnipiac had it at 27 percent Republican, 30 percent Democrat, and 37 percent independent, and 4 percent other or refused to answer.

That 33/24 split among Democrats and Republicans really stands out, as the 2008 exit poll put the split in Ohio at 39 percent Democrat, 31 percent Republican. In other words, does Quinnipiac really think that the makeup of the electorate will be better for Democrats on Election Day 2010 than it was in 2008?

Fishy findings indeed.

Well, I decided to follow up a bit with Quinnipiac and see where they stood in their most recent poll of the competitive Pennsylvania Senate race.

In their April 8th poll, Republican Pat Toomey is leading Democrat Senator Arlen Specter 46-41.

In their March 2nd poll, Specter was up 49-42.

Now take a look at the Party ID for each poll.

April 8: 34% GOP, 34% Dem, 27% Indie, 3% Other, 2% didn't know.
March 2: 29% GOP, 37% Dem, 28% Indie, 4% Other, 2% didn't know.

So based on that Party ID sample, the 13 point swing makes a bit more sense, doesn't it?

In 2008, Party ID in PA broke down this way - 44% Dem, 37% GOP, and 18% Independent.

Under the current environment, it's safe to say that margin between Parties has shrunk considerably and the Independent vote has increased.

But the point of this post is to highlight the volatility of polling. When pollsters test their sample without modeling it after the likely makeup of the electorate, it makes following the polling trends far more difficult.

For example, the story in PA after this poll is that Toomey has stormed back to take the lead. When in reality, the testing sample has simply changed considerably.

How likely is it that in one month the makeup of voters in PA changed by such a margin? I don't care how much people don't like health care reform. That many people switching Parties simply doesn't happen in one month.

If pollsters want to provide a true reflection of how a race is trending, they need to do a better job of reflecting the actual makeup of the electorate. A volatile sample over time doesn't assist in providing a true analysis of the race.

Fortunately, Quinnipiac will eventually begin testing Likely voters over Registered voters. That should provide a much more fair reflection of where things honestly stand.