Since about the beginning of President Obama’s tenure, the Gallup poll has generally been one of the least positive polls for the Democratic party. This has prompted outrage and pressure from the left–even from presidential advisor David Axelrod.
Over the summer Mark Blumenthal of Huffington Post wrote a critique of Gallup’s daily presidential job approval poll. The point of which was that Gallup was over-sampling whites and thus understating President Obama’s position in the adult population. I responded by arguing that Blumenthal’s case was underdeveloped and less-than-met-the-eye, and that was basically where things stood.
Until, that is, this week. President Obama enjoyed a bounce in his Gallup job approval number after the Democratic National Convention, as was to be expected, but there was a twist: it did not disappear. And while Gallup on average had found Obama’s job approval around 47 percent with adults through most of 2012, for the last five weeks it has been regularly above 50 percent. Yesterday, it stood at 53 percent, a number we have not really seen since 2009.
Unusual. So, what’s going on? Alan Abramowitz of Huffington Post and The Democratic Strategist noticed that Gallup has increased its share of nonwhites from 27 percent the week of the convention to 32 percent last week, a nearly 20 percent boost. In other words, Gallup seemed to have tweaked its methodology with just weeks to go until Election Day to reflect the criticism that has come from the left.
And indeed, in a wonky and elliptical statement, Gallup chief Frank Newport essentially confirms the shift:
As we began this election tracking program on Oct.1, our methodologists also recommended modifying and updating several procedures. We increased the proportion of cell phones in our tracking to 50%, meaning that we now complete interviews with 50% cell phones and 50% landlines each night. This marks a shift from our Gallup Daily tracking, which has previously been 40% cell phones. This means that our weights to various phone targets in the sample can be smaller, given that the actual percentage of cell phones and cell-phone-only respondents in the sample is higher. We have instituted some slight changes in our weighting procedures, including a weight for the density of the population area in which the respondent lives. Although all Gallup surveys are weighted consistently to census targets on demographic parameters, we believe that these improvements provide a more consistent match with weight targets.
So, from the looks of it, the left got what it wanted: Gallup altered its methodology with a month to go until Election Day. And the result – at least on the job approval question – is a shift in Obama’s favor. Whether or not this has altered the Romney-Obama head-to-head numbers among likely and registered voters, I cannot say.
I also cannot speak to the merits of the change in methodology. Back in June, I thought there was less than met the eye to Blumenthal’s critiques of Gallup. And I thought Gallup thought the same thing. Maybe the polling outlet changed its mind. Maybe it had other reasons for making the change. Who knows? That stuff is all “black box,” proprietary methodology that is not open for public analysis.
What I can say is that it’s problematic to alter one’s methodological approach to polling elections just five weeks before the biggest election in a generation. In fact, I think this is a highly inopportune time to make such a change; do it in the summer of 2012 or the winter of 2013, but for goodness sake not the fall of 2012!
It is even more problematic to make the shift but not spell out in detail the political effect of it. One utility of the Gallup tracker was that it enabled comparisons across time. Those are now difficult to accomplish because we have to assume what effect these methodological shifts have had. My guess is that it has moved the needle toward Obama by maybe 3 points on job approval, but we cannot know for sure. We also have no idea the extent to which this changes the Romney-Obama head-to-head among registered or likely voters.
What Gallup should have done is similar to what the Bureau of Labor Statistics does when it adjusts the unemployment rate to account for new Census data: Give the number as it is now calculated and as it would have been calculated absent the change, so everybody can know exactly what effect the changes in assumptions have had. Newport fails even to acknowledge whether and how this methodological change helped one side over the other, let alone its extent.
Final point: We absolutely, positively must remember polling in 2012 is politicized as never before, and it is incumbent upon the consumers of political polls not to accept the data naïvely, but to perform due diligence to see what goes into the product.
Source: The Weekly Standard
