Wednesday, November 07, 2012

Statistics will fix everything

I'm a little confused as to the reason for all this recrimination about bad election predictions, as though it was bad predictions and not bad campaigning or ineffective persuasion that lost Republicans the race. The professionally disappointed Conor Friedorsdorf, for example, claims that his party has once again let him down (can we ever hope to live up to your standards, O Great Conor?) by not hiring "the most rigorous forecaster it could find," like the NYT did. But since only one party can win a race, if both sides hired equally rigorous forecasters, nothing would change except that one party would be informed several months in advance that polling indicates that it will lose. What is it to do then? Withdraw pre-emptively because The Opinion Polls Have Spoken?

Conor evidently thinks so, or at least something like that, when he suggests that conservative media has to be "honest" with "the rank-and-file" (the jus' plain folks who exist to follow the pundits). What would such honesty consist in? The National Review will announce in September, "Look guys, it's over. We lost before you even cast a ballot. We now have this science of calling up random people and soliciting their extremely incoherent opinions, and it's frankly way more effective than all this voting and franchise stuff because it tells us how you will vote without your even having to bother doing it. So why don't you just forget about all this politics stuff and focus on going to work and raising kids, ok?" 

I understand how accurate prediction is better than inaccurate prediction, and how it's also highly statistically probable that predictions based on models will be accurate more often than predictions made from the gut or from anecdotes, but whom does that surprise? What I don't understand is how the accuracy of predictions determines the effectiveness of a campaign, or how it replaces the purpose for which political journalism exists - to persuade. That polls show a candidate up by 2 points doesn't tell me anything about whether I should vote for him unless my sole criterion is to vote for the winner. Polling data can obviously be useful in planning a campaign strategy, but even armed with all the best public opinion models in the world, you only know what the average person who doesn't think of you claims he thinks of you, and not what to tell him to get him to think better of you. 

Even Conor concedes that the "misinformation" of the "rank-and-file" wasn't all about the numbers. Something was off about the message too, but not that it failed to persuade, because in the world of numbers and accuracy, there is no persuasion, only an "information disadvantage." And how would a Conor-led defeatist and cringing conservative press improve things? Well, for one thing, it would be liberal. Because the big problem with conservatives is that they're not liberal, and so they do completely incomprehensible things like "waste time on stories the rest of America dismissed as nonsense." Totally - all that nonsense about the HHS mandate and religious liberty, and entitlements spending and health care, and blah blah Libya attack. Who cares about that? Maybe conservatives, but probably not "the rest of America," whose own partisan outlets never engage in less-than-Conorable fear-mongering about the "War on Women" or anything like that. Noted: the War on Women is real, but war in Libya is a delusion. 

But oddly, Conor is not comparing the National Review to Jezebel or Mother Jones, or conservative pundits to liberal ones, but rather NR's conservative pundits to the New York Times, a nominally nonpartisan international newspaper with millions of subscribers and hundreds of reporters all over the world (and not its opinion section either). Why doesn't the National Review deliver the same kind and amount of "information" as the New York Times? Some people say statistics are hard to understand, but I would argue that this discrepancy is much more mysterious.


Alpheus said...

To play a note, I've sounded before: I think this is really all about self-image and the modern culture of narcissism. It's not fun to feel that one has been inside a bubble, drinking the kool-aid (and mixing metaphors). For some conservatives, being shown up as objectively wrong hurts as much as having lost a chance to right the ship of state.

(Meanwhile, those perceptions that are questionable but not strictly falsifiable -- e.g., the notion that people vote Democrat mainly because they want free stuff -- are much less likely to be a source of anxiety to conservatives. This is where a more persuasive, less histrionic version of Friedersdorf would come in handy.)

Miss Self-Important said...

Well, sure, conservatives need better arguments, and part of getting better arguments is understanding one's opponents better. But these arguments aren't contained in election forecasts. And the better argument is not necessarily acceptance or parroting of the opposition's positions because all your liberal friends happen to hold them, and they seem like nice people. Conor seems to be perpetually perplexed about the persistence of opposition in politics, as though if we could only think things through calmly and rationally, that unpleasant element would just disappear.

Personally, I think the right could benefit from more traditional (rather than opinion) journalists interested in political exposes and writers like Matt Labash who are willing to extensively research extremely boring political and agency activities like environmental management and emergency preparedness, or to attend off-the-beaten-path political events and talk to crazy or otherwise unsavory people in order to bring to light in easily digestible ways the mundane corruption and silliness of the left. That's a kind of empiricism, of a non-numerical variety.

Andrew Stevens said...

The sort of wishful thinking Mr. Friedersdorf decries is common in all close elections on both the right and the left. I went into both 2000 and 2004 reasonably convinced by the data that Bush would win, but every Democrat I met was convinced that Gore or Kerry was poised to pull it off. In 2000, it turned out they were very close to right. In 2004, not so much. This didn't stop the left from entertaining fantasies about how Bush must have stolen the election through some sort of vote suppression in Ohio or whatever, even though all the polls (except Gallup, curiously enough) had consistently shown a Bush lead in Ohio and the RCP average margin there was higher for Bush in '04 than it was for Obama in '12. I have heard very little from the right alleging that Obama stole the election, so it seems quite unfair to say that they are uniquely prone to such fantasizing.

There were partisan pundits on the right who agreed that Obama was a pretty heavy favorite.

Miss Self-Important said...

Yes, but as wishful thinking, isn't it limited to wishing you were right in your predictions, not thinking that somehow if your predictions were more accurate, your side would've actually won the election? I'm still completely failing to see how naming the correct "heavy favorite" is an important task of partisan pundits. Even if everyone at every conservative media outlet accepted Nate Silver's predictions, they and their audience would gain nothing from being told that, according to the polls, Romney is doomed. That would just depress turnout and make the predictions self-fulfilling. I just don't understand why all these conservatives are so distraught over the inaccuracy over their predictions, and not about why the party actually lost those races. Do they not understand what an election forecast is and how it differs from an actual election, or am I missing some key piece of this logic?

As far as accusations of stolen elections go, recount demands seem inevitable in any really close election. But the closeness of the election is not, again, determined by pre-election polling data, but by actual vote counts. Or, potentially, this accusation gains traction if the popular and electoral vote outcomes are opposed, but that's also not a matter of prediction. Republicans aren't challenging Obama's victory b/c there's nothing in it - even if they recounted almost all the closer swing states (FL, OH, VA, CO), there still wouldn't be enough votes for Romney to win. That doesn't seem to me to be connected with bearing out any polling-induced pre-election delusions.

Andrew Stevens said...

Sorry, at no point was I arguing against your main point contra Friedersdorf, where I agree with you and not him. I was merely pointing out that those on the left who had such a touching faith in the polls this year had no such faith in 2004 when the polls were against them and didn't even have faith in the vote counts, even though those vote counts agreed with the polls (which, it seems to me, is pretty good evidence that there was no fraud or suppression in Ohio that year).

Miss Self-Important said...

Oh, sure. My question is narrower than the poll-vote count correspondence. It's true that Republicans are not claiming fraud based on persistent belief in the polling of their guts. That's fine. The Dems may be hypocrites for forgetting their own errors of 2004, but perhaps they can just say that king Nate Silver had not yet ascended to his throne.

Andrew Stevens said...

Not saying they're hypocrites, necessarily. Just correcting Friedersdorf's and others' implicit (or explicit) argument that such delusions are unique to the right. Wishful thinking like that is a human condition which nobody is immune to.

Anonymous said...

If rank and file conservatives had a more accurate view of the potential electorate, the real strengths of the Obama machine (amply displayed in 2008), and the advantages of incumbancy, maybe there would have been less pressure on candidates to tack so far right during the primary. In 2000, I think there were plenty of conservatives who understood that Bush's rhetoric of "compassionate conservatism" was necessary in order to win the election. A republican base in 2012 convinced that the country was ready to toss out Obama didn't give its candidates nearly as much leeway. Romney didn't seem to feel he could move to the center until after the convention. I lots of people were shocked and delighted at the Romney they saw in the debates; it might have helped if Romney had been able to roll that persona out earlier.

There's also the related problem of the campaign buying into its own spin and the spin of the conservative commendtators. I wonder if a more realistic appreciation of the makeup of the potential electorate and their position in various states would have led to a better use of resources and more thought given to the messages needed to appeal to groups of voters. If the persistent but slight gap in Ohio that Silver and the better pollsters had long recognized had been understood by the Romney campaign, maybe they would have spent more time testing and focus grouping messages and rhetoric that would appeal to blue collar workers in the Midwest. On the tactical side, why was Paul Ryan visiting Minnesota toward the end of the campaign? Why was so much time spent in Pennsylvania?

Andrew Stevens said...

Anonymous: I agree with you that the right did not take seriously how hard it was going to be to beat Obama, especially during the primaries, but this doesn't really have anything to do with data. The left made the exact same mistake in 2004.

Minnesota and Pennsylvania were clearly Hail Marys by a campaign that found itself stymied in Ohio. They figured if they went to close states which had not received the anti-Romney media barrage of the summer, they might be able to break through. Indeed, it was that move which convinced me that the Romney campaign knew it was in serious trouble, since there was no possible way they could have believed that they had put Ohio away.

Miss Self-Important said...

Anonymous: Well, it's certainly a problem if the campaigns themselves believe the pundits' hot air and follow their advice, but it was my impression that they have more information than either the pundits or the rank-and-file about polling trends and also the specific constituent views in swing states, whereas the rest of us have only RCP and Silver to look to. If that's not the case, then bad on them, I agree. But Conor's complaint didn't seem to be that the campaigns failed to appreciate reality, but rather that the pundits did. There I think the argument is much weaker. The pundits aren't non-partisan and a good part of their job is to persuade voters in a way that augments and goes beyond individual campaigns. Doom-saying and deference to polling data is not a good means of persuasion.

About the primaries though, isn't this the complaint in EVERY primary since the dawn of time/popular primaries? This is the median voter theorum in action - since the party base is more conservative/liberal than the country at large, primaries pull candidates away from the center, general elections pull them back to it. My impression was that Romney was the most moderate option even during the primaries, and he won the nomination largely because he seemed to be the most likely of the bunch to win a general election against Obama.

I don't know if the 2000 comparison is entirely apt. You may be right that Bush appeared centrist in the primaries (I don't really remember; I was 15 at the time), but he was also running against McCain, another very credible moderate. This time around, the field was much more shall we say diverse, and the moderate slot was filled by Hunstman, who was not nearly as serious a candidate as McCain in 2000.

Anonymous said...

I'm thinking in particular of Romney deciding to run to the right of Rick Perry on immigration (this was before Perry imploded). Yes, the dynamic of running to the right or left and then swinging back to the center happens every election. However, I think Romney was particularly damaged by the dynamic this time around. He had to go pretty far to the right and couldn't swing back to the center for a long time.

As for the campaign and the pundits, some of the post-mortem articles that have gone up in the last few days suggest that Romney's campaign really was surprised by what happened. He hadn't written a concession speech and they completely expected to win. Once article even said that the Pennsylvania campaign was an attempt to broaden the victory, not a desparate attempt to find another non-Ohio path to victory. I think the pundits, the in-house pollsters, and the campaign leadership suffered from all living in a conservative media bubble. The Romney pollsters were adjusting their data based on an incorrect assumption about the racial and party-id composition of the electorate. Why did they feel so sure that blacks and democrats weren't going to show up despite what people said when reached by phone? Perhaps it was because they were invested in an overall narrative of the election in which Obama was a sort of second coming of Jimmy Carter.

There is a left-wing media, of course, but it isn't as seperated from fact-based journalism and centerist opinion as is the case on the right. (The NY Times has a bias, but is also much more commited to factual journalism than Fox or the Washington Times. MoJo is very left-wing, but actually uses investigative journalists.) When Obama made mistakes, like completely failing in the first debate, his campaign knew it had happened. They accepted the drop they saw in the polls and the uniform criticism coming from left and center pundits. They responded to the problem and changed the approach for the later debates.

All that being said, I'm not completely supporting Conor's position (he's too idealistic for me at times) and I understand that acting optimistic in that final stretch makes some sense so as not to demoralize one's own side.

Miss Self-Important said...

Well then yes, I agree that the campaign itself should be more realistic and Machiavellian than the pundits. But I don't think that the conservative media was treating Romney like the infallible pope of the party during this cycle generally - they were divided and lukewarm during the primaries, the Tea Party didn't like him, and it was only in the last couple months, after the VP selection, that they made a unified push for him.

And, as I said in the comment to Alpheus above, I agree that the right does need more investigative journalists (and also investigative satirists). But conservative media, since it's niche and by definition not the MSM, has an inherent disadvantage relative to papers like WaPo and the NYT. I'm not sure there is any single conservative outlet that can afford the NYT's extensive domestic and international reporting operations (notably, the NYT itself can barely cover costs). Maybe Fox News could if they did some major restructuring away from opinion and commentary? But other than that, the little magazines and journals on both sides usually operate at a loss as is. Mother Jones does some investigative stuff, but still mostly commentary and news analysis; it just costs way too much to send reporters out into the field for a long time. (And it's most recent "investigation" involved releasing footage of Romney at a dinner - not exactly deep digging.) The other difficulty conservatives face is recruiting younger writers who are willing to be paid very little, since most young people lean left, and those who want to be journalists would prefer to starve writing for, first of all, traditional newspapers, and if not that, then liberal publications. But, the latter problem is easier to overcome than the broader structural disadvantage of not having pre-existing conservative MSM outlets in the way that the WaPo and NYT are pre-existing liberal ones.