William Playfair, Linear Chronology, Exhibiting the Revenues, Expenditure, Debt, Price of Stocks & Bread, 1824,

On January 24, President Obama delivered a State of the Union address to Congress entitled “An America Built to Last.” The speech generally received high marks from members of both parties, and even the right-leaning elements of the pundit class have reacted quite modestly. Rather than trying to tar it with charges of “closet socialism” or “Chicago-style politics,” as they have done in the past, conservative commentators have deprecated the speech by the use of a more subtle and ambiguous smear: “Clintonian.” A typical example was conservative talk show host Joe Scarborough who, speaking on Meet the Press, said,

This was Bill Clinton's 1996-1997 speech all over again, where you take all of these small items, like school uniforms. They're all poll-tested. They're market-driven. You put them all in a speech. There's no overarching theme, and it is a great campaign speech. It's not a great governing document.

Whether or not Scarborough is right to equate the two State of the Union addresses, his justification for doing so merits deeper scrutiny. Scarborough claims that each speech was essentially a mishmash of random, generally well-liked ingredients. The problem with delivering such a public policy casserole is, apparently, that it has “no overarching theme.” Theme is often an ambiguous word, but in Scarborough’s usage it clearly has something to do with a “big idea about government,” a vision for the country and its democracy. In this essay I would like to challenge the casual, almost off-handed way in which Joe Scarborough juxtaposes genuine leadership and the use of public opinion polls. Indeed, the reason Scarborough can do that is that today many people, whether they be pundits, politicians, or everyday citizens, talk about the use of public opinion polls as if it were synonymous with cynical leadership. It has become taboo for a politician to admit that he uses polls, lest he be viewed as “saying anything to get elected.” I believe that this hostility toward polls is based largely on confusion as to what they are, and ignorance as to how they are used by politicians. In reality, a poll is simply a tool, one that is far less effective than people imagine.

Before going on to polls and polling technology, I would like to give an argument as to why the common view is wrong, that the use of public polling is not mutually exclusive with genuine, conviction-driven leadership. To stay grounded in specifics, let’s suppose that Scarborough is correct in thinking that all or most of the policy proposals in Obama’s speech were market-tested. Although this supposition is plausible, it is not possible to agree with Scarborough’s view that the State of the Union lacked an overarching theme. The theme was “teamwork” and Obama invoked it continuously, first by direct reference to the activity of a military unit, and then more subtly by invoking accomplishments that Americans have achieved in the past by concerted action and ones that they can only achieve in the future by working together. He ended the speech with the following words:

Each time I look at that flag, I’m reminded that our destiny is stitched together like those 50 stars and those 13 stripes. No one built this country on their own. This nation is great because we built it together. This nation is great because we worked as a team. This nation is great because we get each other’s backs.

While the terminology of sports is hardly the stuff of constitutions and founding documents, Obama’s remarks were consistent with the vision of government and society he has always proposed. In my view, President Obama is a communitarian, someone who believes that traditional liberal theory has naïvely understood the relationship between society and persons. Whereas adherents to classical liberalism like Ron Paul believe that society and government are produced by the free activity of independent and autonomous persons, communitarians stress that individuals are a product of the society in which they develop, and later help develop that society through their more mature activity. These different interpretations of the relationship between individual and society translate into very different theories of what government should do. Many classical liberals believe that society and government fundamentally have no goal besides protecting the autonomy of individuals. Many communitarians, and, if I am to be believed, also Obama, believe that the end of a society as a whole should mirror the end of a person: to strive for excellence, to realize its potential. By describing the nation as “a team,” Obama encourages Americans to think of themselves as a community. By talking about the destinies of all citizens being stitched together, Obama encourages them to think of their common end and purpose.

From atop the lofty intellectual heights where “Morning Joe” Scarborough sits, communitarianism may seem to be a small idea about government. Nevertheless, it is certainly a theory about government in some sense. In other words, the speech was not simply cynical election politics, in so far as cynical politics means failing to have a vision for government. If Obama’s speech was poll-tested, as most commentators agree, then a poll-tested speech is not necessarily one that fails to lead.

Free from the presumption that polls are necessarily corrupting, or duplicitous, or cynical, one can begin to examine what sort of positive uses polls can have. In fact, public opinion polling is a technology that clearly has enormous potential to advance the aims of democracy. It allows for government officials to better understand their constituents. It allows policymakers to figure out whether people would use programs they might create, how much citizens know about programs that already exist, and where constituents roughly stand on the issues. It has the potential to inform the decisions of representatives in government in a positive and substantive way. To a large extent, it has realized that potential. To cite only one example, it is well known that the census often undercounts people, for example, who do not want to talk to strangers on their doorstep, or who live in difficult to reach places. Telephone polls have been used to identify where undercounted people are, and also to estimate how many of them might be. Public opinion polls have an essential, positive role to play in our democracy.

And yet, paradoxically, public polling has also helped to foster widespread cynicism about political culture in the United States, which is itself an enormous threat to democracy. Rather than discuss the issues and the proposals of the candidates, or even their personal qualifications, most political news coverage today concerns the relative status of politicians in the latest poll. Worse, there is widespread suspicion that every word out of a politician’s mouth has been analyzed by a team of technical wizards armed with charts, tables, and questionnaires. The suspicion is that politics is really a game played between two competing teams of political operatives, all of whom know enough about politics and have done enough polls to know exactly what effects all their actions will have. Where is the space for genuine communication with our leaders when, firstly, none of our votes matter because the pollsters already know how the election will turn out, and, second, everything our leaders tell us is stuff we already know we want to hear?

While this defeatist sentiment is real, it is based on ignorance. Indeed, statistical illiteracy is profound, even among political professionals. There are a few points that everyone should have in mind when evaluating the relationship between politicians, their messaging, and polls:

1. Polling is quite expensive. In order to reach 1,000 people, which is a fairly standard size for a national poll, a polling firm may have to make 500,000 calls or more. Few people answer the phones, others refuse to take part in the survey, others are disqualified from participating because they work for the news, and still others quit before the survey is over. Most polls need to be done in a short period of time, since opinions are often fluid, meaning that a specially trained staff needs to be kept for long hours in the evening over a few days. Since polls are not commissioned on a totally predictable basis, arranging a labor pool of surveyors is not always easy. A typical computerized poll costs about $10,000 to conduct, while a live one costs about $35,000.
2. Because of high costs, few polls are actually conducted. Collectively, major news providers work with maybe 15 or 20 national pollsters, each of which seldom turns around a new multi-state poll more than once every few weeks. Added to these, a few universities sometimes run polls, and then there are also a few partisan pollsters that parties commission. Almost no one regularly uses public opinion polls outside of presidential campaigns. Even congressional races aren’t polled regularly. When such polls are commissioned, it’s usually by one of the national parties in order to decide how much financial support they should pitch in to their reelection fund.
3. A poll that is done badly is not particularly useful, and can even be counterproductive. It might lead one to think something is very popular, when in fact it's not popular or downright unpopular. Additionally, even polls that seem good may have obvious errors in them. For example, in 2004 Pew Research incorrectly declared that Bush’s post-convention bounce had no legs. It turned out that prior to asking the question “Who will you vote for in November,” they asked a series of probing questions about Bush’s National Guard service. Essentially Pew, which is one of the best pollsters around, accidentally conducted a push poll and then collected data on its effectiveness.
4. Even well-designed polls are statistically bogus. To establish the margin of error in a survey one has to assume that the sample one picks is random, or at least predictably nonrandom. For public opinion polling, this assumption is known to be incorrect. No one knows who is a likely voter, no one knows what percentage of them only have cell phones, and no one knows how to get enough African-Americans or men to pick up their phones to create an acceptable correction for the fact that these two groups are systematically under-sampled. Some may object that Nate Silver and other polling aggregators can call an election to a tenth of a percentage point. This feat is possible only because polling aggregators are effectively increasing the sample size to something far larger than what any one poll is. Also, most polling aggregators only remind you of the fact that their prediction was correct when they were, in fact, correct. They tend not to make a big deal about being off by a few percentage points.

The perception that public opinion polling is a part of every politician’s daily life is simply untrue. What could give rise to the common opinion that politicians are consistently looking at polls when almost none of them have consistent access to them? The answer probably has to do with the fact that the most visible politicians, i.e. candidates for president, are the ones who do have that access. Candidates for president are the model politicians, and the only ones most of the country pays attention to.

The problem with polling, in the popular mind, isn’t just that public opinion becomes publicly known; it’s that politicians react by picking and choosing precisely what to say based on what’s inside a poll rather than bucking public opinion. This raises the questions: How do presidential candidates react to polls? What things can they actually put to the test in a poll?

The answer is surprisingly little. It is very difficult to phrase most of the things that go into a complex document like a speech. How does one do an apples-to-apples comparison of something like “the structure of a speech”? No one is fielding, and no one has ever fielded, public opinion polls that ask a question like, “Do you think President X should start his address to the nation by thanking our veterans or by paying homage to the Founding Fathers?” It is also hard to imagine how a campaign would figure out which motif they should use to structure their speech. How do you poll people’s reaction to a concept like “teamwork” versus a concept like “common cause?” It’s simply impossible. Moreover, because a survey can only take maybe 15 or 20 minutes to conduct, the number of questions a campaign can ask in a single survey is fairly limited, around 40 questions. The longest polls are around 60. What I’m getting at is that polls can inform only a small fraction of what goes into the content of an hour-long speech.

And yet that small fraction is what really ticks people off. The “small fraction” is typified, in Joe Scarborough’s mind at least, by school uniforms. This reference to Clinton’s 1996 State of the Union speech has a lot of insider, political-junky knowledge packed into it. It stands for a theory of campaign politics practiced by Clinton’s lead pollster Mark Penn. Mark Penn proposed a theory he called “microtrends,” which suggests that changes in the way a small fraction of the electorate thinks can have big changes on the whole picture. Politics in Penn’s mind is about identifying and cobbling together coalitions of small demographic groups. For example, the “soccer mom” constituency. What Mark Penn noticed is that relatively affluent, suburban women were much less confident in their support of the Republican Party than their husbands were. In his view, one could propose policy geared to this group that was totally orthogonal to the policy discussion of the day, perhaps thereby winning their vote.

To say that polling created the school uniform issue is to ignore the creativity and genius of Mark Penn and Clinton’s other advisors. It’s not like polling told them that school uniforms were the issue. Rather, it emerged from a line of questioning that polling introduced. What do suburban women care about? Their kids, and other kids too. What issues are facing their kids? Gangs, sex, drugs, and quality of education. Studies connected school uniforms with all of those hot button issues. Moreover, school uniforms were not an important subject of the national debate, so taking a stance on the issue would not arouse controversy.

Polling is not a magic hat. It does not simply give politicians and political operatives ideas of what to talk about. What it does do is tell politicians where areas of weakness and relative strength are in the electorate. And, increasingly, it is getting better at breaking apart the electorate, allowing the whole electorate to be more comprehensible as the composite of disjoint constituencies. Consider what it takes to analyze a subgroup defined by three variables, a constituency like “affluent, suburban women.” Out of a database of 500 or 1000 entries, you’d need to pull out all the women, and then from these pull out all those who live in the suburbs, and from these pull out all those whose households pull in over $100,000 per year. Prior to the 1980s, the technical sophistication required to do such database manipulations was rather high. There were not all that many people who knew how to execute a data extraction, and the ones that did were generally working in academia or the software industry. Moreover, because these computational problems are not very easy to solve efficiently, once initiated the extractions took a long time to finish. Finally, the transformation of any polling data into a readable form like a chart was also time-consuming. Powerful database software with low entry-level user requirements started becoming accessible beginning with Multiplan, a Microsoft predecessor to Excel for MS-DOS. By the early 1990s, Excel, R, and SQL were all in roughly the same state that they are today, although calculation speed has improved enormously, as has the ability to visualize datasets.

Today, it is possible for a computer to break a 1000-person poll into all possible permutations of four or even five variables and then alert pollsters to significant results, and also show these results over time. Just how many subgroups is that? Party Identification splits three ways, into Republican, Democrat, and Independent. Ideology splits three ways; gender, union membership, and gun ownership split two ways; race and church attendance split five ways; age splits into 12 categories between 18 and 80; income splits 14 ways; etc. With these variables alone, one splits the electorate into 3x3x2x2x2x5x5x12x14, or 302,400 groups. And that’s only using the traditional demographic variables. Some analysts also look at which network news shows people watch, how often they use the internet, and what media market they’re in. Making the dataset this granular can now be done by a personal computer in a few minutes.

It is easy to make too much of the increasing power of statistical analysis software. As an example of why, consider a Republican pollster who wants to see if there is any subset of the African-American population that might vote Republican in the next election. 5-10% of African-Americans are conservative Republicans, like Herman Cain or Alan Keyes. Our Republican pollster knows about them, and he isn’t curious about the African-American Republican vote. What such a pollster might do is look for “Liberal or Independent African-Americans with frequent church attendance.” His computer program will tell him that “Liberal, Devout, African-American Mormons” are very favorable toward Mitt Romney, and almost all of them can’t decide whether they will vote Republican or Democratic in the next election.

If such a pollster wrote a memo proposing that Republicans should try to target this group in the election, he would be a complete fool. A subgroup like the one I’m describing could make up 0.3% of the population, which is not as insignificant as it sounds, but a poll of 1000 people would expect to find only three individuals matching this type—perhaps two or four, but definitely no more than 10. If such a subgroup makes up 0.3% of an electorate of 130 million, the size of the entire electorate in 2008, their block constitutes 400,000 votes, which could easily be a deciding margin of the electorate in any state. In other words, a campaign that operates based on polls might have a strong motivation to go after this constituency, but they would have to figure out how to do so based on the opinions of three or four individuals.

This idea is lunatic. It does not make sense to spend precious campaign resources on the opinions of three random people who happened to pick up the phone. Probably some campaigns do actually make decisions this way, but none of them do very well. Good campaign pollsters always think about the significance of the data they’re looking at, whether it fits with their intuitions about how the electorate is working. Basically, polls are no better at determining which part of the electorate to go after than common sense. Women tend to favor Democrats, while suburbanites tend to favor Republicans and also tend to be more affluent. Obviously, “soccer moms” are between these two groups and are therefore up for contention. Computer software will not supplant common sense, although it can perhaps help pollsters come to their senses.

So far I have not identified a single essential purpose public opinion polls serve with respect to campaigns or public officials. I have argued that polls cannot help structure a speech, cannot decide which motif is best for basing a campaign around, and though they can help decide which part of the electorate is pivotal, their ability to help is quite marginal. There is one essential function polls do play, however, and that is wording. What Bill Clinton actually ended up saying in the 1996 State of the Union was, “And if it means that teenagers will stop killing each other over designer jackets, then our public schools should be able to require their students to wear school uniforms.” Which part of this line did polling play the pivotal roll in determining? If I am to be believed, it was not the idea of school uniforms. Probably, there was some discussion in the campaign as to whether Clinton should support the adoption of school uniforms everywhere or whether he should support giving schools the right to have school uniforms. Polls did not influence that decision, however; that was a question of policy. Rather, what was probably tested was the part about designer jackets and killing. What was the most appealing justification for school uniforms? Was it to decrease the prevalence of sex, of drugs, of violence, or to promote the quality of education? For each justification, they probably tested a few different ways of phrasing it. “If it means that teenagers will stop fighting each other over their designer clothes, we should support more uniforms.” The content of the proposal is always the same, it’s the tagline that changes. Do people want to “Restore America” or “Bring America Back?” Basically, polling’s essential role in political messaging is to break ties between equally plausible ways of phrasing things. It’s marketing the product; it’s not creating the product. Do we say “Just Do It” or do we just go with “Do It”? The difference in phrasing may be mostly irrelevant, but national elections are often so close that one can hardly blame the campaigners for not wanting to take chances. Of course, they are taking chances, but polling gives the sense that they are taking the right chances, however illusory that sense may be.

Polling effectively should determine nothing but the words. If it determines more, that is because campaigns do not understand the polls that they themselves commission. Obama’s speech does not show signs of the same “microtrends” approach to the electorate that Clinton’s speech did. There were gimmicky elements of the speech, for example setting up a national website for unemployed people to go to, but it is difficult to see what subset of the electorate those gimmicks were targeting. The “small bore” bits commentators noticed in the speech were merely specific examples of the general goal Obama was proposing: the creation of a more dynamic and innovative government. It is simply not true that Obama’s speech was a haphazard composition of popular policies sitting on the shelf.

I would propose that the true problem with Obama’s speech was not a product of polling, but rather of the premise that underlies bad campaign polling in the first place. The premise is that voters are essentially consumers and that a campaign can and should appeal to voter preferences in the same way that companies appeal to customer preferences. Truly good leaders in a democracy have the ability to change voters’ beliefs. They help improve the community by challenging what voters think and winning them over to their viewpoints. No politician in the last 20 years, besides the fictional Jed Bartlett and the fanciful Ron Paul, has doubled down on the position that American voters believe the wrong things. Political courage is taking on the voters who are already convinced, the ones who have made up their minds long before the election. If there was evidence in Obama’s speech that he did lack political courage, it was the fact that he so unabashedly advocated Third Way politics, presenting proposals that have bipartisan support and avoiding fights with the opposition. It should be pointed out that Obama has always advocated for Third Way politics, ever since 2004. The only part of Obama’s politics that has ever been truly courageous, the only part that has really challenged voters’ beliefs, is his own identity. No one thought Americans would elect an African-American intellectual with no substantive executive experience to the highest office in the land. Aside from that, Obama promised to break the gridlock in Washington by putting aside ideology and get things done. Which is to say, he promised to produce for the voting customer. I think it is fair to say that he has done that, achieving far more in his first term since any president since Johnson.

At the same time, one has to point out that for all Obama’s communitarianism, he has not lived up to the goal communitarianism would suggest for its leaders: teaching. Obama has not helped address our consumption-driven model of voting, the essential reason for our day’s political cynicism. Indeed, he has launched into it with abandon. True audacity would involve supporting proposals that have a chance of really addressing the gravest issues facing our nation, despite their unpopularity. Raising the retirement age for people under 40, for example, or putting a tax on gasoline. True audacity would involve telling voters that they can’t have what they want because what they want is unreasonable. Moreover, he should really stand on liberal Democratic convictions: that tax cuts do not always create growth, that oil exploration has an insignificant impact on the national economy, and that if one really wants to save human lives one should give women the right to choose. He should stand up for unpopular positions and positions that Republican-leaning voters do not agree with because they are correct. True political courage would aim at the heart of the problem, that winning elections today is about packaging words and not changing people’s ideas.

---