By Dr. Gary Welton
Election night 2016 opened with scenes of the Clinton campaign assembled at the Javits Center, positioned beneath a symbolic glass ceiling. Supporters of Hillary Clinton were energized, optimistic, and confident. The polling data offered them solid grounds for anticipating that America would elect its first female president. Yet there was one significant problem: the polls failed to predict the outcome, and the celebration never took place.
The last time polling proved this inaccurate was 1948, when Thomas Dewey was not elected. Like that earlier failure, the 2016 error stemmed not from statistical theory but from flaws in research methodology.
Sixteen battleground states were tracked by RealClear Politics, which compiled the most recent polling data for each. These polls were combined to produce an average for every state. The margin of error in polling is mathematically determined by sample size—larger samples yield smaller margins of error, while smaller samples produce larger ones. Though RealClear does not publish an overall margin of error for their state averages, this figure can be calculated using the individual sample sizes and reasonable assumptions.
According to this analysis, the Trump-Clinton gap fell within the margin of error in seven of the 16 battleground states: Arizona, Colorado, Florida, Georgia, Maine, New Hampshire, and Virginia. Clinton’s actual performance exceeded predictions in one state. Nevada presented a narrow Trump lead in the RealClear data, yet Clinton carried the state. The remaining eight polling errors all involved underestimating Trump’s support, occurring in Iowa, Michigan, Minnesota, Missouri, Ohio, Pennsylvania, North Carolina, and Wisconsin. Some of these—Missouri and Ohio—were correctly projected for Trump but by smaller margins than shown. Others like Minnesota were expected to favor Clinton but proved far more competitive. The unexpected shifts in Pennsylvania and Michigan (which remains officially uncalled) ultimately redirected the evening’s festivities from the Javits Center to Trump Tower.
Mike Murphy, a Republican political consultant, stated that “Tonight, data died.” While his observation identified a genuine forecasting failure, it implies a statistical problem. The actual issue, however, lay in the inability to obtain truly representative random samples.
The 1948 error occurred because telephone ownership was limited. Sampling only from homes with phones created a skewed picture of voter preferences, since households without phones typically had fewer resources and different voting concerns. The resulting predictions failed because the sample was not representative of all potential voters.
Some have claimed the polls were deliberately manipulated, but the Nevada error—which ran opposite to the general pattern—contradicts this theory. Pollsters were genuinely attempting to measure voter sentiment accurately. Recent elections have actually demonstrated strong polling accuracy. Using 2004 polling data, I personally predicted the correct winner in every state.
The difference lies in how we now use telephones. Historically, when a phone rang, I answered it—even with wet hands from dishwashing or greasy hands from changing oil. Years of robo-calls have changed this behavior. Now I screen calls using caller ID before deciding whether to answer. I exercise control over my phone use.
This shift makes it increasingly challenging for pollsters to construct representative random samples. Current polls better reflect the behavior of people who do not use caller ID. Those who exercise greater control over their phones might also seek more control over government and other aspects of their lives.
The 2016 polls may have been skewed, but not through deliberate media manipulation. Instead, they were inadvertently distorted by the practical difficulties behavioral scientists face when studying human behavior.
Even with improved research methods, statistics inevitably carry a margin of error. Data offer perspective on reality but, like weather forecasts, provide no certainty. Stephen King captured this truth perfectly in his novel “11/22/63,” where he writes: “There’s always a window of uncertainty.”
Dr. Gary L. Welton is assistant dean for institutional assessment, professor of psychology at Grove City College, and a contributor to The Center for Vision & Values.