The real problem with polls is not their accuracy – it’s a misunderstanding of their purpose

Yet again this election cycle, polling and the polls are a major controversy. The issue is that the polls are all over the map or, again, could simply be wrong, as allegedly they were both in 2016 and 2020 and even in 2022 during the midterm elections. The real problem with the polls is not their accuracy. Instead, it is a misunderstanding of the purpose of polls and the problem of profit-driven polling.

Recent national polls, as reported on Real Clear Politics, especially during those weeks after the Sept. 10 Trump-Harris debate, seem to be all over the place. CNN had Harris up by one point. Rasmussen had Trump up by two. Morning Consult had Harris up by five. This has led some to conclude that this year will again be a mess for polling.

The problem with polling lies both in misunderstanding what polls are meant to do and in the motives for the polling. First, remember that polls are snapshots in time; they are not predictors. Polling is not some type of model that inexorably declares or states what will happen on Nov. 5. Polls merely tell us on any given day what some individuals think about some subject, such as who they are likely to vote for for president.

Many black swans, October surprises or unknown unknowns have already happened in the 2024 race, and many more could still occur, thereby impacting the final decisions of voters regarding whether they will vote and for whom. Ascertaining who is likely to vote, which is critical to polling, is not easily predictable and subject to some guesses and some polls and pollsters are better or worse at doing that. That is the second point to remember. Some polls are more accurate and some have more biases or inaccuracies over time. Casting all polls as of equal value is inappropriate, and one needs to think about good versus bad.

A third issue is interpreting the margin of error. Most polls indicate a specific number in terms of polling results, such as the recent New York Post survey indicating that among likely voters Harris has a 51% to 47% lead over Trump, with a margin of error of plus or minus 3.08%. This is an average margin of error. But with it, Harris could actually have as much as a 7-point lead over Trump — or less than 1 point. But for many polls, these margins seem to range from 4 to 5 points. In part, the margins of error reflect many polls using small samples to reach their conclusions. But to say that somebody has a 1- or 2-point lead, according to a poll, with a margin of error of three to four points tells us very little. It could be that one candidate has a larger lead or a smaller lead than thought, or that with such a margin of error, the other person could be winning,

Deciding who is ahead or behind based on one poll is insufficient. It fails to provide evidence of trends. Even if more than one poll is used but the results in them are both within margins of error, it still may not be enough to establish a trend.

Polls also have confidence levels. Confidence levels refer to the issue of accuracy and sampling certainty. These are questions regarding from a mathematical or statistical perspective how likely a sample of respondents might mirror a larger population. Most standard polls have a confidence level of .05, or 95% certainty. This means that even on the best days, there is a one in 20 chance that the poll will just be wrong. But sometimes polls, to save money, reduce the sample size of those surveyed, thereby reducing the confidence level.

There is then another problem where some websites or aggregators average the different polls to give some type of composite number with the belief that their average is more accurate. Statistically, this is not sound practice. Such composites average good and bad polls together, with different methodologies, dates and questions. One cannot really average them together.

Finally, when it comes to polling, especially national polls for the presidency, ignore them all. We do not elect presidents by national popular vote, and national polls do not tell us anything about what is going to happen in the six or seven swing states that will decide the election. Here it is 150,000-200,000 voters who would be decisive, and polling cannot be done easily at this level of granularity.

But beyond all of these methodological misinterpretations of polling, there is a bigger problem — profit driven polling. It is the habit of some organizations to do repeated polling to make their polls the new stories of the day, as opposed to covering the campaigns or examining the public policy issues that the candidates are espousing. Profit-driven polling is meant to create a horse race and to focus on who is ahead or who is behind.

Profit-driven polling is not about providing accurate reporting of public opinion, but about making money or, in some cases, about organizations releasing polls to confuse or impact public opinion. It is possible that the misunderstandings among many journalists or websites regarding polls is simply a consequence of what polling can and cannot do. But it is also possible that all of this misunderstanding is more intentional in terms of organizations seeking to maximize profits from polling.

David Schultz is a distinguished professor of political science at Hamline University. His most recent books are Presidential Swing States and Generational Politics in the United States.

The post The real problem with polls is not their accuracy – it’s a misunderstanding of their purpose appeared first on MinnPost.