The problem with polling America’s young voters

In the past six weeks, depending on which poll you look at, US President Joe Biden has held anything from a 28-point lead over Donald Trump with under-30s to a 21-point deficit. If realised in November, the former would represent an improved margin with young voters compared with 2020. The latter would be a stunning reversal and surely propel Trump back into the White House.

To state the obvious, one — or more likely both — of these figures does not paint an accurate picture of young people’s views today. But why are they so wildly different?

One factor is simple: opinion polls come with error margins that get wider as the demographic group you’re looking at gets smaller. Most people are familiar with the plus-or-minus three percentage point margin that comes with the typical poll of 1,000 people. When we focus on the under-30s, who sometimes number fewer than 100 respondents, that uncertainty can balloon to plus-or-minus 10 points.

But in this case we’re looking at a more than 50-point gap. The variation is also much wider among the under-30s than in older age groups, and more so than in the past. Clearly, there is more going on below the surface.

As we know from the notorious example of the Literary Digest’s 1936 poll, which confidently predicted defeat for Franklin D Roosevelt in that year’s presidential election (he won in a landslide), sample quality matters more than sample size. Their enormous error was based on a poll of more than 2mn American voters.

Today, all reputable pollsters carefully recruit, sample and weight respondents to ensure that topline results reflect the correct numbers of young people, women, college graduates and so on, making a miss on such a humiliating scale impossible. But what is true of a poll overall is not generally true for its subgroups. Pollsters don’t typically ensure there are the right number of Latinos, say, or high-school dropouts among under-30s.

A recent analysis from Lakshya Jain and colleagues at the US political analysis collective Split Ticket illustrates the issue. They polled several hundred American 18 to 29-year-olds using different methods of sampling, interviewing and weighting, and found that three different approaches — all of which are in use by different mainstream pollsters today — yielded starkly different results. These ranged from +18 for Trump when young adults were not weighted to be internally representative, to +16 for Biden with weights and sample both carefully calibrated.

That pollsters don’t routinely weight within demographic subgroups like this is not an oversight — it’s just not what nationally representative polls are designed for. And as Natalie Jackson, vice-president at US market research firm GQR points out, with small samples weighting can do as much harm as good. So when enthusiasts or political participants “dive into the cross tabs” to look at the results for particular demographics, they’re looking at data that is not intended to be representative of those groups.

This means that the under-30s in one poll might skew more male, another’s might skew more college-educated and so on. This alone can explain a huge amount of the variation we’re seeing this year.

In addition, the old challenge (trying to find young people who will answer the phone) is being replaced by a new one. Pew Research has found evidence suggesting substantial numbers of people who sign up to online polls give false information, and that this is especially true of those who claim to be young voters.

But all is not lost. One regular survey that bucks the trends described above is the Harvard Youth Poll, which interviews 2,000 under-30s, who they recruit using official records. It then weights them to be representative of young American adults at large in terms of everything from age, race and education to income, region and language.

The results are almost exactly the same as Split Ticket’s similarly sampled and weighted version — a solid double-digit lead for Biden, albeit diminished compared with the result in 2020. Which suggests that a large amount of not only the wide variation, but also the apparent rightward shift among under-30s seen elsewhere, may be the result of unrepresentative samples of young voters.

None of this is to say that Democrats should be complacent about youth support, especially amid the unrest over the Gaza conflict. But the alarming picture painted by some polls of a pro-Trump surge among the young is almost certainly a mirage.

john.burn-murdoch@ft.com, @jburnmurdoch

Have your say

Joe Biden vs Donald Trump: tell us how the 2024 US election will affect you