Popular in Political Science

American Politics

Keeping the Republic

Boston College Marc Landy

British Politics

Entrepreneurial thinking: the politics and practice of policy impact

University of Sheffield Matthew Flinders

Canadian Politics

Parties and Elections: An End to Canadian Exceptionalism?

St. Francis Xavier University James Bickerton

Asian Politics

The Politics of Knowledge in Development

University of Delhi Madhulika Banerjee

American Politics

Fifty years of declining confidence and increasing polarization in trust

University of California Berkeley Henry Brady

American Politics

The Opening Dilemma

Cornell University Richard Bensel

Political Economy

Wait and See?

University of Liverpool Mariaelisa Epifanio

Pollster choices vs voters’ decisions

View more insights in Social Sciences - Political Science - American Politics

View more figures from Vanderbilt University

View the page of Joshua Clinton

Next in American Politics

Dominant Personality and Politically Inexperienced Presidents Challenge Term Limits

Ignacio Arana Araya | Carnegie Mellon University

Human Dignity and Political Criticism

Colin Bird | University of Virginia

How much do pollster decisions influence election results, and what might the true Harris vs. Trump margin be when various polling adjustments are considered?

Image courtesy of interviewee. November 20, 2024

Log-in or Sign-up to Faculti
Currently viewing this subject insight as a guest. You have insight(s) remaining for this month. Login to view 8000+ figures on the platform.

Generating Workspace...

Export as PDF

This video discusses the complexities and challenges of polling, particularly in pre-election contexts. It explains that a typical poll may need to contact 40,000 voters to achieve a sample of 800 respondents, considering issues like low response rates (around 2%). This video reveals potential biases since the respondents might not reflect the wider population, especially as certain demographics are less likely to participate.

The author emphasizes that polling becomes increasingly difficult when predicting outcomes of elections, as sample demographics can misrepresent the electorate. Historical examples from the 2016 and 2020 elections illustrate how polls can underestimate or overestimate support for candidates, often due to unaccounted educational or partisan disparities.

The narrative highlights that pollsters must make assumptions about voter behavior and adjust their methods accordingly, which can lead to further inaccuracies. Furthermore, the difficulty of accurately gauging the electorate's composition introduces an inherent uncertainty in predicting close races.

Ultimately, the author argues that while polls can signal whether races are close or help gauge public opinion on various issues, they come with limitations that require careful interpretation. It's essential to recognize the ambiguity in polling results and not to overestimate their precision, as they can provide valuable insights into public sentiment even if not perfectly accurate.

Language:

Export as PDF

Export as PDF
Here are five key points from the video:

1. **Challenges in Polling Methodology**: The video discusses the significant difficulties pollsters face when trying to obtain a representative sample of voters, particularly in pre-election polls. The low response rate (e.g., 2%) means that to get a sample of 800 voters, pollsters must contact around 40,000 people, raising concerns about whether those who respond accurately reflect the larger population of non-respondents.

2. **Issues of Representation and Bias**: Pollsters must consider the biases of their samples, particularly in terms of education and partisanship. Historical polling errors, such as those seen in the 2016 and 2020 elections, are attributed to pollsters not adequately accounting for demographic differences that influence voter behavior. For instance, their samples often over-represented college-educated voters who leaned Democratic.

3. **Political Polarization and Trust**: The video highlights that current political polarization affects polling, with conservative and anti-establishment voters being less willing to participate in polls, which skews the sampling process. Pollsters struggle to determine whether discrepancies in their samples are due to actual voter distributions or reluctance from certain groups to engage with polling efforts.

4. **Limitations of Polling Accuracy**: The inherent limitations of polling are emphasized, particularly the mathematical realities of sampling errors. Even under ideal conditions, the results of polls can only provide a rough estimate of voter preferences and cannot deliver precise percentages, such as 47.3%. Polls are more valuable for indicating whether a race is close rather than providing definitive answers.

5. **Value of Polling in Democratic Processes**: Despite their flaws, the author argues that polling remains a crucial tool for gauging public opinion, assessing government approval, and understanding electoral dynamics. Polls can indicate general trends and help convideoualize election outcomes, providing insights into public sentiment that are not captured through alternative means, such as social media or protests. Thus, while polls may not always be perfectly accurate, they still play an invaluable role in democratic discourse.

Export as PDF
Here are some key quotes from the video you've , which discuss various challenges and insights related to polling, particularly in the convideo of pre-election surveys:

1. **Response Rate Challenge**:
- "It turns out like with a 2% response rate, to get 800 people to take my call, I need to call 40,000 voters."

2. **Sample Representation**:
- "How likely is it those 800 people reflect the 39,200 people who didn't take my poll?"

3. **Bias in Polling**:
- "We know those with college education or molecular to take polls."

4. **Difficulties of Pre-Election Polling**:
- "Pre-election poll is like a polling on level of difficulty, you know, three steps higher."

5. **Mistakes of Previous Polls**:
- "The polls in the United States... missed in large part because the pollsters were not accounting for the fact that education differences mattered."

6. **Partisan Response Issues**:
- "Republican or kind of conservative or anti-establishment, voter populist voters were less likely to take polls than Democrats who are more enthusiastic."

7. **Assumptions in Polling**:
- "If I assume like I'm gonna make my data look like 2020, I'm assuming every single Trump voter in 2020 is gonna vote for 2024, which we know is not true."

8. **Inherent Limitations of Polls**:
- "The rules of math say that if I call a thousand people as a poll... my answers are gonna vary."

9. **Polling Accuracy**:
- "A poll can tell me if it's close or if it's kind of, you know, not close."

10. **Consequences of Misinterpretations**:
- "If we throw away polling... then I think that's real unfortunate."

11. **Value of Polls**:
- "It's kind of remarkable that given all these difficulties, I can talk to 800 people that's held by hundreds of millions of people."

12. **General Political Insight**:
- "Even though they understated Trump, you know, in another sense they helped, they ballpark kind of range of plausible options."

These quotes encapsulate the essence of the video's argument regarding the complexities and difficulties of polling, especially in predicting electoral outcomes and capturing public sentiment accurately.

Export as PDF
Here are a few articles that cover similar themes related to polling, election forecasting, and the challenges associated with measuring public opinion:

1. **"The Trouble with Polling: Why We Can’t Trust Election Surveys"** - This article discusses the various factors that contribute to the inaccuracy of polling data, especially during elections, and examines the shifts in voter behavior that can affect poll outcomes.

2. **"Estimating the Electorate: How Pollsters Predict Who Will Vote"** - This piece looks at how pollsters make educated guesses about voter turnout and composition, delving into the mathematics behind sampling techniques and the implications of those estimates on poll accuracy.

3. **"Polling in a Partisan Era: Navigating Trust and Response Rates"** - This article explores the partisan biases that can affect polling results, particularly in a highly politicized environment, and discusses how pollsters address these biases to strive for representative samples.

4. **"The Evolving Art of Polling: Adapting to Changing Voter Demographics"** - This article examines how demographic shifts and changing political landscapes can complicate polling efforts, and it outlines strategies that pollsters are using to adapt to these changes.

5. **"Polling Accuracy After the 2020 Election: Lessons Learned"** - This article reflects on the polling inaccuracies seen in the 2020 election, analyzing what went wrong and what methodologies are being reconsidered by pollsters in light of recent election outcomes.

6. **"Understanding Polling Margins of Error: What They Mean for Election Predictions"** - This piece breaks down the concept of margins of error in polling, discussing how they affect the interpretation of results and the perception of polling reliability.

7. **"Sampling Bias in Election Polls: Challenges and Solutions"** - This article delves into the concept of sampling bias, including how certain demographic groups may be underrepresented in polls, and potential strategies to correct for those biases.

8. **"Polling and the Myth of Precision: Why Small Differences Can Mislead"** - This article articulates the dangers of overinterpreting slight variations in polling data, emphasizing the importance of recognizing the inherent limitations of polling methodologies.

These articles collectively address the complexities of polling, particularly in the convideo of elections, and provide insights into the statistical and methodological challenges faced by pollsters.

X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:90000

1
00:00:00.000 --> 00:00:01.405
Polls that I wanna do a poll of,

2
00:00:01.415 --> 00:00:02.805
let's say 800 voters, right?

3
00:00:02.805 --> 00:00:04.725
That's the average poll we saw in kind

4
00:00:04.725 --> 00:00:06.005
of pre-election polls in the United States.

5
00:00:06.065 --> 00:00:09.285
And so to get those 800 voters, I can kind of consult a list

6
00:00:09.285 --> 00:00:11.445
of registered voters, which, you know, every state has,

7
00:00:11.785 --> 00:00:13.285
and I can randomly sample voters

8
00:00:13.305 --> 00:00:14.445
and try to call them, right?

9
00:00:14.445 --> 00:00:16.965
Or I can try to email them and kind of get their opinion.

10
00:00:17.345 --> 00:00:20.005
It turns out like with a 2% response rate, which is

11
00:00:20.005 --> 00:00:22.445
what we have these days, that to get 800 people

12
00:00:22.445 --> 00:00:24.645
to take my answer, my call, I need

13
00:00:24.645 --> 00:00:26.845
to call 40,000 voters, right?

14
00:00:27.025 --> 00:00:29.285
So right off the bat, right, I know

15
00:00:29.285 --> 00:00:30.765
that even if I do things perfectly,

16
00:00:30.765 --> 00:00:32.725
even if I randomly sample registered voters

17
00:00:32.985 --> 00:00:35.325
to get the 800 people in my sample, I need

18
00:00:35.325 --> 00:00:36.685
to call 40,000 people.

19
00:00:37.065 --> 00:00:39.885
And so then I'm confronted with a stark real like, alright,

20
00:00:40.505 --> 00:00:44.845
how likely is it those 800 people reflect the 39,200 people

21
00:00:44.865 --> 00:00:46.845
who didn't take my poll, my call?

22
00:00:47.145 --> 00:00:48.725
And you know, right off the bat, they're weird, right?

23
00:00:48.785 --> 00:00:51.005
So, you know, they're different because like those 800 took

24
00:00:51.005 --> 00:00:53.645
my poll, the 39,200 did not, right?

25
00:00:53.645 --> 00:00:55.845
So right off the bat, right, that's kind of

26
00:00:55.845 --> 00:00:57.445
where the math and statistics ends.

27
00:00:57.445 --> 00:00:58.685
And so now it's kind of the art

28
00:00:58.685 --> 00:00:59.965
and the decisions of the pollster.

29
00:01:00.185 --> 00:01:01.885
So now as a pollster, I have to say, well,

30
00:01:02.125 --> 00:01:04.645
I have these 800 people and we have known biases.

31
00:01:04.905 --> 00:01:06.805
We know older people are molecular to take polls.

32
00:01:06.945 --> 00:01:07.845
We know those with college

33
00:01:07.845 --> 00:01:09.285
education or molecular to take polls.

34
00:01:09.285 --> 00:01:10.285
And so now I need

35
00:01:10.285 --> 00:01:12.925
to start making decisions about making those 800 people look

36
00:01:12.925 --> 00:01:14.005
like the population

37
00:01:14.005 --> 00:01:15.805
that I'm interested in, in kind of measuring.

38
00:01:16.105 --> 00:01:17.845
And so this also is kind of an important,

39
00:01:18.105 --> 00:01:19.285
disjunction point.

40
00:01:19.385 --> 00:01:22.325
So if I'm doing a normal poll of just like registered voters

41
00:01:22.585 --> 00:01:25.285
to, to assess public policies or government approval, right?

42
00:01:25.785 --> 00:01:26.885
I'm in a better situation

43
00:01:26.885 --> 00:01:28.925
because like I know what the United States

44
00:01:28.925 --> 00:01:30.765
or the UK looks like demographically, right?

45
00:01:30.765 --> 00:01:32.685
Or if I know what a state looks like demographically.

46
00:01:32.705 --> 00:01:35.765
And so I know exactly in some sense demographically

47
00:01:35.825 --> 00:01:36.885
how my set

48
00:01:36.885 --> 00:01:38.765
of respondents differs from who I'm trying to measure.

49
00:01:39.305 --> 00:01:42.205
But in a pre-election poll, I don't have that luxury, right?

50
00:01:42.205 --> 00:01:45.165
So a pre-election poll is like a polling on level

51
00:01:45.165 --> 00:01:46.765
of difficulty, you know, three steps higher

52
00:01:46.765 --> 00:01:48.325
because now I need to figure out,

53
00:01:48.465 --> 00:01:50.965
or I need to make an educated guess about who's going

54
00:01:50.965 --> 00:01:51.965
to vote in 2024,

55
00:01:52.505 --> 00:01:54.645
and then what are the ways in which my sample may, may,

56
00:01:54.705 --> 00:01:55.925
may, differ from that.

57
00:01:56.265 --> 00:01:57.965
And so, for example, in 2016,

58
00:01:57.985 --> 00:02:01.125
the polls in the United States, they missed in large part

59
00:02:01.125 --> 00:02:03.125
because the pollsters were not accounting for the fact

60
00:02:03.125 --> 00:02:05.165
that education differences mattered, right?

61
00:02:05.225 --> 00:02:07.325
We knew that more educated people are likely to take polls.

62
00:02:07.745 --> 00:02:10.245
But in 2016 in the United States, we saw that those,

63
00:02:10.245 --> 00:02:11.765
the college degrees were more likely to vote

64
00:02:11.765 --> 00:02:13.245
for Secretary Clinton with those,

65
00:02:13.245 --> 00:02:14.685
with less than a college degree were more likely

66
00:02:14.685 --> 00:02:16.525
to support, president Trump.

67
00:02:16.625 --> 00:02:18.405
And so when the pollsters weren't accounting for that,

68
00:02:18.515 --> 00:02:19.525
they had too many ed

69
00:02:19.525 --> 00:02:20.965
college educated people in their samples.

70
00:02:21.505 --> 00:02:22.645
So they weren't accounting for that.

71
00:02:22.665 --> 00:02:25.085
So the polls all overstated, you know, secretary Clinton.

72
00:02:25.385 --> 00:02:26.885
And that kind of led to the kind of result

73
00:02:26.885 --> 00:02:28.725
that we had in the polling where there was a giant mess.

74
00:02:29.145 --> 00:02:32.175
And so again, the problem the pollsters face is

75
00:02:32.175 --> 00:02:33.895
that we don't know what the electors looks like

76
00:02:33.915 --> 00:02:35.295
and we try to figure out kind of

77
00:02:35.295 --> 00:02:36.335
how there's differences there.

78
00:02:36.635 --> 00:02:38.455
And if you get that wrong, right?

79
00:02:38.835 --> 00:02:40.535
Or if you fail to account for that differences,

80
00:02:40.545 --> 00:02:42.575
which we don't know until it happens, right?

81
00:02:42.575 --> 00:02:44.335
You're gonna get the result wrong. You're always fighting

82
00:02:44.335 --> 00:02:45.615
the last battle in polling, right?

83
00:02:45.615 --> 00:02:48.135
And so in 2020, like we knew the education

84
00:02:48.135 --> 00:02:49.215
was a problem, right?

85
00:02:49.215 --> 00:02:50.655
So they were adjusting the results to kind

86
00:02:50.655 --> 00:02:52.615
of make sure the people answer my poll looked like the

87
00:02:52.615 --> 00:02:55.055
population, what we thought the elector was gonna be in

88
00:02:55.055 --> 00:02:56.135
2020, right?

89
00:02:56.135 --> 00:02:57.455
They all the posters were doing that

90
00:02:57.475 --> 00:02:58.615
and we, and they missed, right?

91
00:02:58.615 --> 00:03:00.445
They missed dramatically across the board

92
00:03:00.465 --> 00:03:02.645
by understating kind of support for President Trump.

93
00:03:02.905 --> 00:03:05.325
And so here, there was a new problem that emerged, right,

94
00:03:05.515 --> 00:03:08.085
that kind of also persisted in 2024, which is

95
00:03:08.085 --> 00:03:10.405
that not only is it kind of, we get differences in terms

96
00:03:10.405 --> 00:03:11.645
of education, but now there seems

97
00:03:11.645 --> 00:03:13.565
to be difference partisan responses, right?

98
00:03:13.565 --> 00:03:16.485
And so in particular Republican or kind of conservative

99
00:03:16.485 --> 00:03:19.285
or anti-establishment, voter populist voters were less

100
00:03:19.285 --> 00:03:21.405
likely to take polls than Democrats

101
00:03:21.405 --> 00:03:22.445
who are more enthusiastic.

102
00:03:22.785 --> 00:03:24.605
And so this then presents a super

103
00:03:24.625 --> 00:03:26.245
big quantity for pollsters, right?

104
00:03:26.245 --> 00:03:27.485
Because like I look at my sample

105
00:03:27.745 --> 00:03:29.765
and I say, I've got too many Democrats, right?

106
00:03:30.145 --> 00:03:32.845
Or I have too many Biden supporters, for example, in 2024,

107
00:03:32.845 --> 00:03:35.845
then I do Trump supporters, but like relative to what?

108
00:03:35.845 --> 00:03:38.045
Like how do I know whether that's a problem

109
00:03:38.045 --> 00:03:39.165
with people taking my poll

110
00:03:39.585 --> 00:03:41.765
or whether it's a statement about the electorate, right?

111
00:03:41.765 --> 00:03:44.405
So if I have a lot of Biden voters in 2024, is that

112
00:03:44.405 --> 00:03:46.085
because Democrats are gonna be really enthusiastic

113
00:03:46.085 --> 00:03:47.205
about voting in 2024?

114
00:03:47.625 --> 00:03:50.645
Or is it because basically I have, you know,

115
00:03:51.105 --> 00:03:52.205
not enough republicans

116
00:03:52.205 --> 00:03:54.045
because they're, they're reluctant to take my, my poll,

117
00:03:54.705 --> 00:03:56.805
you know, because like why would I trust a fake poll

118
00:03:56.805 --> 00:03:59.605
or fake news or, or share my views if I think those views

119
00:03:59.605 --> 00:04:00.645
are gonna be mischaracterized

120
00:04:00.645 --> 00:04:02.605
and that lack of trust then can cause

121
00:04:02.605 --> 00:04:03.885
problems that pollsters have to face.

122
00:04:04.185 --> 00:04:05.805
And so now I have a really hard difficulty

123
00:04:05.805 --> 00:04:07.205
because now if I have a partisan balance

124
00:04:07.225 --> 00:04:08.925
who takes my polls, right?

125
00:04:08.945 --> 00:04:11.685
Trying to fix that is like super complicated.

126
00:04:11.915 --> 00:04:14.245
This is why polls have never done this before, really

127
00:04:14.245 --> 00:04:17.565
before 2020 because you're basically in some sense picking

128
00:04:17.565 --> 00:04:18.605
the answer, right?

129
00:04:18.865 --> 00:04:21.725
So like in politics nowadays, everything is so politicized.

130
00:04:21.725 --> 00:04:24.565
Everything is so partisan that if I pick a partisan way

131
00:04:24.565 --> 00:04:26.925
number, right, or if I wait to pass vote,

132
00:04:27.065 --> 00:04:28.205
I'm basically picking the answer, right?

133
00:04:28.325 --> 00:04:31.285
'cause 95% of, of Trump voters in 20,

134
00:04:31.285 --> 00:04:32.805
in 2020 voted for Trump.

135
00:04:33.025 --> 00:04:35.445
You know, 95% of Biden voters voted for Harris.

136
00:04:35.745 --> 00:04:37.085
And so depending upon

137
00:04:37.085 --> 00:04:39.245
what I think those numbers are going to be, right?

138
00:04:39.245 --> 00:04:42.005
In 2024, I'm basically choosing the electorate, right?

139
00:04:42.005 --> 00:04:44.645
If I think the electorate's gonna be 40% Democrat, right?

140
00:04:44.645 --> 00:04:46.885
Versus 40% Republican, right?

141
00:04:46.885 --> 00:04:49.605
That's gonna have a huge impact on my top line number

142
00:04:49.605 --> 00:04:50.805
that I'm gonna, I'm gonna release.

143
00:04:51.105 --> 00:04:53.765
But the problem is I don't know what the truth is, right?

144
00:04:54.065 --> 00:04:55.805
So I don't know what the true balance is

145
00:04:55.805 --> 00:04:56.925
and different pollsters come up

146
00:04:56.925 --> 00:04:58.805
with different measures about partisanship or pass vote.

147
00:04:59.145 --> 00:05:01.925
So what we saw a lot of pollsters doing this go, go

148
00:05:01.925 --> 00:05:03.445
around is basically waiting on pass vote.

149
00:05:03.905 --> 00:05:05.565
So let's break that down what that means.

150
00:05:05.565 --> 00:05:07.165
So what that means is then when are you taking

151
00:05:07.165 --> 00:05:08.285
my poll in 2024?

152
00:05:08.475 --> 00:05:10.765
I'll ask you how you voted in 2020, right?

153
00:05:10.765 --> 00:05:12.245
And you'll say who you vote and I'm gonna assume

154
00:05:12.245 --> 00:05:13.365
that you didn't lie to me, right?

155
00:05:13.495 --> 00:05:15.885
Which is a problem, but you know, probably a small one.

156
00:05:16.145 --> 00:05:19.445
The bigger problem is then, alright, so now I know kind of

157
00:05:19.445 --> 00:05:21.285
what the people who took my poll said they,

158
00:05:21.285 --> 00:05:23.885
how they voted in 2020, but I don't know what

159
00:05:23.885 --> 00:05:25.405
that's gonna look like in 2024, right?

160
00:05:25.425 --> 00:05:26.725
So if I wait to kind of make

161
00:05:26.865 --> 00:05:29.285
who takes my poll this time look like

162
00:05:29.285 --> 00:05:31.685
what the electorate looked like in 2020, I've got a couple

163
00:05:31.685 --> 00:05:32.805
of kind of built assumption

164
00:05:32.805 --> 00:05:34.125
assumptions that I don't know if they're true.

165
00:05:34.125 --> 00:05:35.485
First of all, I don't know

166
00:05:35.485 --> 00:05:37.325
how many new voters there's gonna be, right?

167
00:05:37.325 --> 00:05:38.685
And kind of how that could shift things.

168
00:05:38.985 --> 00:05:41.845
And second of all, if I assume like I'm gonna make my data

169
00:05:41.955 --> 00:05:44.205
look like 2020, I'm assuming

170
00:05:44.205 --> 00:05:47.725
that every single Biden voter in 2020 is gonna vote in 2024.

171
00:05:47.935 --> 00:05:50.565
Every single Trump voter in 2020 is gonna vote for 2024,

172
00:05:50.565 --> 00:05:52.765
which we know is not true, right?

173
00:05:52.945 --> 00:05:54.645
And if there's a slight change in that, right?

174
00:05:54.645 --> 00:05:57.605
So for example, if Trump voters were slightly more likely

175
00:05:57.635 --> 00:06:00.245
than Biden voters should vote in 2024, which again,

176
00:06:00.275 --> 00:06:01.965
preliminary data seems to suggest,

177
00:06:02.275 --> 00:06:03.885
well then I'm gonna be off, right?

178
00:06:03.885 --> 00:06:06.445
Because I'm assuming something about the 2024 electorate

179
00:06:06.635 --> 00:06:09.085
that is turns out, you know, not, not to be true,

180
00:06:09.425 --> 00:06:11.725
but the problem's like even if that's not the case, right?

181
00:06:11.755 --> 00:06:13.525
Even if I nail that right?

182
00:06:13.525 --> 00:06:14.685
Think about what you're actually doing.

183
00:06:14.705 --> 00:06:16.685
So I'm basically holding Biden

184
00:06:16.825 --> 00:06:18.485
And, and Harris Voters fixed,

185
00:06:18.485 --> 00:06:19.485
Right? 'cause they're all gonna vote for

186
00:06:19.485 --> 00:06:20.165
the same candidate.

187
00:06:20.165 --> 00:06:22.405
And so that means the only way the polls are gonna

188
00:06:22.405 --> 00:06:23.685
Vary, and this Is why I think you saw

189
00:06:23.825 --> 00:06:26.925
so many polls in 2024, basically give you the same answer.

190
00:06:26.925 --> 00:06:29.965
Because once I weighed on past vote, the only way

191
00:06:29.995 --> 00:06:33.245
that my polls are gonna vary right from 2020

192
00:06:33.505 --> 00:06:36.565
and from one another is among that small group of voters

193
00:06:36.585 --> 00:06:38.385
who said they did not vote, right?

194
00:06:38.605 --> 00:06:40.825
That's the only, that's kind of free

195
00:06:40.885 --> 00:06:42.905
or that's not being constrained by decisions

196
00:06:42.905 --> 00:06:44.545
that I am making as a pollster effectively.

197
00:06:44.805 --> 00:06:46.905
And so in a poll of 800 people, right?

198
00:06:46.905 --> 00:06:48.225
That's 80 people, right?

199
00:06:48.285 --> 00:06:50.745
So basically all the polling variation, right,

200
00:06:51.125 --> 00:06:55.865
is constrained to what those 80 people out of the 800, out

201
00:06:55.865 --> 00:06:58.985
of the 40,000 that I try to call are kind of responding.

202
00:06:58.985 --> 00:07:01.345
And so once you realize that, you're like, yeah,

203
00:07:01.345 --> 00:07:02.625
that's gonna be really tight to try

204
00:07:02.625 --> 00:07:03.825
to call like a one point race.

205
00:07:03.825 --> 00:07:06.225
So I'm relying on the views of these 80 people who happen

206
00:07:06.225 --> 00:07:09.665
to take my poll and I better hope that they're reflective,

207
00:07:10.005 --> 00:07:11.705
of kind of general population.

208
00:07:11.725 --> 00:07:13.865
And again, you don't know that, until

209
00:07:13.865 --> 00:07:15.585
after the fact. There's an inherent

210
00:07:15.585 --> 00:07:16.585
Limitation in polling.

211
00:07:16.685 --> 00:07:18.665
And I think that that's not very well communicated

212
00:07:18.665 --> 00:07:20.865
because I think we expect more than polling than

213
00:07:20.865 --> 00:07:23.145
what it can do particularly in pre-election polling.

214
00:07:23.145 --> 00:07:24.945
And again, I wanna kind of make this distinction

215
00:07:24.945 --> 00:07:26.465
between pre-election polling where I don't know

216
00:07:26.465 --> 00:07:27.905
what I'm trying to wait to, right?

217
00:07:27.905 --> 00:07:30.385
And I, and errors in that can easily produce the type

218
00:07:30.385 --> 00:07:31.745
of errors that we see in the elections

219
00:07:31.805 --> 00:07:32.985
versus kind of election.

220
00:07:32.985 --> 00:07:34.305
British polling in general,

221
00:07:34.675 --> 00:07:35.865
Which kind of were

222
00:07:35.865 --> 00:07:37.545
Smaller errors are kind of more livable

223
00:07:37.545 --> 00:07:38.785
because they're less consequential.

224
00:07:38.885 --> 00:07:41.705
So the question like how I fix that, the problem is

225
00:07:41.705 --> 00:07:44.425
that it's not really fixable in, in, in a fundamental,

226
00:07:44.425 --> 00:07:46.545
in a fundamental way, which is that just if,

227
00:07:46.565 --> 00:07:48.985
if I'm in a perfect survey world, right? Where

228
00:07:48.985 --> 00:07:51.705
Every single person I call answers my poll, right?

229
00:07:52.405 --> 00:07:53.985
The rules of math say

230
00:07:53.985 --> 00:07:56.185
that if I call a thousand people as a poll,

231
00:07:56.475 --> 00:07:57.475
Right? My

232
00:07:57.475 --> 00:07:58.345
answers are gonna

233
00:07:58.345 --> 00:07:59.345
Vary. And you know, I try

234
00:07:59.345 --> 00:08:00.185
to estimate the proportion

235
00:08:00.185 --> 00:08:01.985
of something my an answers are gonna vary

236
00:08:01.985 --> 00:08:04.025
by like three percentage points plus or minus, right?

237
00:08:04.085 --> 00:08:07.105
My margin of error just because of random sampling, right?

238
00:08:07.285 --> 00:08:08.785
The thousand people I happen to talk

239
00:08:08.785 --> 00:08:10.785
to are gonna be slightly different than a thousand people.

240
00:08:10.945 --> 00:08:12.785
I did the exact same thing at a slightly different time.

241
00:08:12.965 --> 00:08:14.265
And that's unavoidable, right?

242
00:08:14.465 --> 00:08:15.865
I can get a larger and larger sample,

243
00:08:16.205 --> 00:08:17.385
But there's some Inherent

244
00:08:17.385 --> 00:08:18.945
randomness that's in that, right?

245
00:08:18.945 --> 00:08:20.105
Just you can't escape that.

246
00:08:20.185 --> 00:08:21.665
I can try to wait and try to make a smaller,

247
00:08:21.685 --> 00:08:24.305
but then I'm making assumptions that could be wrong,

248
00:08:24.305 --> 00:08:26.265
that you should try to account for that

249
00:08:26.265 --> 00:08:27.505
as well. And so what

250
00:08:27.505 --> 00:08:28.865
Does that mean? So that means

251
00:08:28.865 --> 00:08:32.345
If I have a 3% error, like just if everyone does what I,

252
00:08:32.345 --> 00:08:33.505
you know, polls want them to do,

253
00:08:33.505 --> 00:08:35.865
answer the phone when they're called, then if I'm trying

254
00:08:35.865 --> 00:08:38.745
to like predict an election, that means I'm subtracting two

255
00:08:38.745 --> 00:08:40.145
different estimates that I've made, right?

256
00:08:40.175 --> 00:08:42.985
This one has 3% error, this one has 3% error,

257
00:08:42.985 --> 00:08:44.665
which means my margin, right?

258
00:08:44.665 --> 00:08:45.905
Which is what everyone talks about.

259
00:08:45.905 --> 00:08:47.105
The difference between these two candidates

260
00:08:47.105 --> 00:08:50.105
or these two parties is then gonna be like approximately

261
00:08:50.105 --> 00:08:51.665
like six percentage points, right?

262
00:08:51.845 --> 00:08:54.585
And that's unavoidable, perfect world of polling.

263
00:08:54.805 --> 00:08:56.845
And so once you realize that, it's like

264
00:08:56.845 --> 00:08:58.725
what are we doing right when we're trying

265
00:08:58.725 --> 00:09:00.365
to like poll these races that we think are tied

266
00:09:00.365 --> 00:09:03.125
because there's some inherent unavoidable, you know,

267
00:09:03.125 --> 00:09:05.885
we can't get much closer than that just in a perfect world

268
00:09:06.425 --> 00:09:07.805
unless I happen to get lucky

269
00:09:07.945 --> 00:09:09.605
or correct in my waiting assumptions,

270
00:09:09.775 --> 00:09:11.245
which I don't know if they're true or not.

271
00:09:11.345 --> 00:09:14.005
And so, you know, the polls happen to get it right

272
00:09:14.585 --> 00:09:18.085
in general, but like given the, the difficulty of the task,

273
00:09:18.115 --> 00:09:19.845
like that's something, that's something you can bank

274
00:09:19.865 --> 00:09:21.045
on kind of going forward.

275
00:09:21.185 --> 00:09:23.205
And so I think what that means in terms

276
00:09:23.205 --> 00:09:25.085
of going forward is kind of appreciate

277
00:09:25.085 --> 00:09:27.125
that like a poll can tell you if the race is close

278
00:09:27.145 --> 00:09:28.205
or not, right?

279
00:09:28.345 --> 00:09:30.645
But the idea that you can extract information about whether

280
00:09:30.645 --> 00:09:33.485
or not a one point change is meaningful or,

281
00:09:33.705 --> 00:09:35.005
or just kind of randomness

282
00:09:35.025 --> 00:09:37.205
or the result of a pollster making a slight decision like

283
00:09:37.205 --> 00:09:39.645
that is well beyond, like I can't determine that

284
00:09:39.645 --> 00:09:40.845
and I'm an expert, right?

285
00:09:40.845 --> 00:09:42.765
And so I think the way to do that is just kind of step back

286
00:09:42.765 --> 00:09:45.325
and say, look, a poll can tell me if it's close

287
00:09:45.745 --> 00:09:47.685
or if it's, if it's kind of, you know, not close.

288
00:09:47.825 --> 00:09:49.445
And I, you know, I think we do a disservice

289
00:09:49.505 --> 00:09:51.005
and we try to think we can do more than

290
00:09:51.005 --> 00:09:53.285
that even if we happen to be lucky in some of the past

291
00:09:53.285 --> 00:09:54.485
and kind of, you know, doing that.

292
00:09:54.545 --> 00:09:57.365
And so why that matters, right?

293
00:09:57.365 --> 00:09:59.645
Is that once you recognize that what polls can do,

294
00:09:59.645 --> 00:10:02.125
like the difficulty is that you think of pre-election polls

295
00:10:02.125 --> 00:10:04.165
or if they've missed it again, they understated Trump by two

296
00:10:04.165 --> 00:10:05.405
to three percentage points in 2020.

297
00:10:05.405 --> 00:10:08.325
In 24 they understated by four percentage points in 2020.

298
00:10:08.555 --> 00:10:10.245
This is all garbage. Like I'm, you know,

299
00:10:10.245 --> 00:10:12.045
these pollsters dunno what they're doing, which

300
00:10:13.225 --> 00:10:15.045
is I think unfortunate

301
00:10:15.145 --> 00:10:17.445
and kind of also not exactly the right conclusion.

302
00:10:17.605 --> 00:10:19.485
I mean, 'cause the right conclusion is that it's kind

303
00:10:19.485 --> 00:10:20.805
of remarkable to me, right?

304
00:10:20.915 --> 00:10:24.165
That given all these difficulties, I can talk to 800 people

305
00:10:24.505 --> 00:10:26.845
and get within five points of a margin of an election

306
00:10:26.845 --> 00:10:28.405
that's held by hundreds of millions of people.

307
00:10:29.025 --> 00:10:31.965
Now that's not enough to call Pennsylvania or Wisconsin

308
00:10:32.025 --> 00:10:33.485
or any kinda super close races,

309
00:10:33.505 --> 00:10:35.765
but it's a remarkable testament that I can,

310
00:10:36.105 --> 00:10:38.645
on such small data I can get in the ballpark.

311
00:10:39.225 --> 00:10:41.165
And so long as what you're trying to use polling

312
00:10:41.165 --> 00:10:44.165
for is like in the ballpark, then I think you're okay.

313
00:10:44.265 --> 00:10:46.285
So a lot of public opinion polling is not

314
00:10:46.485 --> 00:10:47.565
trying to predict elections.

315
00:10:47.965 --> 00:10:49.485
A lot of public opinion polling is trying

316
00:10:49.485 --> 00:10:51.285
to say what's the approval of the government?

317
00:10:51.505 --> 00:10:53.165
What's the support for a given policy?

318
00:10:53.625 --> 00:10:56.085
And there the answer whether it's like 47

319
00:10:56.185 --> 00:10:58.005
or 52 doesn't really matter, right?

320
00:10:58.005 --> 00:10:59.165
Because like you look at that

321
00:10:59.165 --> 00:11:01.365
and you, the qualitative conclusion you draw from that,

322
00:11:01.365 --> 00:11:02.565
it's like it's close, right?

323
00:11:02.565 --> 00:11:03.565
It's evenly split

324
00:11:03.905 --> 00:11:05.685
or is there a large support for this policy

325
00:11:05.705 --> 00:11:07.365
or a large support for this government or not?

326
00:11:07.705 --> 00:11:09.805
And so in so far as we're not trying

327
00:11:09.805 --> 00:11:11.485
to make decimal point comparisons, right,

328
00:11:11.485 --> 00:11:13.485
which are far more precise than what we can actually do,

329
00:11:13.715 --> 00:11:15.805
then I think polling gives us useful information.

330
00:11:16.105 --> 00:11:18.285
And I think it's important to hold onto that

331
00:11:18.285 --> 00:11:19.685
because like absent that,

332
00:11:19.685 --> 00:11:22.325
what do we have if we don't have polling, which is kind

333
00:11:22.325 --> 00:11:23.365
of small, the democratic

334
00:11:23.485 --> 00:11:25.725
where we can help elevate the voices of just kind

335
00:11:25.725 --> 00:11:27.925
of normal people who happen to answer the phone, right?

336
00:11:27.925 --> 00:11:29.885
So maybe they're not that normal, but quasi normal,

337
00:11:29.885 --> 00:11:31.965
at least, at least, you know, then we're left to what?

338
00:11:31.965 --> 00:11:34.805
Social media people, well their voices are kind of loud

339
00:11:34.825 --> 00:11:35.885
and kind of unrepresented.

340
00:11:35.905 --> 00:11:38.365
People who are pro pro protesting the streets, again loud

341
00:11:38.505 --> 00:11:40.285
but not reflective of the general population.

342
00:11:40.285 --> 00:11:42.805
And so if we throw away polling right,

343
00:11:42.805 --> 00:11:45.965
because of, of small misses in pre-election polling,

344
00:11:45.965 --> 00:11:47.125
then I think that's real unfortunate.

345
00:11:47.285 --> 00:11:49.085
'cause then we have no ability to assess kind of how

346
00:11:49.085 --> 00:11:51.805
what the governments are doing, reflect the opinions

347
00:11:51.805 --> 00:11:53.845
of the people or kind of what an election even means, right?

348
00:11:53.845 --> 00:11:55.725
So we just had an election in the United States, right?

349
00:11:55.725 --> 00:11:58.365
President Trump was elected. What does that mean? Right?

350
00:11:58.585 --> 00:11:59.685
An election, a ballot box

351
00:11:59.685 --> 00:12:01.085
of vote count doesn't tell you what that means.

352
00:12:01.105 --> 00:12:03.045
It doesn't tell what people were thinking, right?

353
00:12:03.045 --> 00:12:05.885
And therefore kind of what the potential mandates are if, if

354
00:12:05.885 --> 00:12:07.165
that exists among the public,

355
00:12:07.305 --> 00:12:08.965
but only a poll can kind of do that.

356
00:12:09.385 --> 00:12:12.485
And so long as you, you kind of appropriately account

357
00:12:12.545 --> 00:12:15.085
for like, yeah, it's gonna get me in the ballpark,

358
00:12:15.305 --> 00:12:18.205
but it's not gonna get me like 47.3%, right?

359
00:12:18.235 --> 00:12:20.485
It's, it's not able to do that of a precision.

360
00:12:20.925 --> 00:12:22.645
I still think that information is useful

361
00:12:22.865 --> 00:12:24.685
and helps us guide what's going on.

362
00:12:24.705 --> 00:12:26.765
So example, the polls were all kind of, you know,

363
00:12:26.765 --> 00:12:28.165
they all understated Trump this year,

364
00:12:28.165 --> 00:12:30.765
but they were all saying it's gonna be a close race, right?

365
00:12:30.905 --> 00:12:32.405
And so I think that was useful

366
00:12:32.425 --> 00:12:35.125
for people going into the race like, look, you know,

367
00:12:35.385 --> 00:12:38.405
you didn't see a giant outcry about what happened

368
00:12:38.405 --> 00:12:40.885
because like people were kind of got their brains wrapped

369
00:12:40.885 --> 00:12:42.645
around that this could go either way, right?

370
00:12:42.645 --> 00:12:44.005
You could see Vice President Harris winning,

371
00:12:44.005 --> 00:12:45.125
you could see President Trump winning and,

372
00:12:45.125 --> 00:12:46.325
and that's what all the polls were saying.

373
00:12:46.585 --> 00:12:49.125
So even though they understated Trump, you know,

374
00:12:49.125 --> 00:12:51.925
in another sense they helped, they, they ballpark kind

375
00:12:51.925 --> 00:12:53.325
of range of plausible options

376
00:12:53.345 --> 00:12:54.485
or kind of opportunities there.

377
00:12:54.705 --> 00:12:57.365
And I think that's helpful for kind of allowing kind of the,

378
00:12:57.365 --> 00:12:58.605
you know, the transition of power

379
00:12:58.625 --> 00:12:59.405
and kind of helping people

380
00:12:59.405 --> 00:13:00.725
understand, you know, what's going on.

381
00:13:00.725 --> 00:13:03.365
And so again, it's, it's out about putting a proper level

382
00:13:03.365 --> 00:13:05.085
of kind of precision around these numbers.

383
00:13:05.305 --> 00:13:06.805
And I don't think, to be honest,

384
00:13:06.965 --> 00:13:08.685
like we do a good job presenting that

385
00:13:08.705 --> 00:13:10.125
or kind of explaining that to people

386
00:13:10.125 --> 00:13:12.205
because, you know, we like to know the score.

387
00:13:12.425 --> 00:13:13.845
We like to know who's up and who's down

388
00:13:13.845 --> 00:13:16.005
and we want things, you know, precise black

389
00:13:16.005 --> 00:13:17.005
or white binary in the world.

390
00:13:17.005 --> 00:13:19.525
But the reality is it's, it's called kind of mushy and gray

391
00:13:19.945 --> 00:13:22.245
and you have to be a little bit comfortable in the ambiguity

392
00:13:22.245 --> 00:13:24.725
and say, well, we're kind of in the ballpark, right?

393
00:13:24.825 --> 00:13:27.005
But that's still being in the ballpark is better than,

394
00:13:27.145 --> 00:13:28.645
you know, not being in the ballpark at all.

Challenges of Accurate Polling: Sampling and Representation

00:00:00 - Polls that I wanna do a poll of, let's say 800 voters, right? That's the average poll we saw in kind of pre-election polls in the United States. And so to get those 800 voters, I can kind of consult a list of registered voters, which, you know, every state has, and I can randomly sample voters and try to call them, right? Or I can try to email them and kind of get their opinion. It turns out like with a 2% response rate, which is what we have these days, that to get 800 people to take my answer, my call, I need to call 40,000 voters, right? So right off the bat, right, I know that even if I do things perfectly, even if I randomly sample registered voters

00:00:35 - to call 40,000 people. And so then I'm confronted with a stark real like, alright, how likely is it those 800 people reflect the 39,200 people who didn't take my poll, my call? And you know, right off the bat, they're weird, right? So, you know, they're different because like those 800 took my poll, the 39,200 did not, right? So right off the bat, right, that's kind of where the math and statistics ends. And so now it's kind of the art and the decisions of the pollster. So now as a pollster, I have to say, well, I have these 800 people and we have known biases.

00:01:06 - We know those with college education or molecular to take polls. And so now I need to start making decisions about making those 800 people look like the population that I'm interested in, in kind of measuring. And so this also is kind of an important, disjunction point. So if I'm doing a normal poll of just like registered voters to, to assess public policies or government approval, right? I'm in a better situation because like I know what the United States or the UK looks like demographically, right? Or if I know what a state looks like demographically. And so I know exactly in some sense demographically

Challenges in Pre-Election Polling Accuracy

00:01:36 - of respondents differs from who I'm trying to measure. But in a pre-election poll, I don't have that luxury, right? So a pre-election poll is like a polling on level of difficulty, you know, three steps higher because now I need to figure out, or I need to make an educated guess about who's going to vote in 2024, and then what are the ways in which my sample may, may, may, differ from that. And so, for example, in 2016, the polls in the United States, they missed in large part because the pollsters were not accounting for the fact that education differences mattered, right? We knew that more educated people are likely to take polls.

00:02:10 - the college degrees were more likely to vote for Secretary Clinton with those, with less than a college degree were more likely to support, president Trump. And so when the pollsters weren't accounting for that, they had too many ed college educated people in their samples. So they weren't accounting for that. So the polls all overstated, you know, secretary Clinton. And that kind of led to the kind of result that we had in the polling where there was a giant mess. And so again, the problem the pollsters face is that we don't know what the electors looks like and we try to figure out kind of how there's differences there. And if you get that wrong, right?

00:02:40 - which we don't know until it happens, right? You're gonna get the result wrong. You're always fighting the last battle in polling, right? And so in 2020, like we knew the education was a problem, right? So they were adjusting the results to kind of make sure the people answer my poll looked like the population, what we thought the elector was gonna be in 2020, right? They all the posters were doing that and we, and they missed, right? They missed dramatically across the board by understating kind of support for President Trump. And so here, there was a new problem that emerged, right, that kind of also persisted in 2024, which is that not only is it kind of, we get differences in terms

Challenges of Polling Amid Political Partisanship

00:03:11 - to be difference partisan responses, right? And so in particular Republican or kind of conservative or anti-establishment, voter populist voters were less likely to take polls than Democrats who are more enthusiastic. And so this then presents a super big quantity for pollsters, right? Because like I look at my sample and I say, I've got too many Democrats, right? Or I have too many Biden supporters, for example, in 2024, then I do Trump supporters, but like relative to what? Like how do I know whether that's a problem with people taking my poll or whether it's a statement about the electorate, right?

00:03:44 - because Democrats are gonna be really enthusiastic about voting in 2024? Or is it because basically I have, you know, not enough republicans because they're, they're reluctant to take my, my poll, you know, because like why would I trust a fake poll or fake news or, or share my views if I think those views are gonna be mischaracterized and that lack of trust then can cause problems that pollsters have to face. And so now I have a really hard difficulty because now if I have a partisan balance who takes my polls, right? Trying to fix that is like super complicated.

00:04:14 - before 2020 because you're basically in some sense picking the answer, right? So like in politics nowadays, everything is so politicized. Everything is so partisan that if I pick a partisan way number, right, or if I wait to pass vote, I'm basically picking the answer, right? 'cause 95% of, of Trump voters in 20, in 2020 voted for Trump. You know, 95% of Biden voters voted for Harris. And so depending upon what I think those numbers are going to be, right? In 2024, I'm basically choosing the electorate, right?

Challenges in Predicting Voter Behavior for 2024 Polls

00:04:44 - Versus 40% Republican, right? That's gonna have a huge impact on my top line number that I'm gonna, I'm gonna release. But the problem is I don't know what the truth is, right? So I don't know what the true balance is and different pollsters come up with different measures about partisanship or pass vote. So what we saw a lot of pollsters doing this go, go around is basically waiting on pass vote. So let's break that down what that means. So what that means is then when are you taking my poll in 2024? I'll ask you how you voted in 2020, right? And you'll say who you vote and I'm gonna assume

00:05:13 - Which is a problem, but you know, probably a small one. The bigger problem is then, alright, so now I know kind of what the people who took my poll said they, how they voted in 2020, but I don't know what that's gonna look like in 2024, right? So if I wait to kind of make who takes my poll this time look like what the electorate looked like in 2020, I've got a couple of kind of built assumption assumptions that I don't know if they're true. First of all, I don't know how many new voters there's gonna be, right? And kind of how that could shift things. And second of all, if I assume like I'm gonna make my data look like 2020, I'm assuming

00:05:47 - Every single Trump voter in 2020 is gonna vote for 2024, which we know is not true, right? And if there's a slight change in that, right? So for example, if Trump voters were slightly more likely than Biden voters should vote in 2024, which again, preliminary data seems to suggest, well then I'm gonna be off, right? Because I'm assuming something about the 2024 electorate that is turns out, you know, not, not to be true, but the problem's like even if that's not the case, right? Even if I nail that right? Think about what you're actually doing.

Challenges of Polling: Variability and Expectation Management

00:06:16 - And, and Harris Voters fixed, Right? 'cause they're all gonna vote for the same candidate. And so that means the only way the polls are gonna Vary, and this Is why I think you saw so many polls in 2024, basically give you the same answer. Because once I weighed on past vote, the only way that my polls are gonna vary right from 2020 and from one another is among that small group of voters who said they did not vote, right? That's the only, that's kind of free or that's not being constrained by decisions that I am making as a pollster effectively.

00:06:46 - That's 80 people, right? So basically all the polling variation, right, is constrained to what those 80 people out of the 800, out of the 40,000 that I try to call are kind of responding. And so once you realize that, you're like, yeah, that's gonna be really tight to try to call like a one point race. So I'm relying on the views of these 80 people who happen to take my poll and I better hope that they're reflective, of kind of general population. And again, you don't know that, until after the fact. There's an inherent

00:07:16 - And I think that that's not very well communicated because I think we expect more than polling than what it can do particularly in pre-election polling. And again, I wanna kind of make this distinction between pre-election polling where I don't know what I'm trying to wait to, right? And I, and errors in that can easily produce the type of errors that we see in the elections versus kind of election. British polling in general, Which kind of were Smaller errors are kind of more livable because they're less consequential. So the question like how I fix that, the problem is that it's not really fixable in, in, in a fundamental, in a fundamental way, which is that just if,

Understanding Margin of Error in Polling

00:07:48 - Every single person I call answers my poll, right? The rules of math say that if I call a thousand people as a poll, Right? My answers are gonna Vary. And you know, I try to estimate the proportion of something my an answers are gonna vary by like three percentage points plus or minus, right? My margin of error just because of random sampling, right? The thousand people I happen to talk to are gonna be slightly different than a thousand people. I did the exact same thing at a slightly different time. And that's unavoidable, right? I can get a larger and larger sample, But there's some Inherent

00:08:18 - Just you can't escape that. I can try to wait and try to make a smaller, but then I'm making assumptions that could be wrong, that you should try to account for that as well. And so what Does that mean? So that means If I have a 3% error, like just if everyone does what I, you know, polls want them to do, answer the phone when they're called, then if I'm trying to like predict an election, that means I'm subtracting two different estimates that I've made, right? This one has 3% error, this one has 3% error, which means my margin, right? Which is what everyone talks about. The difference between these two candidates or these two parties is then gonna be like approximately

00:08:51 - And that's unavoidable, perfect world of polling. And so once you realize that, it's like what are we doing right when we're trying to like poll these races that we think are tied because there's some inherent unavoidable, you know, we can't get much closer than that just in a perfect world unless I happen to get lucky or correct in my waiting assumptions, which I don't know if they're true or not. And so, you know, the polls happen to get it right in general, but like given the, the difficulty of the task, like that's something, that's something you can bank on kind of going forward.

Understanding the Limitations of Polling Accuracy

00:09:23 - of going forward is kind of appreciate that like a poll can tell you if the race is close or not, right? But the idea that you can extract information about whether or not a one point change is meaningful or, or just kind of randomness or the result of a pollster making a slight decision like that is well beyond, like I can't determine that and I'm an expert, right? And so I think the way to do that is just kind of step back and say, look, a poll can tell me if it's close or if it's, if it's kind of, you know, not close. And I, you know, I think we do a disservice and we try to think we can do more than that even if we happen to be lucky in some of the past

00:09:54 - And so why that matters, right? Is that once you recognize that what polls can do, like the difficulty is that you think of pre-election polls or if they've missed it again, they understated Trump by two to three percentage points in 2020. In 24 they understated by four percentage points in 2020. This is all garbage. Like I'm, you know, these pollsters dunno what they're doing, which is I think unfortunate and kind of also not exactly the right conclusion. I mean, 'cause the right conclusion is that it's kind of remarkable to me, right? That given all these difficulties, I can talk to 800 people

00:10:26 - that's held by hundreds of millions of people. Now that's not enough to call Pennsylvania or Wisconsin or any kinda super close races, but it's a remarkable testament that I can, on such small data I can get in the ballpark. And so long as what you're trying to use polling for is like in the ballpark, then I think you're okay. So a lot of public opinion polling is not trying to predict elections. A lot of public opinion polling is trying to say what's the approval of the government? What's the support for a given policy? And there the answer whether it's like 47

The Value of Polling in Democratic Representation

00:10:58 - Because like you look at that and you, the qualitative conclusion you draw from that, it's like it's close, right? It's evenly split or is there a large support for this policy or a large support for this government or not? And so in so far as we're not trying to make decimal point comparisons, right, which are far more precise than what we can actually do, then I think polling gives us useful information. And I think it's important to hold onto that because like absent that, what do we have if we don't have polling, which is kind of small, the democratic where we can help elevate the voices of just kind of normal people who happen to answer the phone, right?

00:11:29 - at least, at least, you know, then we're left to what? Social media people, well their voices are kind of loud and kind of unrepresented. People who are pro pro protesting the streets, again loud but not reflective of the general population. And so if we throw away polling right, because of, of small misses in pre-election polling, then I think that's real unfortunate. 'cause then we have no ability to assess kind of how what the governments are doing, reflect the opinions of the people or kind of what an election even means, right? So we just had an election in the United States, right? President Trump was elected. What does that mean? Right?

00:11:59 - of vote count doesn't tell you what that means. It doesn't tell what people were thinking, right? And therefore kind of what the potential mandates are if, if that exists among the public, but only a poll can kind of do that. And so long as you, you kind of appropriately account for like, yeah, it's gonna get me in the ballpark, but it's not gonna get me like 47.3%, right? It's, it's not able to do that of a precision. I still think that information is useful and helps us guide what's going on. So example, the polls were all kind of, you know, they all understated Trump this year, but they were all saying it's gonna be a close race, right?

Understanding Polling Uncertainty in Elections

00:12:32 - for people going into the race like, look, you know, you didn't see a giant outcry about what happened because like people were kind of got their brains wrapped around that this could go either way, right? You could see Vice President Harris winning, you could see President Trump winning and, and that's what all the polls were saying. So even though they understated Trump, you know, in another sense they helped, they, they ballpark kind of range of plausible options or kind of opportunities there. And I think that's helpful for kind of allowing kind of the, you know, the transition of power and kind of helping people understand, you know, what's going on. And so again, it's, it's out about putting a proper level

00:13:27 - And I don't think, to be honest, like we do a good job presenting that or kind of explaining that to people because, you know, we like to know the score. We like to know who's up and who's down and we want things, you know, precise black or white binary in the world. But the reality is it's, it's called kind of mushy and gray and you have to be a little bit comfortable in the ambiguity and say, well, we're kind of in the ballpark, right? But that's still being in the ballpark is better than, you know, not being in the ballpark at all.

Faculti AI can make mistakes

Faculti

Popular in Political Science

Keeping the Republic

Entrepreneurial thinking: the politics and practice of policy impact

Parties and Elections: An End to Canadian Exceptionalism?

The Politics of Knowledge in Development

Fifty years of declining confidence and increasing polarization in trust

The Opening Dilemma

Wait and See?

Pollster choices vs voters’ decisions

Next in American Politics

Guide

Platform and Category Pages

Contact Us

FAQs

What is Faculti?

How do you select interviews?

Interview Process

Page Structure

Popular in Political Science

Guide

Platform and Category Pages

Contact Us

FAQs

What is Faculti?

How do you select interviews?

Interview Process

Accessibility Options

Contrast Adjustment

Text Size Adjustment

Text Spacing Adjustment

Line Height Adjustment

Cursor Size Adjustment

Page Structure Options

Page Structure