Popular in Social Sciences

Political Science

V-Dem’s Conceptions of Democracy and Their Consequences

University of Notre Dame Michael Coppedge

Central Monetary Authorities

Macroeconomic modelling of CBDC: a critical review

European Central Bank Ulrich Bindseil

Social Identities and Divisions

A new explanation of antisemitism: Jew hatred as a civilisational phenomenon

Boston University Liah Greenfeld

Central Monetary Authorities

Financing Private Credit

Federal Reserve Bank of New York Nina Boyarchenko

Economics

Failing Banks

Massachusetts Institute of Technology Emil Verner

Culture and Society

The City as a Growth Machine: Toward a Political Economy of Place

New York University Harvey Moltoch

American Politics

Relevance, validity, and evidential reasoning in clinical practice

View more insights in Social Sciences - Health Policy

View more figures from University of Toronto

View the page of Brian Baigrie

Read the Study

Next in Health Policy

The case for Medicare to cover home safety renovations

| USC Marshall

Children’s exposure to alcohol marketing within supermarkets

| Imperial College London

Brian Baigrie examines how quality (reliability/validity) and relevance (weight/importance) affect the vast literature on EBM’s advantages.

Image courtesy of interviewee. March 22, 2024

Log-in or Sign-up to Faculti
Currently viewing this subject insight as a guest. You have insight(s) remaining for this month. Login to view 8000+ figures on the platform.

Generating Workspace...

Export as PDF

The speaker reflects on the concept of evidence-based medicine (EBM) and its implications during the COVID-19 pandemic. They suggest that EBM traditionally focuses too narrowly on population studies and controlled trials, potentially overlooking other valuable forms of evidence, such as observational studies and patient narratives. This video video narrow focus can lead to a lack of responsiveness to real-world complexities, particularly in light of the pandemic.

The speaker discusses the significance of distinguishing between two criteria for evidence: its methodological "goodness" (validity and reliability) and its relevance (persuasiveness and appropriateness). They note that many decisions made by public health organizations during the pandemic—like the initially dismissive stance on the effectiveness of masks and the delayed recognition of airborne transmission—reflect inadequate consideration of these criteria.

Furthermore, they argue that evidence-based medicine’s hierarchical framework, which privileges certain types of research over others, may have hindered a broader understanding of the pandemic's complexities. The pandemic serves as a wake-up call, revealing the limitations of a rigid adherence to traditional EBM principles. The conversation closes by highlighting the importance of integrating diverse forms of medical knowledge and evidence to enhance public health strategies and improve trust in health authorities.

Language:

Export as PDF

X-TIMESTAMP-MAP=LOCAL:00:00:00.000,MPEGTS:90000

1
00:00:00.205 --> 00:00:02.325
I came at this much before the pandemic.

2
00:00:02.515 --> 00:00:04.445
Looking at EBM

3
00:00:04.625 --> 00:00:07.805
and what's considered to be a narrow focus on,

4
00:00:08.725 --> 00:00:10.325
derivations from population studies,

5
00:00:10.795 --> 00:00:14.285
looking at legal literature in other areas where evidence

6
00:00:14.285 --> 00:00:17.325
of reasoning is really been, studied

7
00:00:17.505 --> 00:00:21.525
and theorized, there is standardly a distinction between

8
00:00:22.485 --> 00:00:25.285
evidence that it, I mean, we can use the word goodness,

9
00:00:25.345 --> 00:00:26.605
you may not wanna use that.

10
00:00:26.635 --> 00:00:29.325
That is valid according to standard research methods.

11
00:00:29.425 --> 00:00:33.045
And the law would be how one gives a deposition of some sort

12
00:00:33.045 --> 00:00:36.085
or other on one and relevance that is,

13
00:00:36.265 --> 00:00:38.165
has weight in some sense.

14
00:00:38.505 --> 00:00:41.765
And both of those criteria need to be met for something

15
00:00:41.765 --> 00:00:46.045
as admitted as evidence in, in, in most legal jurisdictions.

16
00:00:46.265 --> 00:00:50.565
So I, I started thinking about EBM from that point of view.

17
00:00:50.705 --> 00:00:54.885
And, you know, the article mentions, pithy comment by my

18
00:00:55.405 --> 00:00:57.685
deceased brilliant colleague Ian Hacking,

19
00:00:57.685 --> 00:00:59.445
who we wrote a lot about statistics

20
00:00:59.445 --> 00:01:01.405
and probability that very, you know,

21
00:01:01.605 --> 00:01:03.165
evidence in principle is something

22
00:01:03.165 --> 00:01:04.405
that reaches beyond itself.

23
00:01:04.505 --> 00:01:06.525
But in fact, as I pointed out,

24
00:01:07.315 --> 00:01:10.765
that sometimes it is not persuasive, even evidence

25
00:01:10.765 --> 00:01:15.125
that sometimes seems to be gleaned using the best re

26
00:01:15.125 --> 00:01:17.925
or what we think of as the best research methods

27
00:01:18.265 --> 00:01:19.365
may not be persuasive.

28
00:01:19.385 --> 00:01:21.525
And I thought, I wrote this at the very beginning

29
00:01:21.525 --> 00:01:24.045
of the pandemic, and it was a time when there was a great

30
00:01:24.045 --> 00:01:26.565
deal of debate about routes of transmission for covid

31
00:01:27.105 --> 00:01:28.645
and, you know, the effectiveness

32
00:01:28.665 --> 00:01:31.165
of some public health measures like social distancing

33
00:01:31.165 --> 00:01:33.245
and face masking and the rest of those things.

34
00:01:33.665 --> 00:01:35.965
And I thought that, well, much

35
00:01:35.985 --> 00:01:37.805
of the conversation is a muddled

36
00:01:37.805 --> 00:01:40.325
because these two things are being rolled together.

37
00:01:40.325 --> 00:01:42.805
What we're looking for here to find our way out

38
00:01:42.805 --> 00:01:45.805
of the pandemic is evidence that's good according

39
00:01:45.905 --> 00:01:48.085
to the certain criteria, in particular

40
00:01:48.765 --> 00:01:49.805
criteria we would associate

41
00:01:49.835 --> 00:01:51.965
with the evidence-based medicine movement,

42
00:01:52.105 --> 00:01:54.845
the World Health Organization, like hundreds

43
00:01:54.845 --> 00:01:59.595
of other organizations bought into EBM, which as you know,

44
00:01:59.735 --> 00:02:02.435
privileges, population studies, tries

45
00:02:02.435 --> 00:02:05.075
to generate policy sometimes during the pandemic,

46
00:02:05.215 --> 00:02:07.435
but tries to generate clinical recommendations

47
00:02:07.455 --> 00:02:11.395
for individual patients based upon the normal, patient

48
00:02:11.575 --> 00:02:13.675
or what happened in the study group.

49
00:02:13.975 --> 00:02:16.515
So I, I wanted to address that in this paper

50
00:02:16.775 --> 00:02:19.835
and led a different framework for thinking about evidence.

51
00:02:20.335 --> 00:02:24.355
You know, a BM has been very masterful at putting a books

52
00:02:24.375 --> 00:02:27.275
of guidelines and all the rest of this stuff and handbooks,

53
00:02:27.615 --> 00:02:31.915
but there is a theory of evidence that underpins, EBM,

54
00:02:31.915 --> 00:02:34.515
but it has never been, like spelled out

55
00:02:34.695 --> 00:02:36.155
as a theoretical position.

56
00:02:36.375 --> 00:02:40.195
So I wanted to lay down a couple of criteria that any theory

57
00:02:40.195 --> 00:02:41.755
of evidence should satisfy.

58
00:02:42.535 --> 00:02:45.235
One related to the goodness, validity, reliability,

59
00:02:45.235 --> 00:02:47.675
depending on your take on that on one side.

60
00:02:47.815 --> 00:02:49.395
But on the other side, the relevance,

61
00:02:49.415 --> 00:02:53.075
persuasiveness appropriateness of evidence as well.

62
00:02:53.255 --> 00:02:56.475
The EBM over a series of publications has kind

63
00:02:56.475 --> 00:02:58.715
of refined their view and changed it

64
00:02:58.715 --> 00:02:59.715
In some ways.

65
00:02:59.835 --> 00:03:03.645
Many people would say not, there haven't been as responsive

66
00:03:03.645 --> 00:03:06.405
to concerns about how patient preference values,

67
00:03:06.785 --> 00:03:08.965
et cetera fits into this kind of framework

68
00:03:08.985 --> 00:03:11.205
for making clinical recommendations.

69
00:03:11.585 --> 00:03:13.085
But there were other concerns as well

70
00:03:13.085 --> 00:03:16.245
because, you know, in, in the most recent iteration

71
00:03:16.965 --> 00:03:18.805
evidence-based medicine, how to practice

72
00:03:18.865 --> 00:03:21.685
and teach it, it says quite explicitly the

73
00:03:21.685 --> 00:03:22.925
best research evidence.

74
00:03:23.545 --> 00:03:25.805
By best research evidence, we mean valid

75
00:03:26.105 --> 00:03:28.085
and clinically relevant research.

76
00:03:28.185 --> 00:03:30.765
So the relevance criterion, which I think needs

77
00:03:30.765 --> 00:03:33.965
to be sorted out from the, the goodness criterion,

78
00:03:33.965 --> 00:03:36.165
they're kind of both in there together.

79
00:03:36.345 --> 00:03:40.005
So, you know, good evidence is evidence that's accrued

80
00:03:40.005 --> 00:03:41.925
through a very particular research method.

81
00:03:42.305 --> 00:03:44.525
And which is kind of an odd thing

82
00:03:44.525 --> 00:03:46.805
because medicine is this place

83
00:03:46.835 --> 00:03:49.325
with many different research method side

84
00:03:49.325 --> 00:03:50.645
by side with one another.

85
00:03:50.645 --> 00:03:53.245
Different sorts of movements about what counts

86
00:03:53.245 --> 00:03:55.885
as medical knowledge and narrative based medicine,

87
00:03:55.915 --> 00:03:58.805
precision medicine, person centered medicine.

88
00:03:59.265 --> 00:04:00.725
And the rest of them, there are many more.

89
00:04:00.865 --> 00:04:03.125
And you know, EBM kind of hold kind

90
00:04:03.125 --> 00:04:04.525
of a Trump position here,

91
00:04:04.905 --> 00:04:08.085
but through the creation of the grade system, right?

92
00:04:08.265 --> 00:04:11.325
The grade framework, which kind of creates a hierarchy

93
00:04:11.465 --> 00:04:14.805
for the confidence we can have in different sources

94
00:04:14.945 --> 00:04:15.965
of medical knowledge,

95
00:04:16.035 --> 00:04:19.925
putting statistically derived information from population

96
00:04:19.925 --> 00:04:22.445
studies, RCTs, randomized controlled trials,

97
00:04:22.665 --> 00:04:25.165
and you know, meta, meta studies.

98
00:04:25.615 --> 00:04:28.285
Those meta studies by the way, only include RCTs.

99
00:04:28.315 --> 00:04:31.085
They don't include observational studies or cohort studies

100
00:04:31.225 --> 00:04:33.165
or many other ways that we could try

101
00:04:33.165 --> 00:04:36.045
to become better informed about any issue at hand.

102
00:04:36.185 --> 00:04:38.645
So I don't think there's been much pivoting in

103
00:04:38.645 --> 00:04:39.845
the evidence-based movement.

104
00:04:39.905 --> 00:04:42.285
It just seems to be rolling along in much the same way.

105
00:04:42.285 --> 00:04:43.885
But I do think the, the pandemic

106
00:04:43.945 --> 00:04:45.565
to some extent was a wake up call

107
00:04:45.565 --> 00:04:48.605
because most public health agencies, as you know,

108
00:04:48.625 --> 00:04:52.565
the s general of the United States lost his position in part

109
00:04:52.565 --> 00:04:54.285
because he was parroting something

110
00:04:54.285 --> 00:04:56.485
that the World Health Organization was saying.

111
00:04:56.585 --> 00:04:57.965
You know, we in Canada,

112
00:04:58.265 --> 00:05:00.965
our so-called top Doctor didn't lose her position,

113
00:05:00.985 --> 00:05:03.085
but there were many people who thought that she should,

114
00:05:03.465 --> 00:05:05.685
I'm not saying I'm one of them, but clearly it was just

115
00:05:05.685 --> 00:05:09.645
following and World Health organization's footsteps on a

116
00:05:09.645 --> 00:05:12.045
number of issues that were pivotable pivotal

117
00:05:12.225 --> 00:05:14.845
to thinking about the best strategies to manage a pandemic

118
00:05:15.115 --> 00:05:16.485
that has been a promise

119
00:05:16.675 --> 00:05:18.965
that evidence-based medicine proponents

120
00:05:18.965 --> 00:05:21.325
of evidence-based medicine, made very early.

121
00:05:21.925 --> 00:05:23.525
I don't suggest in this paper we're talking,

122
00:05:23.585 --> 00:05:25.765
that's the background for our conversation today,

123
00:05:25.825 --> 00:05:29.085
but in other places, the grade system evaluation gives

124
00:05:29.365 --> 00:05:33.645
approval for clinicians to kind of step outside of

125
00:05:34.325 --> 00:05:37.685
directly asking a patient about their various preferences,

126
00:05:37.685 --> 00:05:41.125
information about the patient based on normal patient,

127
00:05:41.275 --> 00:05:43.005
that something called a normal patient

128
00:05:43.115 --> 00:05:46.925
that emerges in the context of trials of one sort or other.

129
00:05:47.345 --> 00:05:49.925
And, there's a, a very explicit statement

130
00:05:49.925 --> 00:05:53.085
that clinicians have a warrant to, proceed

131
00:05:53.115 --> 00:05:56.725
with a strong recommendation without detailed discussion

132
00:05:56.725 --> 00:05:59.565
with the patient, which is kind of striking, I think.

133
00:05:59.785 --> 00:06:03.285
And, it certainly clashes with the best intentions

134
00:06:03.305 --> 00:06:04.965
of person-centered medicine.

135
00:06:04.965 --> 00:06:07.285
Right? The other thing, of course, is that the idea

136
00:06:07.285 --> 00:06:09.045
of typical patients is really

137
00:06:09.255 --> 00:06:12.165
concerning when you think about the heterogeneity

138
00:06:12.165 --> 00:06:13.325
of clinical practice.

139
00:06:13.505 --> 00:06:16.765
You think about psychiatry for a second, right?

140
00:06:16.945 --> 00:06:19.325
You know, if a theory of evidence is being constructed

141
00:06:19.675 --> 00:06:22.645
with little regard for the heterogeneity of practice,

142
00:06:22.875 --> 00:06:24.085
what counts as,

143
00:06:24.115 --> 00:06:27.165
best clinical evidence might very well change

144
00:06:27.195 --> 00:06:28.245
from field to field.

145
00:06:28.345 --> 00:06:31.085
But there's no nuancing in EBM framework.

146
00:06:31.135 --> 00:06:33.285
Definitely is a one size fits all.

147
00:06:33.645 --> 00:06:35.045
A lot of examples about this,

148
00:06:35.065 --> 00:06:37.205
but I'll just focus on two familiar ones

149
00:06:37.205 --> 00:06:39.485
and try to stay away from more obscure ones.

150
00:06:39.485 --> 00:06:42.325
So we'll go back to the, the face coverings.

151
00:06:42.455 --> 00:06:44.725
Refer that word to masking for lots of reasons,

152
00:06:44.725 --> 00:06:47.765
because in Canada we had some issues about masks

153
00:06:47.865 --> 00:06:48.925
and face coverings

154
00:06:48.925 --> 00:06:50.205
and the constitutionality,

155
00:06:50.205 --> 00:06:52.405
which negatively impacted immigrant groups.

156
00:06:52.625 --> 00:06:53.885
I'm sorry if I'm being political,

157
00:06:53.945 --> 00:06:56.765
but you know, they very early in the pandemic,

158
00:06:56.985 --> 00:06:59.325
the World Health Organization, a strong proponent

159
00:06:59.325 --> 00:07:02.685
of evidence-based medicine, advocated that the public

160
00:07:02.785 --> 00:07:05.885
of wearing a face coverings conferred no

161
00:07:05.885 --> 00:07:07.285
benefit over and above.

162
00:07:07.545 --> 00:07:11.565
So distancing and curbing the transmission of COVID-19

163
00:07:11.905 --> 00:07:12.965
and setting Sweden

164
00:07:12.985 --> 00:07:14.365
and a couple of countries aside,

165
00:07:14.795 --> 00:07:17.765
most public health officials kind of parroted that message,

166
00:07:18.065 --> 00:07:21.085
which became kind of standard at the time.

167
00:07:21.175 --> 00:07:25.245
There was really a dear, like, little evidence that sort

168
00:07:25.245 --> 00:07:29.325
of squares with the EBM guideline of what constitutes good

169
00:07:29.385 --> 00:07:33.125
or valid evidence examining the issue, directly.

170
00:07:33.425 --> 00:07:36.365
And the response in the World Health Organization was, well,

171
00:07:36.475 --> 00:07:39.765
okay, there's an absence of evidence that we proponents

172
00:07:39.765 --> 00:07:41.485
of EBM would take as evidence.

173
00:07:41.705 --> 00:07:43.445
So we'll take this as no evidence

174
00:07:43.715 --> 00:07:46.925
that face coverings provide, social benefit, right?

175
00:07:46.945 --> 00:07:50.565
So in the absence of RCTs, were just assumed as no evidence,

176
00:07:50.865 --> 00:07:53.485
the trials that were had been carried on were kind

177
00:07:53.485 --> 00:07:56.645
of compromised in some ways by EBMS criteria

178
00:07:56.705 --> 00:07:58.005
of methodological rigor.

179
00:07:58.265 --> 00:08:00.485
And they were focused not on covid,

180
00:08:00.555 --> 00:08:04.005
they were focused on other infectious agents, influenza,

181
00:08:04.265 --> 00:08:07.765
middle East respiratory syndrome, and sars.

182
00:08:07.905 --> 00:08:10.085
And they were focused on source protection,

183
00:08:10.185 --> 00:08:13.365
but not source control, which is a d issue altogether.

184
00:08:13.585 --> 00:08:16.765
So that's one example I think where kind of thinking that,

185
00:08:16.905 --> 00:08:19.245
you know, in a perfect world, we will have a trial

186
00:08:19.265 --> 00:08:21.085
to tell us what to do so we can feel safe

187
00:08:21.085 --> 00:08:25.285
and certain generalizing to a bigger population where,

188
00:08:25.355 --> 00:08:27.805
kind of a narrowness, did not help manage

189
00:08:27.805 --> 00:08:29.045
because there were all kinds

190
00:08:29.045 --> 00:08:33.005
of stories afoot about field epidemiologists were very early

191
00:08:33.105 --> 00:08:36.765
in the pandemic, early 2020, that people were being

192
00:08:37.325 --> 00:08:39.525
infected, from contagious people

193
00:08:40.035 --> 00:08:41.245
more than a meter apart.

194
00:08:41.465 --> 00:08:44.525
So this is tied in with, routes of transmission

195
00:08:44.785 --> 00:08:46.645
and airborne transmission.

196
00:08:47.185 --> 00:08:49.845
The interesting thing about this is that, I mean,

197
00:08:49.845 --> 00:08:53.445
it took the World Health Organization two years to say,

198
00:08:53.565 --> 00:08:56.285
airborne transmission and put it on the front page.

199
00:08:56.425 --> 00:09:00.725
And there were legions of field epidemiologists, physicians,

200
00:09:01.035 --> 00:09:04.125
aerosol experts, engineers saying, look,

201
00:09:04.235 --> 00:09:05.885
this very disconcerting.

202
00:09:05.945 --> 00:09:09.125
So the interesting question is, you know, why was this sort

203
00:09:09.125 --> 00:09:11.365
of course a public health experts

204
00:09:11.785 --> 00:09:16.565
and aerosol scientists, engineers, why did the who not,

205
00:09:16.665 --> 00:09:17.845
listen to those people?

206
00:09:18.265 --> 00:09:21.645
And one answer that one finds frequently is

207
00:09:21.645 --> 00:09:25.765
because the World Health Organization was on side

208
00:09:25.765 --> 00:09:27.685
with some problematic assumptions about

209
00:09:27.705 --> 00:09:31.405
how the virus attacks us, about how the virus itself works.

210
00:09:31.905 --> 00:09:35.525
But I think the underlying reason is the view about evidence

211
00:09:35.755 --> 00:09:37.885
that impacted the world organization.

212
00:09:38.035 --> 00:09:40.965
That is kind of the legacy of EBM.

213
00:09:41.055 --> 00:09:44.365
Field Epidemiology reports were dismissed

214
00:09:44.365 --> 00:09:46.805
by the World Health Organization as proof

215
00:09:46.805 --> 00:09:49.485
of airborne transmission on the grounds

216
00:09:49.595 --> 00:09:51.565
that the evidence was not relevant.

217
00:09:51.625 --> 00:09:54.285
So it was not a weight, it wasn't appropriate.

218
00:09:54.375 --> 00:09:56.645
We're not gonna think about this right now.

219
00:09:56.905 --> 00:09:59.685
The World Health Organization also relied upon a

220
00:09:59.685 --> 00:10:00.805
narrow range of experts.

221
00:10:01.305 --> 00:10:03.325
So their group, physicians,

222
00:10:03.485 --> 00:10:04.885
I think there were at least 30 of them,

223
00:10:05.035 --> 00:10:07.845
they were thinking about spread of an,

224
00:10:07.845 --> 00:10:11.005
of a pathogen in a closed, hospital

225
00:10:11.225 --> 00:10:13.205
and a clinical environment.

226
00:10:13.705 --> 00:10:17.125
But that group had no expertise in the physics

227
00:10:17.585 --> 00:10:20.245
of airborne transmission transmission outside

228
00:10:20.245 --> 00:10:21.765
of closed clinical environments.

229
00:10:22.365 --> 00:10:24.485
I mean, that's kind of a, one of the other sort

230
00:10:24.485 --> 00:10:26.125
of lessons in the pandemic is,

231
00:10:26.585 --> 00:10:28.725
I'm not criticizing the management of the pandemic.

232
00:10:28.885 --> 00:10:30.685
I certainly would not wanna have that job,

233
00:10:30.945 --> 00:10:34.325
but a lot of assumptions, not just about the best way

234
00:10:34.325 --> 00:10:37.245
to proceed in terms of developing effective strategies,

235
00:10:37.305 --> 00:10:39.525
but the best way to be ethical with respect

236
00:10:39.525 --> 00:10:42.285
to the allocation of scarce resources and research

237
00:10:42.305 --> 00:10:45.365
and all that kind of stuff, was kind

238
00:10:45.365 --> 00:10:47.365
of generalized from clinical settings.

239
00:10:47.465 --> 00:10:50.205
And, you know, there were all sorts of new ethical issues

240
00:10:50.515 --> 00:10:52.325
that emerged during the pandemic

241
00:10:52.475 --> 00:10:54.965
that aren't just a general sort of view of

242
00:10:54.965 --> 00:10:57.005
what was going on in hospitals.

243
00:10:57.225 --> 00:10:58.285
So that, that's two ways.

244
00:10:58.445 --> 00:11:01.245
I think that thinking about evidence more broadly

245
00:11:01.505 --> 00:11:03.725
and thinking in terms of the preponderance

246
00:11:03.725 --> 00:11:07.325
of evidence rather than particular source of evidence,

247
00:11:07.325 --> 00:11:09.845
would've been helpful because we've all been so busy

248
00:11:09.955 --> 00:11:11.765
with the fallout of the pandemic.

249
00:11:11.925 --> 00:11:13.725
I haven't been able to push forward much on this,

250
00:11:13.745 --> 00:11:16.165
but I've been thinking more about misinformation

251
00:11:16.705 --> 00:11:19.445
as an important reason for the erosion

252
00:11:19.585 --> 00:11:21.005
of trust in public health.

253
00:11:21.005 --> 00:11:23.285
That's kind of where I'm focused these days.

254
00:11:23.585 --> 00:11:25.725
But, you know, I wanna say they know that one,

255
00:11:25.895 --> 00:11:28.245
there are many different sources of medical knowledge,

256
00:11:28.265 --> 00:11:29.645
you know, population studies.

257
00:11:30.025 --> 00:11:32.645
One of these medical knowledge can be gleaned in a number

258
00:11:32.905 --> 00:11:33.925
of other ways.

259
00:11:34.275 --> 00:11:36.405
Observation, recording of observations,

260
00:11:36.535 --> 00:11:41.085
basic science reasoning from mechanisms, patient narratives

261
00:11:41.185 --> 00:11:42.965
as a narrative based medicine.

262
00:11:43.345 --> 00:11:47.325
And, information now about a person's own genes

263
00:11:47.325 --> 00:11:49.565
or proteins to prevent, diagnosed

264
00:11:49.865 --> 00:11:52.885
or treat disease as a precision medicine. So we have all

265
00:11:52.885 --> 00:11:54.165
These different kind of different,

266
00:11:54.165 --> 00:11:55.485
medical epistemologies,

267
00:11:55.825 --> 00:11:59.205
and right now it's kind of like a medical sectarianism.

268
00:11:59.205 --> 00:12:02.245
We're all sort of over here and in different places,

269
00:12:02.705 --> 00:12:06.125
and maybe not as usefully connected as they might be,

270
00:12:06.225 --> 00:12:09.405
but it's difficult because we have these,

271
00:12:09.565 --> 00:12:11.165
EBM hierarchies, right?

272
00:12:11.585 --> 00:12:14.085
And so the source of medical knowledge coming from these

273
00:12:14.085 --> 00:12:16.245
other places is kind of set that aside

274
00:12:16.265 --> 00:12:20.845
or thought to be confer, less certainty than randomized,

275
00:12:20.985 --> 00:12:23.605
controlled high, trials down

276
00:12:23.605 --> 00:12:26.765
with the hierarchies has been a man for, for quite a while.

277
00:12:27.105 --> 00:12:30.365
And, you know, Jacobs Senga was a philosopher

278
00:12:30.365 --> 00:12:32.885
of medicine in Cambridge, was at the University of Toronto

279
00:12:32.905 --> 00:12:34.045
for a couple of years,

280
00:12:34.045 --> 00:12:36.325
wrote a fantastic paper with that as the title.

281
00:12:36.785 --> 00:12:39.325
And it's kind of been a rallying cry for people

282
00:12:39.625 --> 00:12:42.805
who are just kind of concerned about the proliferation

283
00:12:42.865 --> 00:12:46.405
and spread, unquestioned spread, EBM.

284
00:12:46.745 --> 00:12:47.965
And so, you know,

285
00:12:48.345 --> 00:12:51.725
and it's really kind of a call to think that for a piece of,

286
00:12:51.865 --> 00:12:54.765
information to pass evidence of muster.

287
00:12:54.885 --> 00:12:56.405
I don't know if that's an actual phrase,

288
00:12:56.465 --> 00:12:58.685
but I'm making it up evidential muster.

289
00:12:58.825 --> 00:13:01.565
Uh, so it's, worthiness, it has

290
00:13:01.565 --> 00:13:03.805
to pass both criteria for evidence,

291
00:13:03.825 --> 00:13:07.005
and it has to be, you know, valid, you know, according

292
00:13:07.105 --> 00:13:10.965
to our research methods, but it also must be appropriate

293
00:13:11.105 --> 00:13:14.445
and relevant and have weight and be socially impactful.

294
00:13:14.625 --> 00:13:16.365
That's another lesson from the Pandemic.

295
00:13:16.495 --> 00:13:20.405
We're all thinking that philosophers like me should be

296
00:13:20.405 --> 00:13:23.965
saying stuff that is socially relevant and impactful.

Reevaluating Evidence-Based Medicine in Pandemics

00:00:00 - I came at this much before the pandemic. Looking at EBM and what's considered to be a narrow focus on, derivations from population studies, looking at legal literature in other areas where evidence of reasoning is really been, studied and theorized, there is standardly a distinction between evidence that it, I mean, we can use the word goodness, you may not wanna use that. That is valid according to standard research methods. And the law would be how one gives a deposition of some sort
00:00:36 - has weight in some sense. And both of those criteria need to be met for something as admitted as evidence in, in, in most legal jurisdictions. So I, I started thinking about EBM from that point of view. And, you know, the article mentions, pithy comment by my deceased brilliant colleague Ian Hacking, who we wrote a lot about statistics and probability that very, you know, evidence in principle is something
00:01:04 - But in fact, as I pointed out, that sometimes it is not persuasive, even evidence that sometimes seems to be gleaned using the best re or what we think of as the best research methods may not be persuasive. And I thought, I wrote this at the very beginning of the pandemic, and it was a time when there was a great deal of debate about routes of transmission for covid and, you know, the effectiveness of some public health measures like social distancing and face masking and the rest of those things.
00:01:35 - of the conversation is a muddled because these two things are being rolled together. What we're looking for here to find our way out of the pandemic is evidence that's good according to the certain criteria, in particular criteria we would associate with the evidence-based medicine movement, the World Health Organization, like hundreds of other organizations bought into EBM, which as you know, privileges, population studies, tries to generate policy sometimes during the pandemic,
00:02:07 - for individual patients based upon the normal, patient or what happened in the study group. So I, I wanted to address that in this paper and led a different framework for thinking about evidence. You know, a BM has been very masterful at putting a books of guidelines and all the rest of this stuff and handbooks, but there is a theory of evidence that underpins, EBM, but it has never been, like spelled out as a theoretical position.
00:02:40 - of evidence should satisfy. One related to the goodness, validity, reliability, depending on your take on that on one side. But on the other side, the relevance, persuasiveness appropriateness of evidence as well. The EBM over a series of publications has kind of refined their view and changed it In some ways. Many people would say not, there haven't been as responsive to concerns about how patient preference values,

Evaluating Evidence in Medicine Practices

00:03:08 - for making clinical recommendations. But there were other concerns as well because, you know, in, in the most recent iteration evidence-based medicine, how to practice and teach it, it says quite explicitly the best research evidence. By best research evidence, we mean valid and clinically relevant research. So the relevance criterion, which I think needs to be sorted out from the, the goodness criterion, they're kind of both in there together. So, you know, good evidence is evidence that's accrued
00:03:42 - And which is kind of an odd thing because medicine is this place with many different research method side by side with one another. Different sorts of movements about what counts as medical knowledge and narrative based medicine, precision medicine, person centered medicine. And the rest of them, there are many more. And you know, EBM kind of hold kind of a Trump position here, but through the creation of the grade system, right? The grade framework, which kind of creates a hierarchy
00:04:14 - of medical knowledge, putting statistically derived information from population studies, RCTs, randomized controlled trials, and you know, meta, meta studies. Those meta studies by the way, only include RCTs. They don't include observational studies or cohort studies or many other ways that we could try to become better informed about any issue at hand. So I don't think there's been much pivoting in the evidence-based movement. It just seems to be rolling along in much the same way.
00:04:43 - to some extent was a wake up call because most public health agencies, as you know, the s general of the United States lost his position in part because he was parroting something that the World Health Organization was saying. You know, we in Canada, our so-called top Doctor didn't lose her position, but there were many people who thought that she should, I'm not saying I'm one of them, but clearly it was just following and World Health organization's footsteps on a number of issues that were pivotable pivotal to thinking about the best strategies to manage a pandemic
00:05:16 - that evidence-based medicine proponents of evidence-based medicine, made very early. I don't suggest in this paper we're talking, that's the background for our conversation today, but in other places, the grade system evaluation gives approval for clinicians to kind of step outside of directly asking a patient about their various preferences, information about the patient based on normal patient, that something called a normal patient that emerges in the context of trials of one sort or other.
00:05:49 - that clinicians have a warrant to, proceed with a strong recommendation without detailed discussion with the patient, which is kind of striking, I think. And, it certainly clashes with the best intentions of person-centered medicine. Right? The other thing, of course, is that the idea of typical patients is really concerning when you think about the heterogeneity of clinical practice. You think about psychiatry for a second, right? You know, if a theory of evidence is being constructed

Challenges in Evidence-Based Medicine and Public Health

00:06:22 - what counts as, best clinical evidence might very well change from field to field. But there's no nuancing in EBM framework. Definitely is a one size fits all. A lot of examples about this, but I'll just focus on two familiar ones and try to stay away from more obscure ones. So we'll go back to the, the face coverings. Refer that word to masking for lots of reasons, because in Canada we had some issues about masks and face coverings and the constitutionality,
00:06:52 - I'm sorry if I'm being political, but you know, they very early in the pandemic, the World Health Organization, a strong proponent of evidence-based medicine, advocated that the public of wearing a face coverings conferred no benefit over and above. So distancing and curbing the transmission of COVID-19 and setting Sweden and a couple of countries aside, most public health officials kind of parroted that message, which became kind of standard at the time.
00:07:25 - of squares with the EBM guideline of what constitutes good or valid evidence examining the issue, directly. And the response in the World Health Organization was, well, okay, there's an absence of evidence that we proponents of EBM would take as evidence. So we'll take this as no evidence that face coverings provide, social benefit, right? So in the absence of RCTs, were just assumed as no evidence, the trials that were had been carried on were kind
00:07:56 - of methodological rigor. And they were focused not on covid, they were focused on other infectious agents, influenza, middle East respiratory syndrome, and sars. And they were focused on source protection, but not source control, which is a d issue altogether. So that's one example I think where kind of thinking that, you know, in a perfect world, we will have a trial to tell us what to do so we can feel safe and certain generalizing to a bigger population where,
00:08:27 - because there were all kinds of stories afoot about field epidemiologists were very early in the pandemic, early 2020, that people were being infected, from contagious people more than a meter apart. So this is tied in with, routes of transmission and airborne transmission. The interesting thing about this is that, I mean, it took the World Health Organization two years to say, airborne transmission and put it on the front page.
00:09:01 - aerosol experts, engineers saying, look, this very disconcerting. So the interesting question is, you know, why was this sort of course a public health experts and aerosol scientists, engineers, why did the who not, listen to those people? And one answer that one finds frequently is because the World Health Organization was on side with some problematic assumptions about

Reevaluating Evidence in Public Health

00:09:31 - But I think the underlying reason is the view about evidence that impacted the world organization. That is kind of the legacy of EBM. Field Epidemiology reports were dismissed by the World Health Organization as proof of airborne transmission on the grounds that the evidence was not relevant. So it was not a weight, it wasn't appropriate. We're not gonna think about this right now. The World Health Organization also relied upon a
00:10:01 - So their group, physicians, I think there were at least 30 of them, they were thinking about spread of an, of a pathogen in a closed, hospital and a clinical environment. But that group had no expertise in the physics of airborne transmission transmission outside of closed clinical environments. I mean, that's kind of a, one of the other sort of lessons in the pandemic is, I'm not criticizing the management of the pandemic. I certainly would not wanna have that job,
00:10:34 - to proceed in terms of developing effective strategies, but the best way to be ethical with respect to the allocation of scarce resources and research and all that kind of stuff, was kind of generalized from clinical settings. And, you know, there were all sorts of new ethical issues that emerged during the pandemic that aren't just a general sort of view of what was going on in hospitals. So that, that's two ways. I think that thinking about evidence more broadly
00:11:03 - of evidence rather than particular source of evidence, would've been helpful because we've all been so busy with the fallout of the pandemic. I haven't been able to push forward much on this, but I've been thinking more about misinformation as an important reason for the erosion of trust in public health. That's kind of where I'm focused these days. But, you know, I wanna say they know that one, there are many different sources of medical knowledge, you know, population studies. One of these medical knowledge can be gleaned in a number
00:11:34 - Observation, recording of observations, basic science reasoning from mechanisms, patient narratives as a narrative based medicine. And, information now about a person's own genes or proteins to prevent, diagnosed or treat disease as a precision medicine. So we have all These different kind of different, medical epistemologies, and right now it's kind of like a medical sectarianism. We're all sort of over here and in different places, and maybe not as usefully connected as they might be,
00:12:09 - EBM hierarchies, right? And so the source of medical knowledge coming from these other places is kind of set that aside or thought to be confer, less certainty than randomized, controlled high, trials down with the hierarchies has been a man for, for quite a while. And, you know, Jacobs Senga was a philosopher of medicine in Cambridge, was at the University of Toronto for a couple of years, wrote a fantastic paper with that as the title.

Reevaluating Evidence in Medicine Post-Pandemic

00:12:39 - who are just kind of concerned about the proliferation and spread, unquestioned spread, EBM. And so, you know, and it's really kind of a call to think that for a piece of, information to pass evidence of muster. I don't know if that's an actual phrase, but I'm making it up evidential muster. so it's, worthiness, it has to pass both criteria for evidence, and it has to be, you know, valid, you know, according
00:13:20 - and relevant and have weight and be socially impactful. That's another lesson from the Pandemic. We're all thinking that philosophers like me should be saying stuff that is socially relevant and impactful.

Faculti AI can make mistakes

Research

Policy

Analysis