Popular in Economics

Central Monetary Authorities

Macroeconomic modelling of CBDC: a critical review

European Central Bank Ulrich Bindseil

Central Monetary Authorities

Financing Private Credit

Federal Reserve Bank of New York Nina Boyarchenko

Economics

Failing Banks

Massachusetts Institute of Technology Emil Verner

Economic History

Golden Fetters

UC Berkeley Barry Eichengreen

Economic Inequalities

Visions of Inequality

City University of New York Branko Milanovic

Economics

School Finance Equalization Increases Intergenerational Mobility

Yale University Barbara Biasi

Economics

Olivier Blanchard on U.S. Pandemic-Era Inflation

Peterson Institute for International Economics Olivier Blanchard

Dos and Don’ts when implementing Randomized Controlled Trials in Developing Countries

View more insights in Social Sciences - Economics - Development Economics

View more figures from Monash University

View the page of Asad Islam

Next in Development Economics

Labor supply responses to rainfall shocks

Pushkar Maitra | Monash University

Dynamic Exports and Labor Markets for Inclusive Growth in Cambodia

Gladys Lopez-Acevedo | World Bank

Randomized Controlled Trials (RCTs) are now widely used in economics, particularly for assessing development programs. Asad Islam discusses the practical aspects of evaluating interventions with RCTs, offering insights, challenges, and best practices.

Image courtesy of interviewee. October 3, 2024

Log-in or Sign-up to Faculti
Currently viewing this subject insight as a guest. You have insight(s) remaining for this month. Login to view 8000+ figures on the platform.

Generating Workspace...

Export as PDF

This video provides an extensive discussion on the importance and methodology of conducting randomized control trials (RCTs) for evaluating development programs. It begins by defining a trial as a tool for evaluating the effectiveness of programs globally, emphasizing the need to select a control group that is similar to the intervention group to avoid selection bias.

Key points include:
1. **Selection and Randomization**: RCTs aim to reduce selection bias by randomly assigning participants to treatment or control groups, ensuring that any differences between groups are minimized.

2. **Program Design and Implementation**: Proper design and implementation are crucial. This video includes determining the target population effectively and ensuring participants are well-informed about the program. Poor implementation can lead to unethical situations and biases.

3. **Attrition and Compliance**: This video discusses challenges like attrition (participants dropping out) and non-compliance, highlighting the importance of following up with participants to measure the program's impact accurately.

4. **Ethical Considerations**: Emphasizing the ethical dimension, the speaker argues that randomization is a fair method for resource allocation but stresses the need to consider community needs and awareness for effective implementation.

5. **Local Context**: This video advocates for involving local researchers to gain insights into the community dynamics and ensure that evaluations are sensitive to the local context.

6. **Integrating Qualitative Research**: The author suggests that combining both experimental and qualitative research methods can provide a more nuanced understanding of program impacts and mechanisms.

7. **Field Engagement**: Finally, the text calls for researchers to engage with communities directly, as understanding the local context is essential for accurate evaluations and effective policy recommendations.

In summary, the text stresses rigorous methodological approaches, ethical considerations, local engagement, and the combination of different research methods to ensure that evaluations of development programs are effective and contextually relevant.

Language:

Export as PDF

Export as PDF
Here are five key points summarizing the main ideas from the video:

1. **Importance of Randomized Control Trials (RCTs)**: The video emphasizes that RCTs are a crucial tool for evaluating development programs, as they help to minimize selection bias by randomly assigning participants to intervention and control groups, ensuring that any observed effects can be attributed to the program itself.

2. **Significance of Experimental Design and Implementation**: Careful planning and execution of the program are essential. This includes ensuring that all intended beneficiaries are informed about and can access the program, as poor implementation can undermine the program’s impact due to reasons like spillover effects and attrition.

3. **Ethical Considerations in Resource Allocation**: RCTs offer an ethical approach to distributing limited resources by ensuring random allocation among eligible individuals, thereby avoiding biases based on characteristics like gender or socioeconomic status. This method promotes fairness in the allocation of benefits.

4. **Role of Local Convideo and Researchers**: Engaging researchers who are familiar with the local convideo is critical for understanding the nuances of program implementation and the socio-economic environment. Collaboration with local researchers can provide valuable insights into the causal mechanisms behind program outcomes.

5. **Combining Quantitative and Qualitative Approaches**: The video suggests that both experimental and non-experimental methods should be used in evaluations to gain a comprehensive understanding of program impacts. Qualitative research can help explore underlying mechanisms and convideoual factors that influence outcomes, leading to better-informed policies.

Export as PDF
Here are some key quotes extracted from the video you , centered around the themes of trial evaluation, program implementation, and the importance of convideo in research:

1. **Purpose of Trials**: "Trial is a tool used for mainly evaluation of any program... to understand the impacts."

2. **Selection Bias**: "The issue that arises is the people who are selecting into the program might be different in many ways that we do not know about them. And therefore, the selection bias that arises due to the non-experimental evaluation of the program."

3. **Randomized Control Trials (RCTs)**: "In randomized control trial, we try to avoid that there are no differences between the group that received the program and control group that are excluded randomly from the program."

4. **Program Design Considerations**: "The first thing is to, of course, you know, the design. And then the second thing is the implementation."

5. **Implementation Challenges**: "Whether you know, the intended beneficiaries, are getting informed about the program."

6. **Ethical Considerations**: "I believe the ethical considerations should be given equal importance for non-experimental program as well."

7. **Communication and Compliance**: "If we can communicate properly about the product, information that we are offering, that will help to reduce the non-compliance and attrition."

8. **Understanding Local Convideo**: "Understanding that local convideo is very important... a research team should not just use the researcher from developing country as a research assistant."

9. **Importance of Local Researchers**: "They should also involve, and the co-author in the research team, because then they will have a stake to understand it better."

10. **Combining Research Approaches**: "We should combine both experimental and non-experimental evaluations in our research."

11. **Field Visits**: "A lot of our city researchers that I know they have written many papers, but they have never visited the field."

12. **Policy Implications**: "Whatever we say has implications in terms of the policy and other things, unless we really understand deeply about the convideo."

These quotes encapsulate the core ideas presented in the video regarding the evaluation of development programs, the importance of appropriate trial design and implementation, and the need for understanding the local convideo to derive meaningful conclusions and recommendations.

X-TIMESTAMP-MAP:LOCAL=00:00:00.000,MPEGTS=900000

00:00.215 --> 00:00.565
Trial.

00:00.735 --> 00:04.885
Trial is a tool used for mainly evaluation

00:05.225 --> 00:06.565
of any program.

00:07.425 --> 00:12.405
Uh, in the context of, many development programs

00:12.635 --> 00:15.445
that are happening in, across the world.

00:15.945 --> 00:18.805
We try to understand, or the impacts,

00:19.145 --> 00:23.285
and in order to understand, we need to find

00:23.885 --> 00:27.605
a p group of people who are similar to the group of people

00:28.185 --> 00:30.165
who receive the benefit or intervention.

00:31.045 --> 00:35.865
And because, you know, if we are not selecting beneficiaries

00:36.385 --> 00:41.105
randomly, the issue that arises is the people who

00:41.765 --> 00:46.625
are selecting into the program might be different in many

00:46.655 --> 00:49.705
ways that we do not know about them.

00:50.325 --> 00:54.425
And therefore, the selection bias that arises due to the

00:55.445 --> 00:58.625
non-experimental evaluation of the program.

00:58.805 --> 01:02.505
And in randomized control trial, we try to avoid

01:03.245 --> 01:05.985
by selecting people randomly so

01:05.985 --> 01:07.385
that there are no differences.

01:07.665 --> 01:09.905
Betweent Pen group that received the program

01:10.365 --> 01:13.705
and control group that are excluded randomly

01:14.215 --> 01:18.185
from the program, key elements depend on the type

01:18.185 --> 01:19.785
of the program or intervention.

01:20.735 --> 01:23.075
But more generally,

01:24.215 --> 01:28.995
we try the first thing to ensure is that the

01:30.705 --> 01:32.185
experimental design,

01:33.085 --> 01:35.415
or in other words, when,

01:35.615 --> 01:39.175
whether the program should be given in a community

01:39.995 --> 01:42.655
or, you know, or to set up people

01:43.315 --> 01:46.335
or, you know, if we are giving a program to classroom,

01:46.715 --> 01:48.375
should be offered to everybody.

01:48.955 --> 01:51.575
So these are the primary consideration of the program.

01:52.555 --> 01:56.055
And, you know, whether given to an individual

01:56.195 --> 02:00.415
or a group of people will largely determine, you know,

02:00.755 --> 02:04.415
how mass, you know, the impact could be

02:04.595 --> 02:07.895
and how many observations we need to sample

02:08.115 --> 02:09.415
to draw any conclusion.

02:09.555 --> 02:11.255
We need to be very careful

02:11.285 --> 02:13.735
because sometimes we think that, you know,

02:13.755 --> 02:15.375
our cities will solve the problem,

02:15.555 --> 02:18.575
but it can induce many other bias

02:18.915 --> 02:22.415
and it will have same kind of outcome

02:22.445 --> 02:25.735
that a non experimental evaluation will have.

02:26.115 --> 02:27.615
So that's why we have to be careful.

02:28.235 --> 02:32.255
The first thing is to, of course, you know, the design.

02:32.675 --> 02:35.175
And then the second thing is the implementation.

02:35.525 --> 02:39.935
Whether you know, the intended beneficiaries,

02:40.685 --> 02:42.135
receiving the, or,

02:42.195 --> 02:44.335
or getting informed about the program.

02:44.685 --> 02:47.375
Sometimes an NGO or a government

02:47.475 --> 02:50.695
or any agency, you know, willing to offer the program,

02:50.955 --> 02:55.735
but the intended participants probably didn't get informed,

02:56.475 --> 02:58.815
or the information about

02:58.835 --> 03:02.325
The product or quality was not given properly.

03:02.825 --> 03:07.325
So we have to make sure the implementation goes really well.

03:08.235 --> 03:12.735
And the, there are no,

03:12.895 --> 03:16.055
participants who we assign them as contract group

03:16.685 --> 03:18.055
receiving the program,

03:18.055 --> 03:21.495
because then that will undermine the impact of the program.

03:22.235 --> 03:24.575
Uh, so that means the spillover effects,

03:24.995 --> 03:28.455
any potential spillover effect that we think we need

03:28.455 --> 03:32.255
to be careful in the design test to minimize the spillover,

03:32.515 --> 03:35.935
the concern for any program evaluation.

03:36.395 --> 03:39.695
It actually, you know, we hear a lot of things that,

03:39.755 --> 03:43.375
you know, in our city, these are more unethical

03:43.435 --> 03:48.135
or, you know, conducting these things, might be,

03:48.475 --> 03:50.495
you know, ethical concerning.

03:50.685 --> 03:55.615
However, I believe the ethical considerations should

03:55.615 --> 03:57.175
be given equal importance

03:57.195 --> 03:59.375
for non-experimental program as well.

03:59.845 --> 04:03.135
Because when we are giving something, we are

04:03.915 --> 04:06.935
of course limited with in terms of the resources.

04:07.435 --> 04:09.575
So we want to allocate resources to the people

04:09.675 --> 04:11.215
who deserve it, right?

04:11.915 --> 04:15.815
But, you know, most of the time we are not able to give

04:15.835 --> 04:17.375
to everyone who deserves it.

04:17.595 --> 04:19.375
So how do you actually allocate it?

04:19.955 --> 04:21.575
And in a randomized contract,

04:21.955 --> 04:24.175
we do it in a more democratic way.

04:24.595 --> 04:25.975
So we do it blindly.

04:26.035 --> 04:29.415
We do not, you know, just people based on the color

04:29.835 --> 04:33.135
or, you know, any kind of gender

04:33.355 --> 04:35.175
or any other characteristics.

04:35.475 --> 04:39.375
We randomly assign people among those who are eligible.

04:40.225 --> 04:43.925
Um, so RCD in fact, in my view,

04:44.385 --> 04:48.805
is an ethical way to offer when resources Palestinian,

04:48.905 --> 04:53.765
in most cases, which we are no resources are always costed.

04:53.795 --> 04:54.845
This is another problem.

04:55.225 --> 04:59.885
The, it might actually induce, the people, some people,

05:00.905 --> 05:05.085
might not be interested in joining the program,

05:05.305 --> 05:07.765
or in receiving the benefit that we are offering.

05:07.935 --> 05:10.485
Again, that's why the implementation is important.

05:11.265 --> 05:15.045
Now, if, if, if we can communicate properly about the

05:15.045 --> 05:17.445
product, information that we are offering,

05:18.115 --> 05:22.165
that will help to reduce the non-compliance and attrition.

05:23.005 --> 05:26.845
'cause people need to be aware about the benefit and,

05:26.945 --> 05:29.805
and, and not, and also knowing that fact

05:29.805 --> 05:32.765
that not everybody has the same level of knowledge

05:32.945 --> 05:35.525
and awareness, as so,

05:36.025 --> 05:39.565
and, it can differ from community to community,

05:39.595 --> 05:41.485
individuals to individuals.

05:41.825 --> 05:46.565
So, to reduce the non-compliance, we must make sure

05:46.755 --> 05:50.965
that the program, implementation really goes well.

05:51.625 --> 05:53.445
And for attrition from,

05:53.615 --> 05:56.725
there are two types of things that can happen. Attrition is

05:57.075 --> 05:59.045
From the survey and also from the program.

05:59.635 --> 06:02.165
Some people we can, can

06:02.845 --> 06:04.285
actually drop out from the program,

06:04.465 --> 06:07.665
but if we can still survey them,

06:08.135 --> 06:11.345
that actually reduces the bias significantly.

06:11.455 --> 06:13.945
However, if the people

06:14.325 --> 06:18.505
who treated the program also are also, the people

06:19.085 --> 06:23.425
who are, not included in the survey, then you,

06:23.485 --> 06:25.865
you know, we do not know whether those are treated

06:26.605 --> 06:28.905
are different from those who remain in the program.

06:29.285 --> 06:30.505
So we want to minimize that.

06:31.005 --> 06:35.065
And so the best approach is if the treat the,

06:35.065 --> 06:38.745
from the program, try to follow them up in the survey.

06:39.285 --> 06:41.825
So then you kind of, know,

06:42.405 --> 06:43.905
the underlying reasons.

06:45.085 --> 06:48.785
But if they are, you know, excluded from both, you know,

06:48.805 --> 06:51.905
survey and from the program, then

06:51.905 --> 06:53.825
that will actually create bias.

06:53.965 --> 06:55.825
And for that we need

06:55.825 --> 06:59.265
to minimize the program implementation teams should be more

06:59.265 --> 07:01.145
careful, but in the end,

07:01.205 --> 07:04.385
if they actually are not observed in the survey

07:04.645 --> 07:08.235
and in the program, then of course we use some,

07:08.615 --> 07:12.715
you know, economatic methodology to try to understand

07:12.815 --> 07:14.195
how large is the attrition

07:14.195 --> 07:16.515
and how, how much bias it can cause.

07:16.975 --> 07:19.115
And there are methods that are applica,

07:19.115 --> 07:23.115
available out there to try to, you know, at least

07:23.215 --> 07:26.235
to estimate the bound of the tip manufac effect,

07:26.235 --> 07:31.195
how much, lower the effect could go,

07:31.775 --> 07:36.235
and how much, you know, the maths, the effect could be,

07:36.815 --> 07:41.075
you know, with, with the, without the presence

07:41.095 --> 07:43.435
of this kind of, attrition that we have.

07:43.755 --> 07:47.235
I wrote this, chapter for Handbook

07:47.235 --> 07:49.155
of Development Economic, handbook

07:49.155 --> 07:50.955
of Experimental Development Economics.

07:50.975 --> 07:54.795
And I emphasize that we should engage with

07:55.675 --> 07:57.915
researcher based in developing countries,

07:58.215 --> 08:00.635
not just developing country researcher like me,

08:00.945 --> 08:03.635
because I am staying in Melbourne, in Australia,

08:03.815 --> 08:05.195
and a lot of us,

08:06.065 --> 08:10.035
because we are away for a long time, are not aware about

08:10.065 --> 08:12.835
what is even thing happening in our community

08:12.845 --> 08:14.155
where we grew up.

08:15.065 --> 08:19.445
Uh, so understanding that local context is very important.

08:20.105 --> 08:22.605
Uh, and a research team should

08:23.425 --> 08:27.285
not just use the researcher from developing country

08:27.585 --> 08:31.135
as a research assistant, that's fine,

08:31.395 --> 08:34.095
but they should also involve,

08:34.315 --> 08:36.695
the researcher based on the ground and,

08:37.355 --> 08:39.655
and the co-author in the research team,

08:39.805 --> 08:44.455
because then they will have a stake to understand it better.

08:45.195 --> 08:48.855
Uh, and also to give more insights

08:49.645 --> 08:54.095
from the field about, things that, you know,

08:54.195 --> 08:55.285
happened.

08:55.545 --> 08:59.925
You know, when we evaluate, we only see the impact,

09:00.395 --> 09:02.005
certain numbers and things,

09:02.465 --> 09:07.245
but how things happened, the causal mechanisms,

09:07.745 --> 09:09.645
we, we cannot establish every

09:10.165 --> 09:12.205
possible challenge and understanding.

09:12.385 --> 09:15.805
And, discussing with them helped

09:15.905 --> 09:18.605
to give us a big picture about

09:18.905 --> 09:23.735
how this program was implemented, why the program actually,

09:24.355 --> 09:27.135
improved certain outcomes

09:27.235 --> 09:29.335
and did not do the other outcomes.

09:29.875 --> 09:32.295
And this, kind of discussion

09:32.355 --> 09:36.255
and understanding, we can happen

09:37.475 --> 09:42.455
better when we have local researcher involved in,

09:42.595 --> 09:43.895
in, in implementation.

09:44.115 --> 09:48.175
And also in, terms of, you know, writing papers,

09:48.695 --> 09:53.415
co-authoring papers, which will help, you know,

09:53.555 --> 09:56.575
for the authors, the other authors

09:57.075 --> 09:59.535
to understand better about the context.

09:59.765 --> 10:03.575
When the, we are writing the, is very important tool

10:03.875 --> 10:04.975
for evaluation

10:05.315 --> 10:08.255
to understand the causal impact of the program.

10:08.875 --> 10:13.135
But we also have to be careful about emphasizing,

10:13.455 --> 10:17.655
a trauma that, you know, without our cities, we probably,

10:18.235 --> 10:20.135
will not get this or that.

10:20.635 --> 10:23.735
Uh, I would say that our city researchers should be open

10:25.125 --> 10:27.725
to qualitative research,

10:28.265 --> 10:33.245
to understand better in terms of the, mechanisms,

10:33.825 --> 10:35.845
to understand the context.

10:36.705 --> 10:39.605
And so we should combine both experimental

10:39.825 --> 10:44.565
and non-experimental evaluations in our research.

10:45.265 --> 10:48.605
Um, a lot of research that we do has a lot

10:48.605 --> 10:53.355
of non-experimental, but we, we go just

10:53.355 --> 10:55.515
after the main results section

10:56.295 --> 10:59.475
and those things are actually sometimes highlighted,

11:00.055 --> 11:04.315
but those is issues are, are not generally

11:05.375 --> 11:07.195
experimentally manipulated.

11:07.775 --> 11:12.355
So, when we are talking about the results,

11:12.375 --> 11:16.035
we should be also careful about interpretation.

11:16.775 --> 11:21.275
And, at the same time when we are, you know,

11:21.615 --> 11:25.555
implementing the program, we need to discuss,

11:26.125 --> 11:27.875
understand the local context.

11:28.255 --> 11:32.355
And if we researcher also are willing

11:32.895 --> 11:37.075
to visit the field, that will be more helpful.

11:37.075 --> 11:39.355
Because a lot of our city research that I know

11:39.865 --> 11:41.395
they have written many papers,

11:42.095 --> 11:43.995
but they have never visited the field.

11:44.145 --> 11:46.155
They neighbor have understood the context.

11:46.865 --> 11:49.875
They are relying on some other third party

11:50.695 --> 11:52.595
to give them some story,

11:53.105 --> 11:55.685
And we should try to minimize that.

11:56.025 --> 11:57.725
I'm not saying that everybody should go,

11:58.185 --> 12:03.145
but at least understanding this issue are very critical

12:03.145 --> 12:06.785
because whatever we say has implication in terms

12:06.785 --> 12:08.065
of the policy and other things,

12:08.605 --> 12:13.145
unless we really understand deeply about the context.

12:13.925 --> 12:16.865
The policies are more complex, all,

12:17.285 --> 12:18.745
and we have to be careful there.

Evaluating Development Programs: The Importance of Randomized Control Trials

00:00:00 - Trial. Trial is a tool used for mainly evaluation of any program. in the context of, many development programs that are happening in, across the world. We try to understand, or the impacts, and in order to understand, we need to find a p group of people who are similar to the group of people who receive the benefit or intervention.

00:00:36 - randomly, the issue that arises is the people who are selecting into the program might be different in many ways that we do not know about them. And therefore, the selection bias that arises due to the non-experimental evaluation of the program. And in randomized control trial, we try to avoid

00:01:05 - that there are no differences. Betweent Pen group that received the program and control group that are excluded randomly from the program, key elements depend on the type of the program or intervention. But more generally, we try the first thing to ensure is that the experimental design, or in other words, when,

Key Considerations for Program Design and Implementation

00:01:39 - or, you know, or to set up people or, you know, if we are giving a program to classroom, should be offered to everybody. So these are the primary consideration of the program. And, you know, whether given to an individual or a group of people will largely determine, you know, how mass, you know, the impact could be and how many observations we need to sample

00:02:09 - We need to be very careful because sometimes we think that, you know, our cities will solve the problem, but it can induce many other bias and it will have same kind of outcome that a non experimental evaluation will have. So that's why we have to be careful. The first thing is to, of course, you know, the design. And then the second thing is the implementation. Whether you know, the intended beneficiaries,

00:02:42 - or getting informed about the program. Sometimes an NGO or a government or any agency, you know, willing to offer the program, but the intended participants probably didn't get informed, or the information about The product or quality was not given properly. So we have to make sure the implementation goes really well. And the, there are no,

Evaluating Ethical Considerations in Program Design and Randomized Control Trials

00:03:16 - receiving the program, because then that will undermine the impact of the program. so that means the spillover effects, any potential spillover effect that we think we need to be careful in the design test to minimize the spillover, the concern for any program evaluation. It actually, you know, we hear a lot of things that, you know, in our city, these are more unethical

00:03:48 - you know, ethical concerning. However, I believe the ethical considerations should be given equal importance for non-experimental program as well. Because when we are giving something, we are of course limited with in terms of the resources. So we want to allocate resources to the people who deserve it, right? But, you know, most of the time we are not able to give

00:04:17 - So how do you actually allocate it? And in a randomized contract, we do it in a more democratic way. So we do it blindly. We do not, you know, just people based on the color or, you know, any kind of gender or any other characteristics. We randomly assign people among those who are eligible. so RCD in fact, in my view, is an ethical way to offer when resources Palestinian,

Addressing Non-Compliance and Attrition in Program Implementation

00:04:53 - This is another problem. The, it might actually induce, the people, some people, might not be interested in joining the program, or in receiving the benefit that we are offering. Again, that's why the implementation is important. Now, if, if, if we can communicate properly about the product, information that we are offering, that will help to reduce the non-compliance and attrition.

00:05:26 - and, and not, and also knowing that fact that not everybody has the same level of knowledge and awareness, as so, and, it can differ from community to community, individuals to individuals. So, to reduce the non-compliance, we must make sure that the program, implementation really goes well. And for attrition from,

00:05:57 - From the survey and also from the program. Some people we can, can actually drop out from the program, but if we can still survey them, that actually reduces the bias significantly. However, if the people who treated the program also are also, the people who are, not included in the survey, then you, you know, we do not know whether those are treated

Minimizing Bias Through Effective Program Follow-Up

00:06:29 - So we want to minimize that. And so the best approach is if the treat the, from the program, try to follow them up in the survey. So then you kind of, know, the underlying reasons. But if they are, you know, excluded from both, you know, survey and from the program, then that will actually create bias. And for that we need to minimize the program implementation teams should be more

00:07:01 - if they actually are not observed in the survey and in the program, then of course we use some, you know, economatic methodology to try to understand how large is the attrition and how, how much bias it can cause. And there are methods that are applica, available out there to try to, you know, at least to estimate the bound of the tip manufac effect, how much, lower the effect could go,

00:07:36 - you know, with, with the, without the presence of this kind of, attrition that we have. I wrote this, chapter for Handbook of Development Economic, handbook of Experimental Development Economics. And I emphasize that we should engage with researcher based in developing countries, not just developing country researcher like me, because I am staying in Melbourne, in Australia,

Importance of Local Context in Research Collaboration

00:08:06 - because we are away for a long time, are not aware about what is even thing happening in our community where we grew up. so understanding that local context is very important. and a research team should not just use the researcher from developing country as a research assistant, that's fine, but they should also involve,

00:08:37 - and the co-author in the research team, because then they will have a stake to understand it better. and also to give more insights from the field about, things that, you know, happened. You know, when we evaluate, we only see the impact, certain numbers and things, but how things happened, the causal mechanisms,

00:09:10 - possible challenge and understanding. And, discussing with them helped to give us a big picture about how this program was implemented, why the program actually, improved certain outcomes and did not do the other outcomes. And this, kind of discussion and understanding, we can happen better when we have local researcher involved in,

Balancing Qualitative and Quantitative Research in Program Evaluation

00:09:44 - And also in, terms of, you know, writing papers, co-authoring papers, which will help, you know, for the authors, the other authors to understand better about the context. When the, we are writing the, is very important tool for evaluation to understand the causal impact of the program. But we also have to be careful about emphasizing,

00:10:18 - will not get this or that. I would say that our city researchers should be open to qualitative research, to understand better in terms of the, mechanisms, to understand the context. And so we should combine both experimental and non-experimental evaluations in our research.

00:10:48 - of non-experimental, but we, we go just after the main results section and those things are actually sometimes highlighted, but those is issues are, are not generally experimentally manipulated. So, when we are talking about the results, we should be also careful about interpretation.

Importance of Local Context in Program Implementation

00:11:21 - implementing the program, we need to discuss, understand the local context. And if we researcher also are willing to visit the field, that will be more helpful. Because a lot of our city research that I know they have written many papers, but they have never visited the field. They neighbor have understood the context.

00:11:50 - to give them some story, And we should try to minimize that. I'm not saying that everybody should go, but at least understanding this issue are very critical because whatever we say has implication in terms of the policy and other things, unless we really understand deeply about the context. The policies are more complex, all,

00:12:17 -

Faculti AI can make mistakes

Research

Policy

Analysis