Description
This course covers the basic principles, concepts, and approaches in doing an impact evaluation of any projects, programs, policies, or interventions. This is also beginner-friendly and is designed for people without any background in impact evaluation or even with the basic statistical tools and approaches used in it. By nature, this course puts more emphasis on the use of statistical approaches in impact evaluation (rather than the conventional economic and financial tools), since these are more widely used and are streamlined by prominent international institutions like the Asian Development Bank and the World Bank. Participants will be introduced to all these approaches; however, due to the nature and complexities of some of the methods, this course will mainly focus on some of the commonly used approaches and will be further utilized and applied during the workshops. Participants will also be introduced to the fundamental statistical concepts and the use of industry-level and open-source software. At the end of this course, participants are expected to develop an impact evaluation proposal and apply the concepts and approaches in the design and implementation.
Course Outline
MODULE 1: BASIC CONCEPTS OF IMPACT EVALUATION
Lesson 1: Introduction to Impact Evaluation: What, Why, and When
1.1 What is Impact Evaluation?
1.2 Impact Evaluation in the Project Management Cycle
1.3 The Difference between Monitoring and Evaluation
1.4 Why conduct an Impact Evaluation?
1.5 When to do an Impact Evaluation?
1.6 Prospective vs Retrospective Impact Evaluation
1.7 Qualitative vs Quantitative Impact Evaluation
1.8 Issues Regarding Impact Evaluation
Lesson 2: How to Conduct an Impact Evaluation: Getting Started
2.1 Overview of the Initial Steps to Implement an IE
2.2 Step 1: Defining the Program and the Outcomes of Interest
2.3 Step 2: Forming a Theory of Change to Refine the Evaluation Questions
2.4 Step 3: Depicting a Theory of Change in a Results Chain
2.5 Step 4: Specifying Evaluation Questions
2.6 Step 5: Selecting Outcome and Performance Indicators
2.7 Toy Example: Evaluating the Impact of a New Math Curriculum
MODULE 2: OVERVIEW OF IMPACT EVALUATION METHODS
Lesson 3: Key Concepts in Impact Evaluation
3.1 Mathematical and Graphical Concept of Impact Evaluation
3.2 Identifying Control and Comparison Groups
3.3 Unit of Assignment, Treatment, and Analysis
3.4 Different Impact Measures
3.5 Biases and Challenges for Causal Inference
3.6 Unintended Behavioral Effects
Lesson 4: Fundamental Concepts of Impact Evaluation Methods
4.1 Impact Evaluation Methods
4.2 Randomized Control Trials (RCT)
4.3 Synthetic Control Method (SCM)
4.4 Instrumental Variables (IV)
4.5 Regression Discontinuity Design (RDD)
4.6 Difference in Differences (DiD)
4.7 Propensity Score Matching (PSM)
MODULE 3: INTRODUCTION TO STATISTICS AND RSTUDIO
Lesson 5: Basic Statistical Concepts
5.1 What is Statistics?
5.2 Branches of Statistics
5.3 Population vs Sample
5.4 Sampling
5.5 Data Collection
5.6 Variables, Data, and Data Types
5.7 Levels of Measurement
Lesson 6: Descriptive and Inferential Statistics
6.1 Descriptive Statistics: Descriptive Measures
6.2 Charts and Graphs
6.3 Inferential Statistics: Hypothesis Testing
6.4 Parametric and Non-parametric Tests
6.5 Basics of Correlation Analysis
6.6 Basics of Regression Analysis
Lesson 7: Basics of R and RStudio
7.1 What is R and RStudio?
7.2 Installing R and RStudio
7.3 RStudio Interface and Key Features
7.4 Basic R Syntax and R Scripts
7.5 Objects in R
7.6 Variables, Data Types, and Operators
Lesson 8: Basic Data Management and Analysis in RStudio
8.1 Data Management in R
8.2 Data Analysis in R
8.3 Data Visualization
8.4 Publication Quality Results Formatting
MODULE 4: DATA COLLECTION USING DIGITAL PLATFORMS
Lesson 9: Introduction to Open Data Kit and Kobo Toolbox
9.1 Introduction to Digital Data Collection
9.2 Digital Data Collection Platforms
9.3 Creating Digital Forms
9.3.1 Using ODK Form Builder
9.3.2 Using Kobo Toolbox Form Builder
9.3.3 Using Microsoft Excel
9.4 Server Configuration and Testing
MODULE 5: APPLICATION OF PSM AND DID USING RSTUDIO
Lesson 10: Impact Evaluation using Propensity Score Matching (PSM)
10.1 Recall: PSM
10.2 Getting to Know the Data
10.3 Step 0: Data Preparation
10.4 Step 1: Make a Data frame for PSM
10.5 Step 2: Make another Data frame for the Observables
10.6 Step 3: Do a Balancing Test (Pre-matching)
10.7 Step 4: Perform Matching (using Nearest Neighbor Algorithm)
10.8 Step 4.1: (Optional) Export the Matched Units
10.9 Step 5: Do a Balancing Test (Post-matching)
10.10 Step 6: Estimate the Impact
10.11 Step 6.1: (Optional) Run PSM Model with Covariates
10.12 Step 6.2: (Optional) Use Other Matching Algorithms
Lesson 11: Impact Evaluation using Difference-in-Differences (DiD)
11.1 Recall: DiD
11.2 Getting to Know the Data
11.3 Step 0: Data Preparation
11.4 Step 1: Make a Data Frame for DiD
11.5 Step 2: Convert the Data Frame to Long or Wide Format
11.6 Step 3: Check for Parallel Trends
11.7 Step 3.1: (Optional) Estimate the Impact (through Manual Calculation)
11.8 Step 4: Estimate the Impact (through Regression)
References:
Books:
1. Gertler, P J., Martinez, S., Premand, P., Rawlings, L. B., and Vermeersch, C. M. (2016). Impact Evaluation in Practice, Second Edition. Washington, DC: Inter-American Development Bank and World Bank.
2. Glewwe, P. and Todd, P. (2022) Impact Evaluation in International Development: Theory, Methods, and Practice. Washington, DC: Inter-American Development Bank and World Bank.
3. Goss-Sampson, M. A. (2022). Statistical Analysis in JASP – A Guide for Students.
4. Gujarati, D. N., Porter, D. C. (2008). Basic Econometrics 5th Edition. McGraw-Hill
5. Ho, R. (2006). Handbook of Univariate and Multivariate Data Analysis and Interpretation with SPSS. Chapman and Hall/CRC
6. Isotalo, Jarkko (n.d.) Basic Statistics.
7. Khandker, S., G. Koolwal, and H. Samad. (2010) Handbook on Impact Evaluation: Quantitative Methods and Practices. Washington, DC: World Bank.
8. White, H. and Raitzer, D. (2017) Impact Evaluation of Development Interventions: A Practical Guide. Manila, Philippines: Asian Development Bank.
9. Wickham, H., Cetinkaya-Rundel, M., Grolemund, G. (2024). R for Data Science 2nd Edition.
Others (Web/Online):
1. Beginning Statistics (2012). https://2012books.lardbucket.org/books/beginning-statistics/
2. Jung, C. Nomad, (2011). Mobile data collection systems a review of the current state of the field. Retrieved from https://www.alnap.org/help-library/mobile-data-collection-systems-a-review-of-the-current-state-of-the-field
3. Mobile Data Collection. Better Evaluation – Global Evaluation Initiative. Retrieved from https://www. betterevaluation.org/ methods- approaches/ methods/ mobile-data-collection
4. Nahhas, R. W. (2024). An Introduction to R for Research. Retrieved from https://bookdown.org/rwnahhas/IntroToR/
5. R Charts. Retrieved from https://r-charts.com/
6. Sjoberg DD, Whiting K, Curry M, Lavery JA, Larmarange J. Reproducible summary tables with the gtsummary package. The R Journal 2021;13:570–80. https://doi.org/10.32614/RJ-2021-053.
7. https://www.statisticshowto.com/probability-and-statistics/how-to-use-slovins-formula/
8. https://www.statisticshowto.com/probability-and-statistics/find-sample-size/#Cochran
9. https://getodk.org/
10. https://www.kobotoolbox.org/about-us/software/
Other Resources:
Do you want to further enrich your learning? You may also enroll in the Google Classroom of this course to assess your progress and learning AT NO COST. Feel free to e-mail me!
Comments
Post a Comment