Experimental & Quasi-Experimental Methods

Summary

These are forms of program evaluation that occur after-the-fact. Using a "counterfactual" study design methodology, investors and organizations aim to compare the social impact on a treatment group (traditionally referred to as "beneficiaries") to a group that was isolated from the intervention. This could take the form of many different tests, which the authors in "Measuring the "Impact" in Impact Investing" define as 1) Randomized Control Trials 2) Historical Baselines 3) Pre/Post Tests 4) Regression Discontinuity Design, and 5) Difference in Differences. For a comparison of these different tests, see the attached image below.

Part of Solution

  • How Impact Investors Actually Measure Impact

  • Additional Information

    From pp. 45 in report

    Spotlight: Root Capital

    Root Capital lends to farmer associations and private businesses that help build sustainable livelihoods by aggregating rural producers in Africa and Latin Americas.

    One of Root Capital’s main goals in measuring impact is to test their primary hypothesis that agricultural businesses enable farmers to achieve improved livelihoods. They design studies of selected clients to evaluate whether and how their client agricultural businesses support farmer livelihoods, to verify that they are truly reaching under-served businesses, and to inform their assumptions about what social and environmental practices truly create positive impacts.

    In 2014, Root Capital released their first multi-site impact study - Improving Rural Livelihoods: A Study of Four Guatemalan Coffee Cooperatives39. This study provides a detailed picture of the impact that Root Capital’s client enterprises have on the livelihoods of smallholder framers and the environment, and seeks to answer the question: Does Root Capital’s financing and training enable their clients to increase their impacts, and if so, how and to what extent?

    This study looked at four of Root Capital’s coffee cooperative clients in Guatemala. The research focused on the cooperatives’ roles in promoting farmer livelihoods, and involved surveying 640 farmers. With each cooperative, they recruited a comparison group of farmers living in the same communities to allow the researchers to correlate differences (e.g., in income, access to services, and production practices) with services provided by the cooperatives.

    Photos

    • Flowchart of phases of parallel randomized trial   modified from consort 2010
    • Pasted graphic 19