Invitation to contribute to early indicators library

November 21, 2019
News

Dear Colleagues,

I am writing to invite you to contribute to a new library of early indicators to predict the long-term impacts of social science interventions, drawing on data from previous experiments you may have analyzed. Specifically, we invite you to either (1) identify early indicators in your own data using a method we've recently developed (described below) or (2) collaborate with us to estimate the long-term impacts of interventions you have studied by linking individual-level data from your intervention to administrative panel data our team has constructed on earnings. We may be able to offer research assistance in both cases.

The Problem: A major challenges in estimating interventions' impacts in social science is that there are long delays in observing long-term outcomes of interest (e.g., impacts of childhood interventions on lifetime earnings). Our aim is to overcome this challenge by building a library of early indicators to predict a variety of long-term outcomes, ranging from earnings to mortality to educational outcomes.

 Our Approach: Our approach is motivated by an NBER working paper just released by Athey, Chetty, Imbens, and Kang (2019), who developed a simple method of combining early indicators into “surrogate indices” – predicted values of long-term outcomes given early indicators -- to predict long-term treatment effects. That paper presents an empirical application showing that the long-term (nine-year) impacts of a job-training program in California can be accurately estimated within 1.5 years using short-term employment indicators as surrogates. A short summary of the study is attached.

We aim to expand this analysis by identifying surrogate indices that perform well in predicting long-term impacts in a variety of other randomized experiments that researchers have conducted. Our team at Opportunity Insights will then collect and synthesize information about which sets of surrogate outcomes predict long-term impacts on a publicly available website. Our hope is that future researchers and policy makers will then be able to refer to this database of early indicators for policy evaluation, expediting the research cycle.

How You Can Contribute: There are two paths through which we invite interested researchers to contribute to the library. The first path is relevant if you have previously estimated both short-term and long-term outcomes in an RCT. In this case, you can contribute to the public library by estimating a surrogate index that could be used to predict long-run outcomes, given the short-run outcomes you observed (as in Athey et al. 2019). We can offer research assistance to facilitate estimating this surrogate index. 

The second path is to construct new estimates of the long-term impacts of interventions you have previously studied by linking data from your RCT to administrative data our team has constructed covering the U.S. population at the individual level. This path is relevant if you have previously studied short-term impacts in an RCT, and believe that long-term impacts could be identified using administrative data on earnings. Note that this route requires an approval process to take new data into the Census Bureau, and hence will be feasible for a small set of projects related to earnings and upward income mobility, which is our group’s primary focus.

The success of this effort hinges on having a large set of studies to draw from, so we very much hope you will consider participating -- both to create a public good and as a vehicle to amplify the impact of your own work. If you or your colleagues/students would like to participate in this effort or have any questions, please email Jimmy Stratton <jstratton@opportunityinsights.org> and Emanuel Schertz <emanuelschertz@g.harvard.edu>. 

News Type: 
Category: