Causal Inference in Scale-Up Research
While randomized controlled trials (RCTs) remain the gold standard for identifying causal effects, resource constraints or ethical considerations often make it difficult if not impossible to employ RCTs in education research. DRDC supported PI Schneider in a project sponsored by The Governing Board of the American Educational Research Association (AERA) Grants Program to “prepare a report that would provide researchers and funding agencies with guidelines for evaluating various methods and analytic approaches for drawing causal inferences.” This project was critical to DRDC and broader IERI efforts to enhance educational researchers’ capacity to conduct scientific research on scale-up. Schneider has presented on this issue at several professional meetings, most recently in a debate with Alan Kazdin, President of the American Psychological Association, on the topic “Do evidence-based psychotherapy and education require randomized trials?”
A related program of DRDC-supported research was concerned with the secondary analyses of large-scale datasets to estimate causal effects in education. Such analyses may greatly enhance or provide “alternative methods for making valid causal inferences with observational data.” However limited use is made of methods to adjust for observed and unobserved characteristics when making comparisons across groups using such observational data. DRDC supported a program of research directed by Schneider at MSU to provide guidelines for analyzing national and regional datasets in order to generate hypotheses, replicate findings, inform the development of RCTs, and estimate causal effects in educational settings. She discusses these issues in a presentation to the Midwestern Regional Educational Laboratory (MREL) on "Improving the Effectiveness of Education: The Need for Rigorous Research." Other presentations and publications from this line of research include:
- Schneider, B. (2007). Estimating causal effects using observational and experimental data. Korean Society for the Study of Sociology of Education.
- Wyse, A.E., Keesler, V.A., & Schneider, B. (in press). Assessing the effects of small school size on mathematics and achievement: A propensity score-matching approach. Teachers College Record.