NEPC Review: The Effects of Test-Based Retention on Student Outcomes Over Time: Regression Discontinuity Evidence from Florida

Cimpian, Joseph

Other Scholarly Content

NEPC Review: The Effects of Test-Based Retention on Student Outcomes Over Time: Regression Discontinuity Evidence from Florida Public Deposited

Analytics

Citations

Citeable URL: https://scholar.colorado.edu/concern/defaults/hm50ts52x

Abstract

A recent NBER working paper examines Florida’s policy to retain many low-scoring third graders. The report concludes that third-grade retention has immediate positive effects on the following year’s test results, but these effects fade over the next six years, with no effect on graduation. The regression discontinuity methods used to estimate the effects, comparing students immediately above and below the law’s cut-score, are generally good for making causal claims. But there is a serious shortcoming in the design—namely, the law requires that students below the cut-score receive intensive extra services intended to raise their subsequent achievement, and this applies to those retained and those promoted. This means the researchers do not know if these positive outcomes for those below the cut-score were due to the greater likelihood of retention or to the assurance of additional services. Also, two-thirds of students who fall below the cut-off score are nonetheless promoted because they fall into exception categories. Finally, the researchers exacerbate outcome differences by using an Instrumental Variable approach, which attributes the entire above/below-threshold difference to just retained students, effectively making the outcome difference appear more than three times as large. Because the policy stipulates that promoted students below the threshold also receive extra services that promoted students above the threshold do not receive, the IV approach is inappropriate. Even setting aside these problems, the study has extremely limited generalizability, restricted to students at or very near the threshold and directly affected by the policy. These and other problems call into serious question any causal claims of the longer-term effects and any policy utility for the report.

Creator