Statistical power for the comparative regression discontinuity design with a nonequivalent comparison group

Abstract

In the "sharp" regression discontinuity design (RD), all units scoring on one side of a designated score on an assignment variable receive treatment, whereas those scoring on the other side become controls. Thus the continuous assignment variable and binary treatment indicator are measured on the same scale. Because each must be in the impact model, the resulting multi-collinearity reduces the efficiency of the RD design. However, untreated comparison data can be added along the assignment variable, and a comparative regression discontinuity design (CRD) is then created. When the untreated data come from a non-equivalent comparison group, we call this CRD-CG. Assuming linear functional forms, we show that power in CRD-CG is (a) greater than in basic RD; (b) less sensitive to the location of the cutoff and the distribution of the assignment variable; and that (c) fewer treated units are needed in the basic RD component within the CRD-CG so that savings can result from having fewer treated cases. The theory we develop is used to make numerical predictions about the efficiency of basic RD and CRD-CG relative to each other and to a randomized control trial. Data from the National Head Start Impact study are used to test these predictions. The obtained estimates are closer to the predicted parameters for CRD-CG than for basic RD and are generally quite close to the parameter predictions, supporting the emerging argument that CRD should be the design of choice in many applications for which basic RD is now used.