Fuzzy RD Is IV

In a regression rite of passage, social scientists around the world link student achievement to the average ability of their schoolmates. Such regressions reliably reveal a strong association between the performance of students and the achievement of their peers. Among all Boston exam school applicants, a regression of students’ seventh-grade math scores on the average fourth-grade scores of their seventh-grade classmates generates a coefficient of about one-quarter. This putative peer effect comes from the regression model where Yt is student z’s seventh-grade math score, X{ is z’s fourth-grade math score, and ^(0 is the average fourth-grade math score of z’s seventh-grade classmates (the subscript “(і)” reminds us that student z is not included when calculating the average achievement of his or her peers). The estimated coefficient on peer quality (в2) is around.25, meaning that a one standard deviation increase in the ability of middle school peers, as measured by their elementary school scores and controlling for a student’s own elementary school performance, is associated with a.25o gain in middle school achievement.

Parents and teachers have a powerful intuition that “peers matter,” so the strong positive association between the achievement of students and their classmates rings true. But this naive peer regression is unlikely to have a causal interpretation for the simple reason that students educated together tend to be similar for many reasons. Your authors’ four children, for example, precocious high-achievers like their parents, have been fortunate to attend schools attended by many children from similar families. Because family background is not held fixed in regressions like equation (4.6). the observed association between students and their classmates undoubtedly reflects some of these shared influences. To break the resulting causal deadlock, we’d like to randomly assign students to a range of different peer groups.

Exam schools to the rescue! Figure 4.8 documents the remarkable difference in peer ability that BLS admission produces, with a jump of four-fifths of a standard deviation in peer quality at the BLS cutoff. The jump in peer quality at exam school admissions cutoffs arises—by design—from the mix of students enrolled in selective schools. This is just what the econometrician ordered by way of an ideal peer experiment (this improvement in peer quality also makes many parents hope and dream of an exam school seat for their children). Moreover, while peer quality jumps at the cutoff, cross-cutoff comparisons of variables related to applicants’ own abilities, motivation, and family background—the sources of selection bias we usually worry about—show no similar jumps. For example, there’s no jump in applicants’ own elementary school scores. Peers change discontinuously at admissions cutoffs, but exam school applicants’ own characteristics do not.-

Hopes, dreams, and the results from our naive peer regression (equation (4.6)) notwithstanding, the exam school experiment casts doubt on the notion of a causal peer effect on the achievement of Boston exam school applicants. The seeds of doubt are planted by Figure 4.9. which plots seventh – and eighth-grade math scores (on tests taken
after 1 or 2 years of middle school) against ISEE scores (the exam school running variable) for applicants scoring near the BLS cutoff. Admitted applicants are exposed to a much stronger peer group, but this exposure generates no parallel jump in applicants’ middle school achievement.

As in equation (4.2). the size of the jump in Figure 4.9 can be estimated by fitting an equation like

Уі~ац +pDi+ +ещ. (4.7)

Here, D; is a dummy variable indicating applicants who qualify, while Rt is the running

variable that determines qualification. In a sample of seventh-grade applicants to BLS, where Y{ is a middle school math score as in the figures, this regression produces an estimate of -.02 with a standard error of .10, a statistical zero in our book.



w і

« 1.5







3 £


Eo, o


How should we interpret this estimate of p? Through the lens of the corresponding first stage, of course! Equation (4.71 is the reduced form for a 2SLS setup where the endogenous variable is average peer quality, The first-stage equation that goes with this reduced form is

— + Ф&І 4* T eV* (4-8)

where the parameter ф captures the jump in mean peer quality induced by an exam school offer. This is the jump shown in Figure 4.8. a precisely estimated,80a.

The last piece of our 2SLS setup is the causal relationship of interest, the 2SLS second stage. In this case, the second stage captures the effect of peer quality on seventh – and eighth-grade math scores. As always, the second stage includes the same control variables

as appear in the first stage. This leads to a second-stage equation that can be written

= al + + e2h (4.9}

where A is the causal effect of peer quality, and the variable is the first-stage fitted value produced by estimating equation (4.8V

Note that equation (4.91 inherits a covariate from the first stage and reduced form, the running variable, Rt. On the other hand, the jump dummy, D;, is excluded from the second stage, since this is the instrument that makes the 2SLS engine run. Substantively, we’ve assumed that in the neighborhood of admissions cutoffs, after adjusting for running variable effects with a linear control, exam school qualification has no direct effect on test scores, but rather influences achievement, if at all, solely through peer quality. This assumption is the all-important IV exclusion restriction in this context.

The 2SLS estimate of A in equation (4.9) is -.023 with a standard error of.132.- Since the reduced-form estimate is close to and not significantly different from zero, so is the corresponding 2SLS estimate. This estimate is also far from the estimate of.25o generated by OLS estimation of the naive peer effects regression, equation (4.6V On the other hand, who’s to say that the only thing that matters about an exam school education is peer quality? The exclusion restriction requires us to commit to a specific causal channel. But the assumed channel need not be the only one that matters in practice.

A distinctive feature of the exam school environment besides peer achievement is racial composition. In Boston’s mostly minority public schools, exam schools offer the opportunity to go to school with a more diverse population, where diversity means more white classmates. The court-mandated dismantling of segregated American school systems was motivated by an effort to improve educational outcomes. In 1954, the U. S. Supreme Court famously declared: “Separate educational facilities are inherently unequal,” laying the framework for court-ordered busing to increase racial balance in public schools. Does increasing racial balance indeed boost achievement? Exam schools are relevant to the debate over racial integration because exam school admission sharply increases exposure to white peers. At the same time, we know that if we replace peer quality, xm, with peer proportion white, this too will produce a zero second-stage coefficient, a consequence of the fact that the underlying reduced form is unchanged by the choice of causal channel.

Exam schools might differ in other ways as well, perhaps attracting better teachers or offering more Advanced Placement (college-level) courses than nonselective public schools. Importantly, however, school resources and other features of the school environment that might change at exam school admissions cutoffs seem likely to be beneficial. This in turn suggests that any omitted variables bias associated with 2SLS estimates of exam school peer effects is positive. This claim echoes that made in Chapter 2 regarding the likely direction of OVB in our evaluation of selective colleges. Because omitted variables with positive effects are probably positively correlated with exam school offers, the 2SLS estimate using exam school qualification as an instrument for peer quality is, if anything, too big relative to the pure peer effect we’re after. All the more surprising, then, that this estimate turns out to be zero.

As with any IV story, fuzzy RD requires tough judgments about the causal channels through which instruments affect outcomes. In practice, multiple channels might mediate causal effects, in which case we explore alternatives. Likewise, the channels we measure readily need not be the only ones that matter. The causal journey never ends; new questions emerge continuously. But the fuzzy framework that uses RD to generate instruments is no less useful for all that.

master stevefu: Summarize RD for me, Grasshopper.

grasshopper: The RD design exploits abrupt changes in treatment status that arise when treatment is determined by a cutoff.

master stevefu: Is RD as good as a randomized trial?

grasshopper: RD requires us to know the relationship between the running variable and potential outcomes in the absence of treatment. We must control for this relationship when using discontinuities to identify causal effects. Randomized trials require no such control.

master stevefu: How can you know that your control strategy is adequate?

grasshopper: One can’t be sure, Master. But our confidence in causal conclusions increases when RD estimates remain similar as we change details of the RD model.

master stevefu: And sharp versus fuzzy?

grasshopper: Sharp is when treatment itself switches on or off at a cutoff. Fuzzy is when the probability or intensity of treatment jumps. In fuzzy designs, a dummy for clearing the cutoff becomes an instrument; the fuzzy design is analyzed by 2SLS.

master stevefu: You approach the threshold for mastery, Grasshopper.

Leave a reply

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>