Category THE PATH FROM CAUSE TO EFFECT

Masters of ’Metrics: Donald Campbell

The RD story was first told by psychologists Donald L. Thistlethwaite and Donald T. Campbell, who used RD in 1960 to evaluate the impact of National Merit Scholarship awards on awardees’ careers and attitudes.- As many of our readers will know, the American National Merit Scholarship program is a multi-round process, at the end of which a few thousand high-achieving high school seniors are awarded a college scholarship. Selection is based on applicants’ scores on the PSAT and SAT tests, the college entrance exams taken by most U. S. college applicants.

Successful candidates in the National Merit competition have PSAT scores above a cutoff (and have their PSAT scores validated by doing well on the SAT, taken later)...

Read More

Ceteris Paribus?

topic: Briefly describe experiences, challenges, and accomplishments that define you as a person.

essay: I am a dynamic figure, often seen scaling walls and crushing ice. I cook Thirty-Minute Brownies in twenty minutes. I am an expert in stucco, a veteran in love, and an outlaw in Peru. On Wednesdays, after school, I repair electrical appliances free of charge.

I am an abstract artist, a concrete analyst, and a ruthless bookie. I wave, dodge, and frolic, yet my bills are all paid. I have won bullfights in San Juan, cliff­diving competitions in Sri Lanka, and spelling bees at the Kremlin. I have played Hamlet, I have performed open-heart surgery, and I have spoken with Elvis.

But I have not yet gone to college.

From an essay by Hugh Gallagher, age 19.

(Hugh later went to New York Universit...

Read More

Beware Bad Control

Perhaps more controls are the answer. Why not control for occupation, for example? Many data sets that report earnings also classify workers’ jobs, such as manager or laborer. Surely occupation is a strong predictor of both schooling and earnings, possibly capturing traits that distinguish Mick and Johan from more average Joes. By the logic of OVB, therefore, we should control for occupation, a matter easily accomplished by including dummy variables to indicate the types of jobs held.

Although occupation is strongly correlated with both schooling and wages, occupation dummies are bad controls in regressions meant to capture causal effects of schooling on wages...

Read More

LATE for Charter School

The KIPP lottery exemplifies an IV chain reaction. The components of such reactions have been named, so masters can discuss them efficiently. We’ve noted that the original
randomizer (in this case, a KIPP offer) is called an instrumental variable or just an instrument for short. As we’ve seen, the link from the instrument to the causal variable of interest (in this case, the effect of lottery offers on KIPP attendance) is called the first – stage, because this is the first link in the chain. The direct effect of the instrument on outcomes, which runs the full length of the chain (in this case, the effect of offers on scores), is called the reduced form. Finally, the causal effect of interest—the second link in the chain—is determined by the ratio of reduced form to first-stage estimates...

Read More

The Oregon Trail

master kan: Truth is hard to understand.

kwai chang caine: It is a fact, it is not the truth. Truth is often hidden, like a shadow in darkness.

Kung Fu, Season 1, Episode 14

The HIE was an ambitious attempt to assess the impact of health insurance on health-care costs and health. And yet, as far as the contemporary debate over health insurance goes, the HIE might have missed the mark. For one thing, each HIE treatment group had at least catastrophic coverage, so financial liability for health-care costs was limited under every treatment. More importantly, today’s uninsured Americans differ considerably from the HIE population: most of the uninsured are younger, less educated, poorer, and less likely to be working...

Read More

Differences-in-Differences

master kan: If while building a house, a carpenter strikes a nail and it proves faulty by bending, does the carpenter lose faith in all nails and stop building? So it is with empirical work.

Kung Fu, Season 1, Episode 7

Our Path

Credible instrumental variables and dramatic policy discontinuities can be hard to find;

you’ll need other ’metrics tools in your kit too. The differences-in-differences (DD) method recognizes that in the absence of random assignment, treatment and control groups are likely to differ for many reasons. Sometimes, however, treatment and control outcomes move in parallel in the absence of treatment. When they do, the divergence of a post­treatment path from the trend established by a comparison group may signal a treatment effect...

Read More