Alas, there’s more to earnings than sex, schools, and SAT scores. Since college attendance decisions aren’t randomly assigned, we must control for all factors that determine both attendance decisions and later earnings. These factors include student characteristics, like writing ability, diligence, family connections, and more. Control for such a wide range of factors seems daunting: the possibilities are virtually infinite, and many characteristics are hard to quantify. But Stacy Berg Dale and Alan Krueger came up with a clever and compelling shortcut.- Instead of identifying everything that might matter for college choice and earnings, they work with a key summary measure: the characteristics of colleges to which students applied and were admitted.
Consider again the tale of Uma and Harvey: both applied to, and were admitted to, U – Mass and Harvard. The fact that Uma applied to Harvard suggests she has the motivation to go there, while her admission to Harvard suggests she has the ability to succeed there, just like Harvey. At least that’s what the Harvard admissions office thinks, and they are not easily fooled.- Uma nevertheless opts for a cheaper U-Mass education. Her choice might be attributable to factors that are not closely related to Uma’s earnings potential, such as a successful uncle who went to U-Mass, a best friend who chose U-Mass, or the fact that Uma missed the deadline for that easily won Rotary Club scholarship that would have funded an Ivy League education. If such serendipitous events were decisive for Uma and Harvey, then the two of them make a good match.
Dale and Krueger analyzed a large data set called College and Beyond (C&B). The C&B data set contains information on thousands of students who enrolled in a group of moderately to highly selective U. S. colleges and universities, together with survey information collected from the students at the time they took the SAT, about a year before college entry, and information collected in 1996, long after most had graduated from college. The analysis here focuses on students who enrolled in 1976 and who were working in 1995 (most adult college graduates are working). The colleges include prestigious private universities, like the University of Pennsylvania, Princeton, and Yale; a number of smaller private colleges, like Swarthmore, Williams, and Oberlin; and four public universities (Michigan, The University of North Carolina, Penn State, and Miami University in Ohio). The average (1978) SAT scores at these schools ranged from a low of 1,020 at Tulane to a high of 1,370 at Bryn Mawr. In 1976, tuition rates were as low as $540 at the University of North Carolina and as high as $3,850 at Tufts (those were the days).
Table 2.1 details a stripped-down version of the Dale and Krueger matching strategy, in a setup we call the “college matching matrix.” This table lists applications, admissions, and matriculation decisions for a (made-up) list of nine students, each of whom applied to as many as three schools chosen from an imaginary list of six. Three out of the six schools listed in the table are public (All State, Tall State, and Altered State) and three are private (Ivy, Leafy, and Smart). Five of our nine students (numbers 1, 2, 4, 6, and 7) attended private schools. Average earnings in this group are $92,000. The other four, with average earnings of $72,500, went to a public school. The almost $20,000 gap between these two groups suggests a large private school advantage.
The college matching matrix
Note: Enrollment decisions are highlighted in gray.
The students in Table 2.1 are organized in four groups defined by the set of schools to which they applied and were admitted. Within each group, students are likely to have similar career ambitions, while they were also judged to be of similar ability by admissions staff at the schools to which they applied. Within-group comparisons should therefore be considerably more apples-to-apples than uncontrolled comparisons involving all students.
The three group A students applied to two private schools, Leafy and Smart, and one public school, Tall State. Although these students were rejected at Leafy, they were admitted to Smart and Tall State. Students 1 and 2 went to Smart, while student 3 opted for Tall State. The students in group A have high earnings, and probably come from upper middle class families (a signal here is that they applied to more private schools than public). Student 3, though admitted to Smart, opted for cheaper Tall State, perhaps to save her family money (like our friends Nancy and Mandy). Although the students in group A have done well, with high average earnings and a high rate of private school attendance, within group A, the private school differential is negative: (110 + 100)/2 – 110 = -5, in other words, a gap of -$5,000.
The comparison in group A is one of a number of possible matched comparisons in the table. Group В includes two students, each of whom applied to one private and two public schools (Ivy, All State, and Altered State). The students in group В have lower average earnings than those in group A. Both were admitted to all three schools to which they applied. Number 4 enrolled at Ivy, while number 5 chose Altered State. The earnings differential here is $30,000 (60 – 30 = 30). This gap suggests a substantial private school advantage.
Group C includes two students who applied to a single school (Leafy), where they were admitted and enrolled. Group C earnings reveal nothing about the effects of private school attendance, because both students in this group attended private school. The two students in group D applied to three schools, were admitted to two, and made different choices. But these two students chose All State and Tall State, both public schools, so their earnings also reveal nothing about the value of a private education. Groups C and D are uninformative, because, from the perspective of our effort to estimate a private school treatment effect, each is composed of either all-treated or all-control individuals.
Groups A and В are where the action is in our example, since these groups include public and private school students who applied to and were admitted to the same set of schools. To generate a single estimate that uses all available data, we average the group – specific estimates. The average of -$5,000 for group A and $30,000 for group В is $12,500. This is a good estimate of the effect of private school attendance on average earnings, because, to a large degree, it controls for applicants’ choices and abilities.
The simple average of treatment-control differences in groups A and В isn’t the only well-controlled comparison that can be computed from these two groups. For example, we might construct a weighted average which reflects the fact that group В includes two students and group A includes three. The weighted average in this case is calculated as
0 X -5,000^ + 0 x 30,000^ = 9,000.
By emphasizing larger groups, this weighting scheme uses the data more efficiently and may therefore generate a statistically more precise summary of the private-public earnings differential.
The most important point in this context is the apples-to-apples and oranges-to-oranges nature of the underlying matched comparisons. Apples in group A are compared to other group A apples, while oranges in group В are compared only with oranges. In contrast, naive comparisons that simply compare the earnings of private and public school students generate a much larger gap of $19,500 when computed using all nine students in the table. Even when limited to the five students in groups A and B, the uncontrolled comparison generates a gap of $20,000 (20 = (110 + 100 + 60)/3 – (110 + 30)/2). These much larger uncontrolled comparisons reflect selection bias: students who apply to and are admitted to private schools have higher earnings wherever they ultimately chose to go.
Evidence of selection bias emerges from a comparison of average earnings across (instead of within) groups A and B. Average earnings in group A, where two-thirds apply to private schools, are around $107,000. Average earnings in group B, where two-thirds apply to public schools, are only $45,000. Our within-group comparisons reveal that much of this shortfall is unrelated to students’ college attendance decisions. Rather, the crossgroup differential is explained by a combination of ambition and ability, as reflected in application decisions and the set of schools to which students were admitted.