Which Grad Program Will Be The First To Drop the GRE?

This year I am the chair of the admissions committee for the Harvard PhD in Astronomy and Astrophysics. Around the country, committees such as ours are setting to work to select the incoming class. I thought it timely to share some worries I have been having about one required element of our application: the Graduate Record Examinations (GRE) General Test, and the Subject Test in Physics.

My worry stems from three considerations:

First, I worry that the GRE doesn't actually measure the thing we really care about, namely the applicant's likelihood of success at creative and demanding research in astrophysics. So far, my search for such evidence has turned up empty: I can't find a persuasive study showing that the GRE Physics or General test scores provide a key measure of future research success. Yes, the scores do correlate with performance in graduate coursework, but to be honest I don't really care about that as an end in itself. If a student working with me finds Earth2.0, I am prepared to overlook a B- in one of our survey courses. And this isn't just my opinion: When is the last time the Hubble fellowship committee asked to see your transcript?

Second, I worry about the impact of GRE scores on the evaluation of applicants who are women and/or members of underrepresented minority groups. I encourage you to read the recent Nature article "A test that fails" by our colleagues Casey Miller and Keivan Stassun. The authors present data for the Quantitative Reasoning portion of the General Exam showing that women score on average 80 points lower in the physical sciences than men, and African Americans score on average 200 points lower than white people. I have encountered a variety of opinions among members of our community as to why they think this is the case: Some believe these scores accurately reflect the levels of preparation of the students, others thinking it is primarily the result of stereotype threat. My point (and the point made by Miller and Stassun) is that regardless of the cause, any process that effectively introduces a cut based on a minimum score can have a dramatic impact on the diversity of the applicant pool. As noted by the authors: "In the physical sciences, only 26% of women, compared to 73% of men, score above 700 on the GRE Quantitative measure. For minorities, this falls to 5.2%, compared with 82% for white and Asian people." A score of 700 is a typical minimum score used by many graduate programs in the physical sciences. (Yes, many of you may serve on committees that don't employ a strict cut-off. But how many times have you heard a committee member express concern about that low score on the physics GRE...?)

Third, the test is very expensive. Taking the General test costs $195 and the Physics exam costs $150, and each additional score report costs $27 per test. Thus an applicant who takes both tests (as we require) and needs 10 additional score reports is on the hook for $885, and many applicants spend much more if they retake a test. Yes, there is a means to request reduced fees based on financial need, but I worry that the extra burden it imposes -- and more subtle factors like the cost of traveling to a test site, or the support one's institution provides for preparation -- work to discourage some students.

For some time, my colleagues and I have been thinking about doing a retrospective study of graduate student performance to determine which factors (the GRE would just be one we would consider) best predict success in graduate school. This is difficult, because we don't so much want to compare between students in our program as compare students we admitted to ones we didn't admit. But then I would need to correct for the impact of attending a different school, and the different opportunities afforded the student, which seems impossible. (I am eager to hear, either in the comments or by direct email to me, from anyone who has tackled this.)

In summary, I am worried. I worry that many faculty members continue to believe that these tests predict success in a research-intensive program (and an applicant's ultimate ability to succeed as a scientist) but that these beliefs are unfounded. I worry that many committees effectively impose cuts that serve unintentionally to bias strongly the pool of applicants to favor white and Asian men. And I worry that we are asking applicants to spend a lot of money for a test that doesn't inform our decision making, and we are not be seeing applications from students who are deterred by the cost.

In keeping with the suggestions of the article by Miller and Stassun, our committee will be employing interviews and searching for qualities that aren't measured by the GRE, including creativity, grit, adaptability, and perseverance. But we have already gathered GRE scores and surely they will continue to factor into our decision making.

What if GREs really don't provide important information on the potential of an individual to succeed in research, and instead have caused groups of otherwise well-meaning faculty to discriminate against women and minority students in their admission process? What would it take for a department to step forward and actually not ask for GRE scores anymore?

If an evaluator of our admissions process asked why we require the GRE, how would I respond? Could I truthfully say that I had data proving that the scores were worth the cost, both in terms of diversity, and in dollars? Or would the most truthful answer be that this is more akin to a tradition? And perhaps, like so many traditions that have in the past been used to exclude women and minorities from astrophysics, this one is also worth revisiting.