David Roodman, Senior Advisor in GHW Cause Prioritization, recently published an experimental “replication opinion” about a disputed study on whether air temperature affects asylum decisions in the U.S. In his report, Roodman informally adjudicates the dispute and discusses the larger incentive structures that make it difficult for academic journals to effectively resolve similar issues.
Abstract
Heyes and Saberian (2019) finds that U.S. immigration judges are less likely to grant asylum in cases heard on warmer days. Spamann (2022) corrects errors in that paper, enlarges the sample, proposes additional revisions, and strongly challenges the conclusion. In a rejoinder, Heyes and Saberian (2022) incorporates many of these comments, yet maintains that “results…are qualitatively unchanged.” Experimenting with a new academic-literary form, I review the case as a judge might, to offer a take that is more independent and legible than the partisans can offer. I agree with Spamann (2022): the only viable explanation for the combined evidence is publication bias or other forms of result filtration.
You can read the full report here.