Forecasting the COVID-19 Pandemic

As part of our work on biosecurity and pandemic preparedness, we have contracted Good Judgment Inc. to expand its efforts to aggregate, publish, and track forecasts about the COVID-19 pandemic, with the hope that these forecasts can help improve planning by health security professionals and the broader public, limit the spread of the virus, and save lives.

The initial set of predictions, available here, are aggregated from forecasts by professional “Superforecasters,” who qualified by being in the most accurate 1-2% of forecasters from a large-scale, government-funded series of forecasting tournaments that ran from 2011-2015 (see Superforecasting) and, since then, by being in the top handful of forecasters from Good Judgment’s public forecasting platform, Good Judgment Open.

We may commission additional forecasts related to COVID-19 in the coming months, and we welcome suggestions of well-formed questions for which regularly updated forecasts would be especially helpful to public health professionals and the broader public. If you would like to suggest one or more questions for potential forecasting, please fill out this short form, especially if you are a medical or public health professional, and especially if you know how to state the forecasting question(s) precisely enough that it’s clear how to decide later how the question(s) resolved.

We’ve been funding scientific research and policy analysis on biosecurity and pandemic preparedness for several years and are glad to support the work many of our grantees are already doing to respond to this crisis. We’re continuing to support them and are pursuing other opportunities to help mitigate the effects of this pandemic, which we expect to share more about in the future.

Comments

Thank you for funding this work! Do you know if Good Judgment will publish commentary, analysis or citations behind the numbers?

If you click ‘Show More’ on a particular aggregated forecast, you can see a selection of comments the superforecasters have made about the reasoning behind their individual forecasts.

Sweet, thank you. I recall wondering why that button didn’t seem to work but it does now. :-/ Maybe I pushed it when the area where the details appear was outside my browser window or I didn’t scroll.

Is there a way to view the forecast history for past days?

I tried saving the page ~daily throughout the last week of March using Wayback Machine, but then realized that the estimates were not actually being saved: https://web.archive.org/web/*/https://goodjudgment.io/covid/dashboard/

Hi William,

Yes, you can view a question’s forecast history by clicking “Show More.”

Thanks!

What about an effective therapy? It would be interesting to see the likelihood of that given that it could come much sooner.

Hi Chaitanya,

We’ve been thinking about ways to specify a “timing of an effective therapy” question, but haven’t yet found a satisfying specification. However, there is one such question that Good Judgment Inc. has posted elsewhere (without our help): https://goodjudgment.io/covid-recovery/#1384

Could you help me to understand how this question will be answered/scored?

– How many total cases of COVID-19 in the U.S. will be reported/estimated as of 31 March 2021? –

Will it be based on ‘Confirmed Cases’ as reported by Johns Hopkins? If its not confirmed cases, but rather “Estimated Cases” is there any current estimates for where we stand today? I.e. How many “Estimated Cases” are there in the US today?

Thanks!

Hi Samuel,

There is some (necessary at the time) ambiguity in each question’s resolution criteria about which sources to use to resolve that question at its close date. You can see each questions resolution crtieria by clicking “Show More.” E.g. for the question you named, the resolution criteria paragraph says:

The outcome will be determined based on reporting and estimates provided by Johns Hopkins of total cases of COVID-19 in the U.S. through and including 31 March 2021 as available in April/May 2021 (Johns Hopkins Coronavirus Resource Center, Johns Hopkins Interactive Dashboard, Johns Hopkins Medicine). For an example of reporting and estimates provided by Johns Hopkins for COVID-19 and influenza, please see: Coronavirus Disease 2019 vs. the Flu. If Johns Hopkins does not provide estimates of total cases of COVID-19 in the U.S. as of 31 March 2021 in April/May 2021, this question will be resolved using reporting and estimates from other credible public health sources, starting with the World Health Organization (e.g., WHO Coronavirus disease (COVID-19) outbreak, WHO Seasonal Flu Estimates, CDC COVID-19, CDC Flu Estimates, Harvard Center for Communicable Disease Dynamics). If the reports and/or estimates provided are a range, the high point of the range will be used to determine the outcome of the question. The outcome will be determined based on reporting and estimates inclusive of cases before this question’s launch. For additional background information on the COVID-19 outbreak, see: BBC News, The Lancet, medRxiv COVID-19, medRxiv SARS-CoV-2, Harvard Center for Communicable Disease Dynamics, New York Times.

Ok, thank you. The short answer seems to be that this is a forecast for “Estimated Cases”, not “Confirmed Cases”. Is that correct?

Yes, though if there are not sufficiently compelling estimates, then question resolution may “fall back to” confirmed cases or something close to it.

Leave a comment