• Partner With Us
  • Focus Areas
    • Cause Selection
    • Global Health & Wellbeing
      • Abundance & Growth
      • Effective Giving & Careers
      • Farm Animal Welfare
      • Global Aid Policy
      • Global Health & Development
      • Global Health R&D
      • Global Public Health Policy
      • Scientific Research
    • Global Catastrophic Risks
      • Biosecurity & Pandemic Preparedness
      • Forecasting
      • Global Catastrophic Risks Capacity Building
      • Potential Risks from Advanced AI
    • Other Areas
      • History of Philanthropy
  • Grants
  • Research & Updates
    • Blog Posts
    • In the News
    • Research Reports
    • Notable Lessons
  • About Us
    • Grantmaking Process
    • How to Apply for Funding
    • Careers
    • Team
    • Operating Values
    • Stay Updated
    • Contact Us
  • Partner With Us
  • Focus Areas
    • Cause Selection
    • Global Health & Wellbeing
      • Abundance & Growth
      • Effective Giving & Careers
      • Farm Animal Welfare
      • Global Aid Policy
      • Global Health & Development
      • Global Health R&D
      • Global Public Health Policy
      • Scientific Research
    • Global Catastrophic Risks
      • Biosecurity & Pandemic Preparedness
      • Forecasting
      • Global Catastrophic Risks Capacity Building
      • Potential Risks from Advanced AI
    • Other Areas
      • History of Philanthropy
  • Grants
  • Research & Updates
    • Blog Posts
    • In the News
    • Research Reports
    • Notable Lessons
  • About Us
    • Grantmaking Process
    • How to Apply for Funding
    • Careers
    • Team
    • Operating Values
    • Stay Updated
    • Contact Us

Trading Off Upside vs. Track Record

  • Content Type: Blog Posts
Published: January 19, 2012 | by Holden Karnofsky

Note: The Open Philanthropy Project was formerly known as GiveWell Labs. Before the launch of the Open Philanthropy Project Blog, this post appeared on the GiveWell Blog. Uses of “we” and “our” in the below post may refer to the Open Philanthropy Project or to GiveWell as an organization. Additional comments may be available at the original post.

We previously listed our five chief criteria for GiveWell Labs (a new arm of our research process that will be open to any giving opportunity, no matter what form and what sector). This post further discusses the first two of these criteria – “upside” and “high likelihood of success” – and the tradeoff between them.

Upside

We use “upside” to refer to the possibility that a philanthropic project will have a huge/outsized impact. While it’s a good thing to fund projects that have this kind of potential – and while a single hugely successful project can make up for many failures – we also see danger in overthinking the upside of projects.

We don’t have experience with venture capital, but our impression is that in the for-profit world it’s extremely difficult to read the “upside” of different ventures, and thus many investments with “big win” potential are needed for one big win. Not only will many of the most hyped, funded, seemingly limitless-potential companies flop, but many of the biggest success stories won’t necessary look high-potential early on. For example, Twitter began as a frivolous-looking side project of another startup; Facebook’s founder was presenting a relatively unambitious-sounding vision of a college-only network a year after its founding. Another striking example of the danger of overthinking upside is provided by a published debate over AirBNB: Union Square Ventures saw a talented team and successful product, but declined to invest specifically because “I’m just not sure how big it’s going to be.” AirBNB later appears to have become a billion-dollar company.

Our feeling is that this sort of uncertainty extends to philanthropy as well. It’s hard to say in advance how generalizable and widely applicable an innovation is and how big the “latent market” for it is, and it’s also hard to say how an early-stage project will look in a few years. Because of this, we are against putting heavy weight on quantification of potential impact, and in favor of looking for projects with real-world feedback loops, so that they can learn and adapt as they go, hopefully eventually hitting on an approach that works and can be scaled and/or generalized.

Upside vs. likelihood of success

We do feel that there are probably some signs that distinguish high-upside-potential from low-upside-potential projects. The main one that strikes us is the extent to which a project’s goals revolve around development and testing of new knowledge, ideas and approaches, as opposed to repeating what has been done before or relying heavily on particular individual talents.

To the extent that that’s right, there is an inherent tension between upside and likelihood of success. To maximize the former, one looks for things that haven’t been done before; to maximize the latter, one largely looks for things that have worked before. For most of GiveWell’s history, we’ve focused on finding interventions and organizations with reasonably strong track records (though we’ve taken risks and considered upside as well); GiveWell Labs may give us more leeway for even higher-risk, higher-upside projects.

Our basic approach to the tradeoff:

  • We consider likelihood of success much easier to evaluate than upside, and we’d prefer a project with only the former going for it to a project with only the latter going for it. However, we could recommend a project with no track record, high upside and strong performance on our other criteria (strong people, room for more funding, and plans for getting real-world feedback and learning from our results rather than a binary “success/failure” outcome).
  • There may be “best of both worlds” opportunities in the “valley of death.” That is, it may be possible to find cases where there is a new approach whose basic promise has been demonstrated at a small scale, but which still needs more demonstration before it can reach very large scale. It isn’t clear to us whether funders often overlook these opportunities, but if they do, this could result in outstanding opportunities for us and our donors.
Subscribe to new blog alerts
Open Philanthropy
Open Philanthropy
  • We’re Hiring!
  • Press Kit
  • Governance
  • Privacy Policy
  • Stay Updated
Mailing Address
Open Philanthropy
182 Howard Street #225
San Francisco, CA 94105
Email
info@openphilanthropy.org
Media Inquiries
media@openphilanthropy.org
Anonymous Feedback
Feedback Form

© Open Philanthropy 2025 Except where otherwise noted, this work is licensed under a Creative Commons Attribution-Noncommercial 4.0 International License.

We use cookies on our website to give you the most relevant experience by remembering your preferences and repeat visits. By clicking “Accept All”, you consent to the use of ALL the cookies. However, you may visit "Cookie Settings" to provide a controlled consent.
Cookie SettingsAccept All
Manage consent

Privacy Overview

This website uses cookies to improve your experience while you navigate through the website. Out of these, the cookies that are categorized as necessary are stored on your browser as they are essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may affect your browsing experience.
Necessary
Always Enabled
Necessary cookies are absolutely essential for the website to function properly. These cookies ensure basic functionalities and security features of the website, anonymously.
CookieDurationDescription
cookielawinfo-checkbox-analytics11 monthsThis cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Analytics".
cookielawinfo-checkbox-functional11 monthsThe cookie is set by GDPR cookie consent to record the user consent for the cookies in the category "Functional".
cookielawinfo-checkbox-necessary11 monthsThis cookie is set by GDPR Cookie Consent plugin. The cookies is used to store the user consent for the cookies in the category "Necessary".
cookielawinfo-checkbox-others11 monthsThis cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Other.
cookielawinfo-checkbox-performance11 monthsThis cookie is set by GDPR Cookie Consent plugin. The cookie is used to store the user consent for the cookies in the category "Performance".
viewed_cookie_policy11 monthsThe cookie is set by the GDPR Cookie Consent plugin and is used to store whether or not user has consented to the use of cookies. It does not store any personal data.
Functional
Functional cookies help to perform certain functionalities like sharing the content of the website on social media platforms, collect feedbacks, and other third-party features.
Performance
Performance cookies are used to understand and analyze the key performance indexes of the website which helps in delivering a better user experience for the visitors.
Analytics
Analytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics the number of visitors, bounce rate, traffic source, etc.
Advertisement
Advertisement cookies are used to provide visitors with relevant ads and marketing campaigns. These cookies track visitors across websites and collect information to provide customized ads.
Others
Other uncategorized cookies are those that are being analyzed and have not been classified into a category as yet.
SAVE & ACCEPT