One of our core values is our tolerance for philanthropic “risk.” Our overarching goal is to do as much good as we can, and as part of that, we’re open to supporting work that has a high risk of failing to accomplish its goals. We’re even open to supporting work that is more than 90% likely to fail, as long as the overall expected value is high enough.

And we suspect that, in fact, much of the best philanthropy is likely to fail. We suspect that high-risk, high-reward philanthropy could be described as a “hits business,” where a small number of enormous successes account for a large share of the total impact — and compensate for a large number of failed projects.

If this is true, I believe it calls for approaching our giving with some counterintuitive principles — principles that are very different from those underlying our work on GiveWell. In particular, if we pursue a “hits-based” approach, we will sometimes bet on ideas that contradict conventional wisdom, contradict some expert opinion, and have little in the way of clear evidential support. In supporting such work, we’d run the risk of appearing to some as having formed overconfident views based on insufficient investigation and reflection.

In fact, there is reason to think that some of the best philanthropy is systematically likely to appear to have these properties. With that said, we think that being truly overconfident and underinformed would be extremely detrimental to our work; being well-informed and thoughtful about the ways in which we could be wrong is at the heart of what we do, and we strongly believe that some “high-risk” philanthropic projects are much more promising than others.

This post will:

  • Outline why we think a “hits-based” approach is appropriate.
  • List some principles that we think are sound for much decision-making, but — perhaps counterintuitively — not appropriate for hits-based giving.
  • List principles that we think are helpful for making sure we focus on the best possible high-risk opportunities.

There is a natural analogy here to certain kinds of for-profit investing, and there is some overlap between our thinking and the ideas Paul Graham laid out in a 2012 essay, Black Swan Farming.

The basic case for hits-based giving

Conceptually, we’re focused on maximizing the expected value of how much good we accomplish. It’s often not possible to arrive at a precise or even quantified estimate of expected value, but the concept is helpful for illustrating what we’re trying to do. Hypothetically, and simplifying quite a bit, we would see the following opportunities as equally promising: (1) a $1 million grant that would certainly prevent exactly 500 premature deaths; (2) a $1 million grant that would have a 90% chance of accomplishing nothing and a 10% chance of preventing 5000 premature deaths. Both would have an expected value of preventing 500 premature deaths. As this example illustrates, an “expected value” focus means we do not have a fundamental preference for low-risk philanthropy or high-risk, potentially transformative philanthropy. We can opt for either, depending on the details. As a side note, most other funders we‘ve met have strong opinions on whether it’s better to take big risks or fund what’s reliable and proven; we may be unusually agnostic on this question.

That said, I see a few basic reasons to expect that an “expected value” approach will often favor high-risk and potentially transformative giving.

1. History of philanthropy. We previously gave a high-level overview of some major claimed successes for philanthropy. Since then, we’ve investigated this topic further via our History of Philanthropy project, and we expect to publish an updated summary of what we’ve learned by the end of 2016. One of our takeaways is that there are at least a few cases in which a philanthropist took a major risk — funding something that there was no clear reason to expect to succeed — and ended up having enormous impact, enough to potentially make up for many failed projects.

Here are some particularly vivid examples (note that these focus on magnitude of impact, rather than on whether the impact was positive):

  • The Rockefeller Foundation invested in research on improving agricultural productivity in the developing world, which is now commonly believed to have been the catalyst for a “Green Revolution” that Wikipedia states is “credited with saving over a billion people from starvation.” (The Wikipedia article discusses the role of the Rockefeller Foundation, as does this post on the HistPhil blog, which is supported by the Open Philanthropy Project.)
  • In The Birth of the Pill, Jonathan Eig credits philanthropist and feminist Katharine McCormick — advised by Margaret Sanger — with being the sole funder of crucial early-stage research leading to the development of the combined oral contraceptive pill, now one of the most common and convenient birth control methods.
  • In The Rise of the Conservative Legal Movement, Prof. Steve Teles argues that conservatives put a great deal of funding into long-term, high-risk goals with no way of predicting their success. He also argues that their ultimate impact was to profoundly change the way the legal profession operates and the general intellectual stature of political conservatism.

If accurate, these stories would imply that philanthropy — and specifically, philanthropy supporting early-stage research and high-risk projects — played a major role in some of the more significant developments of the last century.1 A philanthropic “portfolio” containing one of these projects, plus a large number of similar failed projects, would probably have a very strong overall performance, in terms of impact per dollar.

2. Comparative advantage. When trying to figure out how to give as well as possible, one heuristic to consider is, “what are philanthropists structurally better-suited (and worse-suited) to do compared with other institutions?” Even major philanthropists tend to have relatively less funding available than governments and for-profit investors, but philanthropists are far less constrained by the need to make a profit or justify their work to a wide audience. They can support work that is very “early,” such as new and unproven ideas or work that is likely to take many decades to have an impact. They can support a number of projects that fail in order to find the ones that succeed. They can support work that requires great depth of knowledge to recognize as important and is hard to justify to a wide audience. All of these things seem to suggest that when philanthropists are funding low-probability, high-upside projects, they’re doing what they do best, relative to other institutions.

3. Analogy to for-profit investing. Many forms of for-profit investing, such as venture capital investing, are “hits businesses.” For some description of this, see the article I mentioned previously. Philanthropy seems similar in some relevant ways to for-profit investing: specifically, it comes down to figuring out how to allocate a set amount of funds between projects that can have a wide variety of outcomes. And many of the differences between for-profit investing and philanthropy (as discussed above) seem to imply that hits-based giving is even more likely to be appropriate than hits-based investing.

“Anti-principles” for hits-based giving

This section discusses principles that we think are sound for much decision-making, but not appropriate for hits-based giving. For clarity, these are phrased as “We don’t _____” where _____ is the principle we see as a poor fit with this approach.

A common theme in the items below is that for a principle to be a good fit, it needs to be compatible with the best imaginable giving opportunities — the ones that might resemble cases listed in the previous section, such as the Green Revolution. Any principle that would systematically discourage the biggest hits imaginable is probably not appropriate for hits-based giving, even if it is a good principle in other contexts.

We don’t: require a strong evidence base before funding something. Quality evidence is hard to come by, and usually requires a sustained and well-resourced effort. Requiring quality evidence would therefore be at odds with our interest in neglectedness. It would mean that we were generally backing ideas that others had already explored and funded thoroughly — which would seem to decrease the likelihood of “hit”-sized impact from our participation. And some activities, such as funding work aiming to influence policy or scientific research, are inherently hard to “test” in predictively valid ways. It seems to me that most past cases of philanthropic “hits” were not evidence-backed in the sense of having strong evidence directly predicting success, though evidence probably did enter into the work in less direct ways.

We don’t: seek a high probability of success. In my view, strong evidence is usually needed in order to justifiably assign a high probability to having a reasonably large positive impact. As with venture capital, we need to be willing to back many failures per success — and the successes need to be big enough to justify this.

We don’t: defer to expert opinion or conventional wisdom, though we do seek to be informed about them. Similar to the above point, following expert opinion and conventional wisdom is likely to cut against our goal of seeking neglected causes. If we funded early groundwork for changing expert opinion and/or conventional wisdom on an important topic, this would be a strong candidate for a “hit.” We do think it would be a bad sign if no experts (using the term broadly to mean “people who have a great deal of experience engaging with a given issue”) agreed with our take on a topic, but when there is disagreement between experts, we need to be willing to side with particular ones. In my view, it’s often possible to do this productively by learning enough about the key issues to determine which arguments best fit our values and basic epistemology.

We don’t: avoid controversial positions or adversarial situations. All else equal, we would rather not end up in such situations, but making great effort to avoid them seems incompatible with a hits-based approach. We’re sympathetic to arguments of the form, “You should be less confident in your position when intelligent and well-meaning people take the opposite side” and “It’s unfortunate when two groups of people spend resources opposing each other, resulting in no net change, when they instead could have directed all of their resources to something they agree on, such as directly helping those in need.” We think these arguments give some reason to prefer GiveWell’s top charities. But we feel they need to be set aside when aiming for “hits.”

We feel many “hits” will involve getting a multiplier on our impact by changing social norms or changing key decision-makers’ opinions. And our interest in neglectedness will often point us to issues where social norms, or well-organized groups, are strongly against us. None of the “hits” listed above were without controversy. Note that the combined oral contraceptive is an example of something that was highly controversial at the time (leading, in my view, to the necessary research being neglected by government and other funders) and is now accepted much more broadly; this, to me, is a key part of why it has been such a momentous development.

We don’t: expect to be able to fully justify ourselves in writing. Explaining our opinions in writing is fundamental to the Open Philanthropy Project’s DNA, but we need to be careful to stop this from distorting our decision-making. I fear that when considering a grant, our staff are likely to think ahead to how they’ll justify the grant in our public writeup and shy away if it seems like too tall an order — in particular, when the case seems too complex and reliant on diffuse, hard-to-summarize information. This is a bias we don’t want to have. If we focused on issues that were easy to explain to outsiders with little background knowledge, we’d be focusing on issues that likely have broad appeal, and we’d have more trouble focusing on neglected areas.

A good example is our work on macroeconomic stabilization policy: the issues here are very complex, and we’ve formed our views through years of discussion and engagement with relevant experts and the large body of public argumentation. The difficulty of understanding and summarizing the issue is related, in my view, to why it is such an attractive cause from our perspective: macroeconomic stabilization policy is enormously important but quite esoteric, which I believe explains why certain approaches to it (in particular, approaches that focus on the political environment as opposed to economic research) remain neglected.

Process-wise, we’ve been trying to separate our decision-making process from our public writeup process. Typically, staffers recommend grants via internal writeups. Late in our process, after decision-makers have approved the basic ideas behind the grant, other staff take over and “translate” the internal writeups into writeups that are suitable to post publicly. One reason I’ve been eager to set up our process this way is that I believe it allows people to focus on making the best grants possible, without worrying at the same time about how the grants will be explained.

A core value of ours is to be open about our work. But “open” is distinct from “documenting everything exhaustively” or “arguing everything convincingly.” More on this below.

We don’t: put extremely high weight on avoiding conflicts of interest, intellectual “bubbles” or “echo chambers.” There will be times when we see a given issue very differently from most people in the world, and when the people we find most helpful on the issue will be (not coincidentally) those who see the issue similarly. This can lead to a risk of putting ourselves in an intellectual “bubble” or “echo chamber,” an intellectually insulated set of people who reinforce each others’ views, without bringing needed alternative perspectives and counterarguments.

In some cases, this risk may be compounded by social connections. When hiring specialists in specific causes, we’ve explicitly sought people with deep experience and strong connections in a field. Sometimes, that means our program officers are friends with many of the people who are best suited to be our advisors and grantees.

Other staff, including myself, specialize in choosing between causes rather than in focusing on a specific cause. The mission of “choosing between causes to do the most good possible” is itself an intellectual space with a community around it. Specifically, many of our staff — including myself — are part of the effective altruism community, and have many social ties in that community.

As a result, it sometimes happens that it’s difficult to disentangle the case for a grant from the relationships around it. For an example, see our extensive relationship disclosures around our 2015 grant to the Future of Life Institute.

When these situations occur, there’s a greatly elevated risk that we aren’t being objective, and aren’t weighing the available evidence and arguments reasonably. If our goal were to find the giving opportunities most strongly supported by evidence, this would be a major problem. But the drawbacks for a “hits-based” approach are less clear, and the drawbacks of too strongly avoiding these situations would, in my view, be unacceptable.

To use myself as an example:

  • My strong interest in effective altruism and impact-focused giving has led me to become friends — and live in the same house — with similarly interested people.
  • I spend a lot of time with the people I have found to most strongly share my values and basic epistemology, and to be most interesting and valuable as intellectual peers.
  • If I had a policy of asking my friends to recuse themselves from advising me or seeking support from the Open Philanthropy Project, this would mean disallowing input from some of the people whose opinions I value most.
  • Under a “hits-based” approach, we can expect the very few best projects to account for much (or most) of our impact. So disallowing ideas from some of the people who most closely share our values could dramatically lower the expected value of our work.

This issue is even more pronounced for some of our other staff members, since the staffers who are responsible for investigating funding opportunities in a given area tend to be the ones with the deepest social connections in the relevant communities.

To be clear, I do not believe we should ignore the risks of intellectual “bubbles” or conflicts of interest. To mitigate these risks, we seek to (a) always disclose relevant connections publicly; (b) always make a strong active effort to seek out alternative viewpoints before making decisions, including giving strong consideration to the best counterarguments we can identify; (c) aim for key staff members to understand the most important issues themselves, rather than relying on the judgment of friends and advisors, to the extent that this is practical; (d) always ask ourselves how our relationships might be distorting our perception of a situation; (e) make sure to seek input from staff who do not have relevant conflicts of interest or social relationships.

But after doing all that, there still will be situations where want to recommend a grant that is strongly supported by many of our friends, while attracting little interest from those outside our intellectual and social circles. I think if we avoided recommending such grants, we would be passing over some of our best chances at impact — an unacceptable cost for a “hits-based” approach.

We don’t: avoid the superficial appearance — accompanied by some real risk — of being overconfident and underinformed.

When I picture the ideal philanthropic “hit,” it takes the form of supporting some extremely important idea, where we see potential while most of the world does not. We would then provide support beyond what any other major funder could in order to pursue the idea and eventually find success and change minds.

In such situations, I’d expect the idea initially to be met with skepticism, perhaps even strong opposition, from most people who encounter it. I’d expect that it would not have strong, clear evidence behind it (or to the extent it did, this evidence would be extremely hard to explain and summarize), and betting on it therefore would be a low-probability play. Taking all of this into account, I’d expect outsiders looking at our work to often perceive us as making a poor decision, grounded primarily in speculation, thin evidence and self-reinforcing intellectual bubbles. I’d therefore expect us to appear to many as overconfident and underinformed. And in fact, by the nature of supporting an unpopular idea, we would be at risk of this being true, no matter how hard we tried (and we should try hard) to seek out and consider alternative perspectives.

I think that a “hits-based” approach means we need to be ready to go forward in such situations and accept the risks that come with them. But, as discussed below, I think there are better and worse ways to do this, and important differences between engaging in this sort of risk-taking and simply pursuing self-serving fantasies.

Working principles for doing hits-based giving well

The previous section argues against many principles that are important in other contexts, and that GiveWell fans might have expected us to be following. It is reasonable to ask — if one is ready to make recommendations that aren’t grounded in evidence, expert consensus, or conventional wisdom — is there any principled way to distinguish between good and bad giving? Or should we just be funding what we’re intuitively excited about?

I think it’s hard to say what sort of behavior is most likely to lead to “hits,” which by their nature are rare and probably hard to predict. I don’t know enough about the philanthropists who have been behind past “hits” to be able to say much with confidence. But I can outline some principles we’re working with to try to do “hits-based” giving as well as possible.

Assess importance, neglectedness and tractability. These are the key criteria of the Open Philanthropy Project. I think each of them, all else equal, makes “hits” more likely, and each in isolation can often be assessed fairly straightforwardly. Much of the rest of this section pertains to how to assess these criteria in difficult situations (for example, when there is no expert consensus or clear evidence).

Consider the best and worst plausible cases. Ideally, we’d assign probabilities to each imaginable outcome and focus on the overall expected value. In practice, one approximation is to consider how much impact a project would have if it fully achieved its long-term goals (best plausible case), and how much damage it could do if it were misguided (worst plausible case). The latter gives us some indication of how cautiously we should approach a project, and how much work we should put into exploring possible counterarguments before going forward. The former can serve as a proxy for importance, and we’ve largely taken this approach for assessing importance so far. For example, see the Google spreadsheets linked here, and particularly our estimates of the value of major policy change on different issues.

Goals can often be far more achievable than they appear early on (some examples here), so I believe it’s often worth aiming for a worthy but near-impossible-seeming goal. If successes are rare, it matters a great deal whether we choose to aim for reasonably worthy goals or maximally impactful ones. Despite the uncertainty inherent in this sort of giving, I believe that the question, “How much good could come of the best case?” will have very different answers for different giving opportunities.

Aim for deep understanding of the key issues, literatures, organizations, and people around a cause, either by putting in a great deal of work or by forming a high-trust relationship with someone else who can. If we support projects that seem exciting and high-impact based on superficial understanding, we’re at high risk of being redundant with other funders. If we support projects that seem superficially exciting and high-impact, but aren’t being supported by others, then we risk being systematically biased toward projects that others have chosen not to support for good reasons. By contrast, we generally aim to support projects based on the excitement of trusted people who are at a world-class level of being well-informed, well-connected, and thoughtful in relevant ways.

Achieving this is challenging. It means finding people who are (or can be) maximally well-informed about issues we’ll never have the time to engage with fully, and finding ways to form high-trust relationships with them. As with many other philanthropists, our basic framework for doing this is to choose focus areas and hire staff around those focus areas. In some cases, rather than hiring someone to specialize in a particular cause, we try to ensure that we have a generalist who puts a great deal of time and thought into an area. Either way, our staff aim to become well-networked and form their own high-trust relationships with the best-informed people in the field.

I believe that the payoff of all of this work is the ability to identify ideas that are exciting for reasons that require unusual amounts of thought and knowledge to truly appreciate. That, to me, is a potential recipe for being positioned to support good ideas before they are widely recognized as good, and thus to achieve “hits.”

Minimize the number of people setting strategy and making decisions. When a decision is made as a compromise between a large number of people with very different perspectives, it may have a high probability of being a defensible and reasonable decision, but it seems quite unlikely to be an extraordinarily high-upside decision. I would guess that the latter is more associated with having a distinctive perspective on an issue based on deep thought and context that would be hard to fully communicate to others. Another way to put this is that I’d be more optimistic about a world of individuals pursuing ideas that they’re excited about, with the better ideas gaining traction as more work is done and value is demonstrated, than a world of individuals reaching consensus beforehand on which ideas to pursue.

Formally, grant recommendations currently require signoff from Cari Tuna and myself before they go forward. Informally, our long-term goal is to defer to the staff who know the most about a given case, such that strategy, priorities and grants for a given cause are largely determined by the single person who is most informed about the cause. This means, for example, that we aspire for our criminal justice reform work to be determined by Chloe Cockburn, and our farm animal welfare work to be determined by Lewis Bollard. As stated above, we expect that staff will seek a lot of input from other people, particularly from field experts, but it is ultimately up to them how to consider that input.

Getting to that goal means building and maintaining trust with staff, which in turn means asking them a lot of questions, expecting them to explain a significant amount of their thinking, and hashing out key disagreements. But we never require them to explain all of their thinking; instead, we try to drill down on the arguments that seem most noteworthy or questionable to us. Over time, we aim to lower our level of engagement and scrutiny as we build trust.

I hope to write more about this basic approach in the future.

When possible, support strong leadership with no strings (or minimal strings) attached, rather than supporting unremarkable people/organizations to carry out plans that appeal to us. The case for this principle is an extension of the case for the previous principle, and fits into the same basic approach that I hope to write more about in the future. It’s largely about shifting decision-making power to the people who have the deepest context and understanding.

Understand the other funders in a space, and hesitate to fund things that seem like a fit for them. This is an aspect of “Aim for deep understanding …” that seems worth calling out explicitly. When we fund something that is a conceptual fit for another funder, there’s a good chance that we are either (a) moving only a little more quickly than the other funder, and thus having relatively little impact; or (b) funding something that another funder declined to fund for good reasons. Having a good understanding of the other funders in a space, and ideally having good relationships with them, seems quite important.

Be wary of giving opportunities that seem unlikely (from heuristics) to be neglected. This is largely an extension of the previous principle. When an idea seems to match quite well with conventional wisdom or expert consensus, or serves a particular well-resourced interest, this raises questions about why it hasn’t already attracted support from other funders, and whether it will stay under-funded for long.

Bottom line. The ideal giving opportunity, from my perspective, looks something like: “A trusted staff member with deep knowledge of cause X is very excited to support — with few or no strings attached — the work of person Y, who has an unusual perspective and approach that few others appreciate. The staff member could easily imagine this approach having a massive impact, even if it doesn’t seem likely to. When I first hear the idea, it sounds surprising, and perhaps strange, counterintuitive or unattractive, but when I question the staff member about possible failure modes, concerns, and apparent gaps in the case for the idea, it seems that they are already well-informed and thoughtful about the questions I ask.” This basic setup seems to me to maximize odds of supporting important work that others won’t, and having a chance down the line of changing minds and getting a “hit.”

Reconciling a hits-based approach with being open about our work

A core value of ours is to be open about our work. Some reasons for this:

  • We’d like others to be able to take advantage of what we’ve learned, in order to better inform themselves.
  • We’d like others to be able to understand, question and critique our thinking.
  • We’d like there to be a more sophisticated public dialogue about how to give well.

There is some tension between these goals and the fact that, as discussed above, we expect to do many things that are hard to justify in a convincing way to outsiders. We expect that our writeups will frequently not be exhaustive or highly persuasive, and will often leave readers unsure of whether we’ve made a good decision.

However, we think there is room to achieve both goals — being open and having a “hits-based” approach — to a significant degree. For a given decision, we aim to share our thinking to the point where readers can understand:

  • The major pros and cons we perceive.
  • The premises that are key to our views.
  • The process we’ve followed.
  • What sorts of things the reader might do in order to come to the point of confidently agreeing or disagreeing with our thinking, even if they aren’t sure how to feel based on a writeup alone.

A couple of examples:

We believe that this sort of openness can accomplish a lot in terms of the goals above, even though it often won’t be exhaustive or convincing on its own.

In general, this discussion might help clarify why the Open Philanthropy Project is aimed primarily at major philanthropists — people who have the time to engage deeply with the question of where to give — rather than at individual donors. Individual donors do, of course, have the option to trust us and support us even when our views seem unusual and hard to justify. But for those who don’t already trust us, our writeups (unlike, in my view, GiveWell’s writeups) will not always provide sufficient reason to take us at our word.

“Hits-based mentality” vs. “arrogance”

As discussed above, I believe “hits-based giving” will often entail the superficial appearance — and a real risk of — having overconfident views based on insufficient investigation and reflection. I use “arrogance” as shorthand for the latter qualities.

However, I think there are important, and observable, differences between the two. I think a “hits-based mentality” can be a reasonable justification for some behaviors commonly associated with arrogance — in particular, putting significant resources into an idea that is controversial and unsupported by strong evidence or expert consensus — but not for other behaviors.

Some specific differences that seem important to me:

Communicating uncertainty. I associate arrogance with being certain that one is right, and communicating accordingly. I find it arrogant when people imply that their favorite causes or projects are clearly the best ones, and especially when they imply that work being done by other people, on other causes, is unimportant. A hits-based mentality, by contrast, is consistent both with being excited about an idea and being uncertain about it. We aim to clearly communicate our doubts and uncertainties about our work, and to acknowledge there could be much we’re getting wrong, even as we put resources into our ideas.

Trying hard to be well-informed. I associate arrogance with jumping to conclusions based on limited information. I believe a well-executed “hits-based mentality” involves putting significant work into achieving a solid understanding of the case both for and against one’s ideas. We aspire to think seriously about questions and objections to our work, even though we won’t be able to answer every one convincingly for all audiences.

Respecting those we interact with and avoiding deception, coercion, and other behavior that violates common-sense ethics. In my view, arrogance is at its most damaging when it involves “ends justify the means” thinking. I believe a great deal of harm has been done by people who were so convinced of their contrarian ideas that they were willing to violate common-sense ethics for them (in the worst cases, even using violence).

As stated above, I’d rather live in a world of individuals pursuing ideas that they’re excited about, with the better ideas gaining traction as more work is done and value is demonstrated, than a world of individuals reaching consensus on which ideas to pursue. That’s some justification for a hits-based approach. But with that said, I’d also rather live in a world where individuals pursue their own ideas while adhering to a baseline of good behavior and everyday ethics than a world of individuals lying to each other, coercing each other, and actively interfering with each other to the point where coordination, communication and exchange break down.

On this front, I think our commitment to being honest in our communications is important. It reflects that we don’t think we have all the answers, and we aren’t interested in being manipulative in pursuit of our views; instead, we want others to freely decide, on the merits, whether and how they want to help us in our pursuit of our mission. We aspire to simultaneously pursue bold ideas and remember how easy it would be for us to be wrong.

  • 1.

    There is some possibility of survivorship bias, and a question of how many failed projects there were for each of these successes. However, I note that the examples above aren’t drawn from a huge space of possibilities. (All of the philanthropists covered above would probably — prior to their gifts — have made fairly short lists of the most prominent philanthropists interested in their issues.)

Comments

Why are you calling it “hits-based” giving? That name is unintuitive to me.

I appreciate how honest, thoughtful, and transparent this post is. I’m very happy with OpenPhil’s approach and choices, despite disagreeing with several of their specific decisions, and this made me more confident that they’re doing the right thing. I’m not confident that any change in this process would make them better at doing good, but there is one I want to mention.

It seems like there’s one failure mode of this approach not mentioned in the article. Taking macroeconomic policy as an example, because it seems representative of this approach, let’s say someone has insider knowledge about lobbying, or works for the Fed, or has connections there. They are the sort of person who is tremendously valuable for OpenPhil’s goals there. Because of the constraints on the process that OpenPhil discusses in this blog post, the public information about OpenPhil’s grants in this area are unconvincing (by which I don’t mean that I am personally unconvinced, though I am, but that people with relevant expertise who I’ve encouraged to read it have also been unconvinced.)

OpenPhil says that’s a good thing: if an approach did not initially seem wacky and unwise to experts, there’d be other people doing it and it would be less neglected. They also say that if they expected their grant officers to be able to justify their grants fully, they worry researchers would be less likely to pursue out-there ideas. On the other hand, though, if a lobbyist with expertise in this area or a person who works at the Fed reads your grant explanations, it would be tremendously good if at least some of them come away convinced.

Because everyone knows that there’s lots of information not publicly available, people aren’t sure what is worth debating and arguing and when we should just trust that our concerns would be answered if we knew all the nuanced conversations that we know are happening at OpenPhil.

And while I’m pretty much willing to grant “they’re really good at what they do; I’d probably agree with them if I had the information they had, and they’ve probably considered my counterarguments even if nothing public answers them”, I’m willing to grant that because I trust and respect the people involved. A hypothetical insider who has expertise OpenPhil would benefit from doesn’t have the trust in OpenPhil to go “even though this decision is not justified by the information published in favor of it, the people involved are capable enough that I believe them anyway.”

Sounding weird to experts at first glance is a good thing to target: sounding weird to experts (or non-experts who have a highly related skill set which could be put to bear on the problem) who have read all of your public-facing information could be a failure of communication. I’m hopeful that with more time and expertise OpenPhil will figure out if there’s a way to gain in convincingness without costs to their internal process.
(crossposted from Facebook).

*(Crossposted from Facebook.)*

(I’m sort of conflating Holden and Open Phil here because Holden has written most of the stuff about Open Phil’s large-scale strategy, and I’m not sure which bits represent Open Phil strategy and which are just his personal opinion.)

I more or less agree that it’s a good idea to operate in the ways described here, but I’m not convinced that this is how Open Phil actually operates in practice. Sometimes Holden claims that he wants to maximize expected value and take high-risk bets, but then doesn’t seem to act consistently with this claim. (I can’t say with confidence since I don’t have the complete picture.)

For example, Open Phil made grants on AI safety and is actively supporting FLI (http://www.openphilanthropy.org/…/future-life-institute…), but decided not to fund cultured meat (http://www.openphilanthropy.org/…/animal-product…) because they don’t believe it will be scientifically feasible in the near future. Cultured meat might not have a great chance of success, but I don’t see how it could possibly be less likely to succeed than AI safety research, especially considering that a number of experts believe cultured meat is viable and are betting a lot of money on their belief.

And I don’t think you can argue (from Open Phil’s perspective) that AI safety is enough better than cultured meat to be worth it. The main argument for x-risk work having extremely high value is that if we colonize lots of planets, the far future could be worth a lot more than anything going on right now, so it’s really important to ensure that we make it that far. But Open Phil has explicitly said they don’t put much weight on this argument. In that case, cultured meat looks better because if it works, it has a high probability of permanently ending factory farming.

On minimizing the number of people setting strategy and making decisions:

A potentially useful frame here is one used in the IC and in some business analysis: separating decision making from decision support. You want to analyze any given problem through multiple lenses, and often this work is detailed enough that you want to divy up the work via said lenses. A concrete example is interviewing experts, first principles reasoning, structured probability estimates (scenario analysis) as you mention in discussion of plausible upsides and downsides), reference class forecasting, and Red Teaming might all be things that are assigned to separate people, with their outputs then flowing to the decision maker who ideally will follow best practices in judgment aggregation.

On strategy, priorities and grants for a given cause are largely determined by the single person who is most informed about the cause:

The findings on judgmental bootstrapping in the forecasting literature implies we should be skeptical when an expert is unable to convert their analysis of a domain into an additive linear model that can then easily be shared with others. I’m curious if such models are generated and shared when projects require coordination between multiple researchers, and if not, what the major bottleneck to piloting that would be. Having the affordance of a google sheets template for them for instance could increase decision quality if it causes more and better models to get created and shared on the margin.

On support strong leadership with no strings (or minimal strings) attached, rather than supporting unremarkable people/organizations to carry out plans that appeal to us:

Investing in teams vs ideas seems correct given the reference class of private investing. The potential leak in porting from that reference class is the motivation structure of the teams being dissimilar in the non-profit world vs the private world. It seems difficult to predict in advance which teams will be able to maintain high levels of effectiveness without the same remuneration levers. The traits/tendencies/world views underlying motivation in the non-profit world seem more complex and potentially fragile, though I don’t have any base rate information on that. Might be worth investigating (stability of executive management at non-profits over time?)

I don’t get it. You choose not to take any steps to avoid becoming egregiously overconfident… but still expect to end up correctly evaluating expected outcomes? How?

Are you worried about black swans in the other direction? It seems that food aid could cause famines, and interventions focusing on political change have a nasty tendency to end in millions of corpses. How does that figure in the calculations?

Also, your comment form requires Javascript for no reason whatsoever.

Thanks for this great explanation of your position. I’m very happy to see this development – or presumably a clarification of what you’ve been thinking for a while. It is important to have a set of robustly supported giving opportunities (which have advantages such as allowing others to check the objective evidence and providing a target for skeptical donors), but the most important thing is to be funding the work that has the most positive impact. What you outline here is perfectly compatible with the use of evidence and reason to help others as much as possible.

At Giving What We Can, our work has mostly fallen between GiveWell’s traditional work and this new outline, with our charity recommendations closer to the former and our policy work closer to the latter.

Thanks for the comments, all!

Kelsey, I think it is unrealistic to expect our public writeups to persuade experts to change their minds. An expert in field X generally has years’ worth of (conscious and unconscious) observations, impressions, and relations affecting their views. I wouldn’t expect to be able to overturn all of this with the relatively brief writeups we produce. I might expect to help change minds, in some cases, via supporting partners who can engage in much deeper defenses of ideas.

Michael:

  • The thinking behind the term “hits-based giving” is given in the second paragraph of the post. We considered multiple other options by staff survey, FWIW.
  • I won’t claim we are acting with perfect consistency, but I disagree with the way you’ve characterized the comparison between potential risks of advanced AI and cultured meat. The post I think you’re referring to re: the far future states, “I place only limited weight on the specific argument given by Nick Bostrom in Astronomical Waste – that the potential future population is so massive as to clearly (in a probabilistic framework) dwarf all present-day considerations.” By “place only limited weight,” I mean (as elaborated later in the post) that I don’t allow this argument to carry so much decision-weight that it completely swamps all other considerations combined. That is a far cry from completely dismissing the value of future generations, which we do not (and which that post says we do not). A more accurate way to put my position on this topic is that even considering only 10 generations, affecting the trajectory of civilization as advanced artificial intelligence might seems far more important than speeding a reduction in (or even the end of) factory farming by some number of years; and I think there is some argument, which gets some weight, that we should consider far more than 10 generations.
  • Something we haven’t written about yet, though we have talked about it at events, is the idea that we want to entertain multiple different plausible worldviews and recommend some giving based on each. Such an approach could mean having a significant chunk of our giving that behaves as though the “Astronomical Waste” argument is entirely correct; this would still be consistent with putting limited weight on the argument overall. By a worldview that estimates a very large number of potential future generations (along with some other assumptions), mitigating potential risks from advanced AI seems considerably more important than speeding the end of factory farming; by a worldview that looks less far into the future and has a relatively high estimate of the moral relevance of animal suffering, I believe that other animal-welfare-oriented reforms (such as corporate campaigns) seem more promising than funding work on cultured meat.
  • I have additional points of disagreement with how you’ve characterized the comparison, but will leave it at the above for now.

Romeo:

  • I agree in theory about having multiple angles on decision support even when there is a single decision-maker. However, the type of decision support you describe can be quite expensive (particularly in terms of time).
  • Our process does call for review of expert reasoning in written form. Decision-makers review written arguments, including cost-effectiveness estimates and predictions where possible, from grant investigators. But in many cases it is clear that we would need to invest substantial time in order to assess particular judgment calls and claims being made, and in these cases we often simply defer instead. We do try to make sure we understand the key premises of a given grant, though the effort we put in to do so varies depending on the stakes of the grant and the level of trust we’ve built with the investigator.
  • I don’t think I follow your third point. It seems to me that supporting strong leaders with few strings attached generally maintains or increases their “remuneration,” broadly construed, relative to the counterfactual.

Leo: you say we “choose not to take any steps to avoid becoming egregiously overconfident,” but I think that is contradicted by the body of the post, which describes multiple steps we take toward this end. We do consider downsides and worst cases, as stated in the post.

Thanks for the comments, Holden. There are a few points of disagreement here, but I know you’re busy so I’ll just respond to the part that I believe matters most.

You seem to be implicitly arguing that factory farming will definitely end eventually, and anything we do now only has the effect of marginally accelerating the end of factory farming. If we take this as axiomatic, I agree that animal advocacy doesn’t look that important in the long term. But to say that x-risk looks much better than animal advocacy you have to argue that either (a) factory-farmed animals are worth way less than humans, or (b) it is overwhelmingly likely that factory farming will end eventually, no matter what we do. I do believe factory farming will end eventually even if we don’t do anything, but it’s not overwhelmingly likely—I’m only about 70% confident of that. If, instead, our actions permanently reduce factory farming, or increase the probability of factory farming ending eventually, animal advocacy looks important even across many future generations.

Additionally, even if factory farming does end, there’s a non-negligible probability that humans’ attitude toward non-human animals (or other kinds of sentient beings that may exist in the future) is extremely important. Our actions have massive effects on wild animals and will continue to do so; although we don’t know much about how to do this now, it’s critically important that we reduce wild-animal suffering. It’s much more likely that this will happen eventually if we do things now that make the moral value of animals more salient. I expect that any method of eliminating factory farming would help because if people don’t eat meat they have less cognitive dissonance, but I’m more optimistic about interventions that actively shift people’s values. (Lewis claims that cage-free campaigns do this; I’m somewhat skeptical but I haven’t heard his arguments yet so I don’t want to make any strong claims.)

In short, to say that animal advocacy doesn’t have a big effect on the medium/long-term future you have to be overwhelmingly confident (>99%) that either non-human animals won’t exist, or society will end up with correct values no matter what we do. I don’t believe this level of confidence is justified.

Thanks for the thoughts, Michael. I don’t believe the end of factory farming is inevitable. What I do believe is that if cultured meat proves a viable and tractable way to bring it about, then the most we will have been able to do is speed it up. I.e., I think that cultured meat will either turn out to be intractable on relevant timescales, or it will be developed eventually by the same sorts of people who are currently working on plant-based alternatives. Thus, I think we might be able to accelerate things (though tractability seems poor to me), but I don’t see a path to our funding making the difference between “end factory farming someday” and “never end factory farming.”

I don’t think we need to be overwhelmingly confident in anything to justify our choices, and I am generally suspicious of any reasoning arguing that “lack of overwhelming confidence in proposition X” directly implies one ought to choose a specific area to focus time and resources on.

FWIW, I also endorse (as a best guess) your “(a) factory-farmed animals are worth way less than humans”, though I’m not confident in it and am happy to be recommending some giving that doesn’t rely on this idea.

> I am generally suspicious of any reasoning arguing that “lack of overwhelming confidence in proposition X” directly implies one ought to choose a specific area to focus time and resources on.

Fair enough, I wasn’t trying to say that animal advocacy is definitely the best thing, I was more saying that you seem overconfident in your claim that it doesn’t have important effects on the far future or that its value is swamped by x-risk reduction.

I think if we’re looking at far-future impacts, averting a potential risk from advanced AI would beat (by a big margin) speeding up cultured meat even by a decade or so, which seems like roughly a best-case scenario for cultured meat research funding. I’m not overwhelmingly confident in this view but don’t think I need to be. This is leaving aside tractability/neglectedness, so it isn’t a complete reason to prefer the AI work, but I do prefer the latter all things considered.

Speeding up cultured meat is by a decade is probably less important, yes. (One plausible scenario where this isn’t true is if our values get locked in at some point in the near future, e.g. by a singleton AI, and developing cultured meat substantially affects this.) I was more talking about advocacy that shifts people’s values in the long term, which is likely to be way more important than accelerating the end of factory farming. I believe there are strong arguments for either AI safety or animal advocacy being more important for the far future.

I haven’t found the arguments for animal advocacy as an especially strong far-future cause compelling, and I also haven’t found them to clearly point toward different priorities for animal welfare than the ones we already have, but it’s possible I just haven’t engaged with them enough. If there is a good writeup somewhere, I’d appreciate a pointer.

I formed this opinion through personal conversations and reading a bunch of threads on Felicifia. I don’t know of any particularly strong writing on this. I may write something up because I believe this is really important.

Regarding Open Phil-specific strategy:

For Open Phil to fund some form of advocacy other than cage-free campaigns for the sake of improving the far future, it would have to be the case that (1) this is worthwhile in the first place, and (2) something works better than cage-free campaigns.

On (2), a priori it would be surprising if the best intervention for preventing factory farming in the short term is also best for helping animals in the long term. A posteriori, it doesn’t seem that plausible to me that cage-free reforms have particularly good long-term effects, although Lewis disagrees and I haven’t heard his reasoning.

On (1), this depends on details of Open Phil’s strategy that I don’t know about. If you’re only funding one intervention per category, then you’d have to argue that some sort of animal advocacy is the most important thing for the far future. This is how Open Phil’s grants have looked so far, but I suspect this is just because making grants takes a long time and you’re focusing on breadth before depth. But right now these fields are small enough that Open Phil could fully fund most x-risks work and fully fund animal advocacy for less than 10% of its annual budget. In particular, far future-focused animal advocacy is an extremely small field right now.

(Personal note: several months ago, I was fairly confident that x-risk reduction was more important for the far future than animal advocacy. But I’ve recently spent some time carefully quantifying the arguments for each, and far future-focused animal advocacy actually looks better, even if I assume you’re correct that non-human animals are far less important than humans. I’m going to publish a writeup on this in the near future.)

Our general approach is that we have one person (sometimes a proportion of a person, though there is a full-time person in the case of farm animal welfare) leading our work; what we do is constrained by their time and by guidelines we give about the overall level of annual giving we’re hoping for. The choice to focus first on cage-free campaigns was based on Lewis’s view that this would give the best initial return on his time in terms of reducing farm animal suffering.

I don’t agree with sentiments that we should be surprised when the best short-term option is also the best long-term option. There is plenty of reason to expect this to be disproportionately likely when there is little other basis for forecasting differential long-term impacts.

“There is plenty of reason to expect this to be disproportionately likely when there is little other basis for forecasting differential long-term impacts.”

That’s an interesting perspective. Do you have any sources that argue in favor of this claim?

I think that with some work, I could put together a substantial case for this idea, but it’s pretty out of scope for this discussion and I’ll pass on doing so for now. I think that if you talked to people who work on social mobilization and advocacy in other areas you’d hear a lot of arguments in favor of this idea.

@ Michael and Holden - nice debate I enjoyed reading it.

In terms of the content of the article, I understand and very much respect the approach you are taking and theoretically it makes a lot of sense. However, I would be concerned that it is extremely difficult for any group of people, no matter how smart, humble and well meaning they are, to achieve their goals without feedback. The lack of a natural feedback cycle being a big reason why so many charities are inefficient is a well discussed topic amongst all involved in effective giving, and I think with good reason.
With a hits based approach, the a priori probability of any action succeeding is low and therefore, even with a reasonable number of observations about the outcomes of various actions taken, the information provided by this feedback will likely be reasonably insignificant. Further, some of the grants the OPP is currently undertaking have almost no plausible method for judging impact either way. This will make it very difficult to assess the hits based strategy as a whole.
I wonder should organizations like the OPP consider a strategy where they are strongly in favour of more certain or at least measurable actions. This is not to suggest that they rule out high risk or difficult to measure actions (I think you have put forward several good arguments as to why such actions are important and may be underfunded by other institutions), but that some discount might be applied to high risk or difficult to measure projects. In that case these ventures would require a higher perceived expected value-add to be considered equivalent to those with more certainty and measurability.

Thanks for the thoughts, Jamie. I do think there is room for feedback in our work - we lay out short- and long-term hopes and predictions for grants (often including intermediate goals such as organizational growth) and check in periodically. And we do, all else equal (though it generally isn’t), have a preference for lower-uncertainty work. That said, someone who puts high enough weight on the importance of tangible measurement and feedback would probably prefer GiveWell’s work.

Hi Holden,
I am a big fan of Givewell’s work, but I also recognize that it is not optimal behaviour to focus solely on projects that have already been proven. I think the OPP is great because it allows you guys to consider a wider range of possible actions to find the most effective giving opportunities.
My worry about what I see in the current approach; with hits based giving and providing grants without any way of measuring the impact of the grant is that you maybe are losing the benefits of a scientific approach. If you are not learning by observing the results of your actions - either because the expected probability of success is so low that several failed or disappointing outcomes yield little statistically viable information, or simply because there is no way at all to measure the outcome at all, then you risk becoming ‘no better than the medieval doctors and their leaches’, to quote Esther Duflo.
I’m not arguing against all grants which will not be measurable or high risk. I think you’ve made some good points in the past about why it’s important to be opportunistic, especially when it comes to influencing policy, and cited cases where the right grant at the right time can have a huge impact. My point is simply that you may be undervaluing the cost of uncertainty and therefore focusing too much on these higher risk approaches.
For example, I would love to see a part of the OPP dedicated specifically to finding or founding interventions or charities which could be the next Givewell Top Charity.

p.s. I’ve just caught up on the Givewell blog and I see the issue of developing new potential top charities is being addressed by them (via funding from Good Ventures). Given that is the case it makes more to me sense that the OPP is focusing on more speculative/high risk work.

Hi Jaime,

That’s right, the “next GiveWell Top Charity” work is at GiveWell. I agree that being able to track progress is a benefit all else equal, but: (a) There are informal, intermediate ways to track progress even on “hits-based” grants, such as seeing whether the organizations we support grow and do more and better work over time; (b) I think it is overstated to say that more speculative grants are “no better than the medieval doctors and their leeches,” given that these grants are often based on a great deal of analysis, and given that speculative grants seem to have a reasonably good track record (as outlined in this post).

I feel like the main argument in favour of hits-based giving presented here is a handful of examples where some particular philanthropic pursuit has been hugely successful. That’s certainly enough to show that the strategy *can* work, but I don’t feel convinced that it *will* work, or that it will work the best.

- Is there good reason to believe that the hits observable in retrospect could have been anticipated beforehand?
- What about false positives? What can we say about the number of failed promising philanthropical projects, and the amount of resources wasted on them? Do we know anything about where the average “hit probability” lies? Obviously producing a number would be impossible, but is it plausible that it’s so low that the OPP might do the best possible work and still never find even one?

I agree that the examples we’ve provided aren’t sufficient - on their own - to show that hits-based giving has a high overall expected value. I think the key question, as you say, is how many failures we should expect for each success along those lines, and that’s something we don’t have a good way of figuring out at this point.

A few considerations that I think point in favor of trying hits-based giving, though:

  • A “hits-based” approach works for many for-profit investors, whose overall returns are more knowable. To the extent that nonprofit work is different from for-profit world, many of the differences would seem to suggest a higher expected value for nonprofit “hits-based giving” since there is less overall available capital, less “competition” between funders and between projects, and less need to monetize/internalize the benefits of a project.
  • My overall sense, from what I know about the history of philanthropy so far (previous blog post; I hope we will have an update this year), is that “hits” are rare but not overwhelmingly rare, and that the hits-based approach looks competitive with a more risk-averse approach.
  • If we try the “hits-based approach,” we can look back on it in the future - say, in 20 years - and have a much better basis for saying how promising it looks; this could inform our future work and that of other funders.

Leave a comment