It appears possible that the coming decades will see substantial progress in artificial intelligence, potentially even to the point where machines come to outperform humans in many or nearly all intellectual domains, though it is difficult or impossible to make confident forecasts in this area. These advances could lead to extremely positive developments, but could also potentially pose risks from misuse, accidents, or harmful societal effects, which could plausibly reach the level of global catastrophic risks [1]. We’re interested in supporting technical research that could reduce the risk of accidents, as well as strategic and policy work that has the potential to improve society’s preparedness for major advances in AI.

For more on why we consider this an important cause, see Potential Risks from Advanced Artificial Intelligence: The Philanthropic Opportunity [2] (May 2016). Also see our process for selecting focus areas [3].

Illustrative grants

A complete list of our grants in the area of potential risks from artificial intelligence can be found here [4]. Grants include:

• Georgetown University — Center for Security and Emerging Technology [5]
• UC Berkeley — Center for Human-Compatible AI [6]
• Ought — General Support [7]
• GoalsRL — Workshop on Goal Specifications for Reinforcement Learning [8]
• Machine Intelligence Research Institute — General Support [9]
• OpenAI — General Support [10]

Other content

• The Open Phil AI Fellowship [11]
• Blog post: Potential risks from advanced artificial intelligence: the philanthropic opportunity [12]
• Blog post: Some background on our views regarding advanced artificial intelligence [13]
  
• Public cause report (August 2015) [14] used in our process for selecting focus areas [3]

• What should we learn from past AI forecasts? [15]
• What do we know about AI timelines? [16]
• Blog post: Concrete Problems in AI Safety [17]