Potential Risks from Advanced Artificial Intelligence

We aim to support research and strategic work that could reduce risks and improve preparedness.

In recent years, we’ve seen rapid progress in artificial intelligence. And within a decade or two, we think it’s plausible that AI systems will arrive that can outperform humans in nearly all intellectual domains (as do many of the world’s foremost AI experts).

These systems could have enormous benefits, from accelerating scientific progress to vastly increasing global GDP. However, they could also pose severe risks from misuse, accidents, or drastic societal change — with potentially catastrophic effects. We’re interested in supporting technical, strategic, and policy work that could reduce the risk of accidents or help society prepare for major advances in AI.

Funding opportunities and requests for proposals

• RFP on technical AI safety: Support work across 21 research areas to help make AI systems more trustworthy, rule-following, and aligned, even as they become more capable. We’re open to proposals for grants of many sizes and purposes, from rapid funding for API credits to seed funding for new research organizations. The deadline has passed for EOI submissions to this RFP, but the submission form will stay open until June 15 to accommodate applicants who received a “revise and resubmit” response to their EOIs. During this 2-month grace period, we will also accept late submissions from other applicants but will have a high bar for considering them.
• RFP on AI governance: Supports work in six areas: technical AI governance, policy development, frontier company policy, international AI governance, law, and strategic analysis and threat modeling. We’re evaluating expressions of interest on a rolling basis.