Focus Areas

AI governance

Advanced AI could have enormous benefits but could also pose catastrophic risks. We're particularly interested in preventing advanced AI from being misused or falling into the wrong hands, including through technical work on compute governance and information security.

AI welfare

As AI systems advance, they may develop sentience and require moral consideration. The NYU Center for Mind, Ethics, and Policy and Eleos AI are some of our grantees in this area.

Cooperative AI

We support research on promoting cooperation between advanced AI systems. The Center on Long-Term Risk and the Cooperative AI Foundation are among our grantees in this area. We are also interested in work that guides AI character towards cooperativeness and benevolence.

Some of history's worst catastrophes have followed the erosion of democratic institutions, the rule of law, and reasoned public discourse. We support work that strengthens these and other safeguards to reduce risks from authoritarianism and stable totalitarianism. Advanced AI is a central concern here, since it could create new pathways for power to be highly concentrated and entrenched—in the worst case, in the hands of fanatical or malevolent actors.

Both farmed and wild animals likely undergo great suffering in huge numbers. We are interested in interventions to reduce animal suffering, including that of especially neglected populations such as farmed fish and invertebrates.

Improving the world—especially from a long-term perspective—is fraught with extreme uncertainty. It's entirely possible that our current efforts are misguided. For example, most of our grants so far have supported academic and independent research but we'd also like to support more practical and entrepreneurial projects.

We actively seek external input, including critiques of our current work, to refine our approach to reducing long-term risks. This may lead us to pursue different focus areas or projects.