Solving the Right Problem: Towards Researcher Consensus on AI Existential Threat Models

A systematic review of existential AI threat models and taxonomies

Existential Risk Observatory · MIT FutureTech · Future of Life Institute

About this project

This project conducts a systematic review of existing threat models and taxonomies of existential AI risk. A threat model describes the causal mechanisms and pathways through which AI could lead to an existential catastrophe, such as human extinction or permanent loss of humanity’s long-term potential.

Despite growing concern about existential AI risks, the field lacks a consolidated overview of how these risks have been modeled. Without such an overview, researchers, policymakers, and AI safety teams may be working with different or unexamined threat models, making it harder to coordinate research priorities, policy recommendations, and risk communication.

This systematic literature review is the first stage of a multi-paper project that aims to catalogue existing threat models, identify gaps and disagreements, and work toward researcher consensus on how AI could pose existential risks. The project is carried out by the Existential Risk Observatory, MIT FutureTech, and the Future of Life Institute.

Research questions

Primary: What are the existing threat models of existential AI risks?
Secondary: What are the frameworks and categories used to classify threat models of existential AI risks?

Expert consultation

Your expertise is valuable to us. As researchers active in this field, you may be aware of relevant work that our systematic search has not captured, and we would greatly appreciate your input. Here is a suggested workflow for identifying documents that may be missing:

  1. Think of existential AI threat models or taxonomies/classifications of threat models you are aware of.
  2. Think of where you may have read or written about these, and try to locate the source.
  3. Check whether the source is publicly accessible and has an identifiable author and a stable URL or DOI. While we have sourced our initial documents from academic databases and the Alignment Forum, you are also welcome to submit documents such as: book chapters, websites, and blog posts from other platforms.
  4. Search the document list below to check whether it is already included.
  5. If it is not included, submit it using the form linked below.

Other approaches to identifying missing documents are equally welcome. You can submit multiple documents in a single form submission, and you can submit the form multiple times if you think of more later.

What we’re looking for

We are looking for documents that:
  • Propose or substantially develop a threat model of existential AI risk, including causal mechanisms and pathways
  • Propose or substantially develop a taxonomy or framework for classifying existential AI threat models
We are not looking for:
  • Documents focused on non-existential AI risks (bias, privacy, misinformation) without connection to existential outcomes
  • Documents that only cite existing threat models without developing their own
  • Mitigation-only documents where threat models are assumed rather than substantively developed

Please err on the side of inclusion when in doubt. Our team will screen all suggestions against the full criteria. For the complete inclusion and exclusion criteria, see our pre-registered protocol on OSF: https://osf.io/vk5gy/files/5rndj.

Submit a document

You can suggest documents using our short form: Open submission form →

You can also reply directly to the email you received. A title and link (DOI or URL) for each suggestion is sufficient. You can submit multiple documents at once, and you can submit the form more than once if you think of additional documents later.

Recognition

Contributors can choose how they are recognized:

You can indicate your preference in the submission form.

Current document list

This list contains documents identified during our search process. These have not all been fully reviewed yet; they are candidates we will inspect as the review progresses.

Title Author(s) Year Source type

Contact

Questions about the project or this consultation? Contact Hugo Save at hugo@existentialriskobservatory.org.