AI existential risk probabilities are too unreliable to inform policy

Governments face challenges in assessing the existential risks posed by artificial intelligence (AI) due to the lack of consensus among researchers and the speculative nature of these risks. The reliance on probability estimates to inform policy decisions is problematic, as these forecasts are often unreliable and can be misleading. The authors argue that while AI x-risk forecasting can be valuable in academic contexts, its application in public policy lacks justification. They critique the three main methods of probability estimation: inductive, deductive, and subjective. Inductive estimates are flawed due to the absence of a relevant reference class for AI risks, making it difficult to draw parallels with past events. Deductive estimates fail because there is no reliable theoretical model to predict AI-related outcomes. Subjective probabilities, which are essentially guesses based on personal judgment, vary widely and lack a solid foundation. The authors highlight a forecasting exercise that revealed significant discrepancies in risk estimates among experts, underscoring the uncertainty in this field. They emphasize the need for policymakers to critically evaluate the basis of any probability estimates before making decisions that could have far-reaching consequences. Ultimately, the authors advocate for a more evidence-based approach to understanding AI risks, recognizing the limitations of current forecasting methods and the importance of transparency in the decision-making process.