Forecasting Transformative AI

Abstract

Technological forecasting is challenging, and the forecasting of AI progress is even more challenging. Forecasting transformative AI – AI that has great potential for societal transformation – which presents even more difficult challenges. This study addresses the research question of “How can we best forecast transformative AI?” To this end it explores a wide variety of techniques that are used for technological forecasting, demonstrates their viability in the context of transformative AI and evaluates their value for use in future efforts to forecast transformative AI.

The study focuses on scenario analysis techniques, a variety of judgmental forecasting techniques and simple statistical forecasting techniques. These include a survey, interviews, bibliometric analysis and the Delphi technique. The literature review identifies a new subclass of scenario planning techniques called scenario mapping techniques. These techniques are well- suited for forecasting transformative AI because of their incorporation of large numbers of scenarios (i.e., future technologies) in directed graphs. Two novel techniques that meet the criteria of scenario mapping techniques are developed and demonstrated.

The two methods, along with a variety of other forecasting techniques, are components of a proposed holistic forecasting framework for transformative AI. More generally, the holistic forecasting framework suggests that a combination of judgmental, statistical and scenario analysis techniques are necessary for forecasting complex future technologies such as transformative AI. This study is the first of which we are aware of to demonstrate the use of a holistic forecasting framework in any context. However, another framework(s) that satisfies the proposed criteria is identified and discussed.

The results of the study include forecasts generated from two of the techniques demonstrated along with significant insights for future work on forecasting transformative AI. A research agenda for forecasting AI progress was also created in demonstrating and evaluating the Delphi technique for the purpose of using expert elicitation to generate questions of interest for use as forecasting targets in prediction markets or forecasting tournaments. The results from the survey also generate insights into limitations of existing methods used for future of work research.

The study produces numerous novel contributions including theoretical elements from the definition of transformative AI, the holistic forecasting framework, the two novel methods and the concept of the subclass of scenario mapping methods. Beyond this, a large variety of insights are identified and reported as a result of the demonstration of the various techniques. Moreover, elicitation of some of the world’s leading experts on specific subdisciplines of AI yields some significant insights regarding future progress in AI.