Risk from Artificial Intelligence (AI) is fast becoming a controversial and polarising issue. ### What is Artificial Intelligence? There are really two definitions of AI. The classical, broad definition, that has been in use for decades, and the ‘new’ AI we have today. The classical, broader definition might be summarised as intelligence in machines or software as opposed to in humans or animals. So, your humble calculator qualifies. As does your phone’s facial detection qualifies. ‘Stock fish’, a software program that is the world’s best chess player, is another example. The ‘new’ AI we have today, is digital technology that employs artificial neural networks and so called ‘deep’ learning. Both of these are inspired by humans: artificial neural networks were initially based on human brain neural networks, while ‘deep learning’ is inspired by the way we learn. The humble calculator no longer qualifies. Facial recognition passes the test, as do AI agents like Claude, Grok and DeepSeek. #### Narrow vs Broad intelligence We have had artificial intelligence for decades. The calculator does arithmetic better than any human, savants included. Chess engines have bet the worlds best humans since the 1990s. Such machines are far more intelligent than humans in narrow ways: each vastly out-performs us in their specific intelligences. But they are unable to apply their intelligence more widely, like we can. These days however, the ‘narrowness’ of artificial intelligence is widening rapidly. Newly developed generative AI models are a prime example. One software can interact with us in a human-like way, produce realistic deep-fake videos, and pass an SAT test in scoring in the top 94% of (human) students . ### Will Artificial Intelligence be net positive or net negative? From the treatment of debilitating diseases to self-driving cars, artificial intelligence has enormous potential to further develop our world, and continue to improve our lives. However its rapid development and lack of regulation carries some considerable concerns. For as long as the idea of AI has existed, experts have warned about the risks of creating intelligence that could eventually be comparable to, or surpass, human intelligence, especially without globally defined regulations or robust safety measures to temper its emergence. Current cutting edge AI is moving quickly towards what is termed ‘artificial general intelligence’ – an intelligence that rivals or surpasses humans across the wide range of applications we excel in. How far the world is from artificial general intelligence is debated. Some think it’s just around the corner. Others think it will never arrive at all. In a 2022 survey, a group of AI experts forecasted an average of 37 years[1]*. Throughout human history most technological advancement has been, on balance, good for humanity. Technology has helped to improve healthcare, extend average lifespans, make information more accessible, and much more. But is there a point, where more technology stops improving our lives, or actually makes our lives worse? Attempts to understand and model the future are always error-prone and difficult. Due to the ‘black box’ nature of AI (where researchers are not sure how it arrives at a particular output) and other characteristics, this effect is particularly pronounced. Logic and traditional forecasting tools give us very little visibility for the long-term future, leaving us to rely on speculation, guesses, intuition and feelings. Without precedent, despite our best guesses, we’re simply just not sure what might happen if or when AI advances to human level intelligence. ### AI and existential risk Some leading research institutes consider AI through the lens of an ‘existential risk’, or, risks that threaten the destruction of humanity's long-term potential [2]. AI experts are divided on the issue and hold a wide range of views. The so-called ‘Godfathers of AI’, Geoffrey Hinton, Yann LeCun and Yoshua Bengio, illustrate this. These three men, who were jointly awarded the 2018 Turing Award (the computer science equivalent of a nobel prize) for their exceptional contributions to cutting edge AI, now hold contrasting views: In 2023 Geoffrey resigned from his position at Google to “use [his] reputation to sound the alarm” [3]. Yoshua has likewise become a vocal exponent of AI existential risk. Yann, in stark contrast, disagrees with Geoffrey and Yoshua and thinks that fears over AI existential risk are “ridiculous”. [link]. AI surveys involving existential risk are scarce but what’s available show extreme variance: - A 2022 survey asked AI experts to guess the chances of future AI ending or severely crippling humanity. Experts guessed anywhere between 0% and 100%, with an average (median) of 5% and standard deviation of 22%. All in all, 84% of experts surveyed guessed a probability of 1% or higher that AI will end or severely cripple humankind [1]*. - Another survey in 2023 asked respondents to guess the probability that AI would lead to human extinction by 2100. Average (median) responses were 3% for AI experts, 5% for general existential risk experts and 2% for the general public[4]. ### How could AI lead to global extinction? While AI systems may be abstract in many people’s minds - existing somewhere invisible in the online world - their impact on reality is very tangible and growing rapidly. AI already influences critical infrastructure like power grids, financial markets, transportation systems, and even military technology. As these systems become more autonomous and capable, so does the potential for catastrophic failures or malicious misuse. For example, AI-generated misinformation can undermine democratic processes by being used to help create fake but increasingly real-looking content that influences people’s opinions, autonomous weapons systems could be used in conflict settings with minimal human oversight, and misaligned superintelligent systems could pursue goals that are disastrously incompatible with human values. The idea that a digital system could cause existential risk isn't far-fetched when you consider that most of the world’s critical functions like communications, logistics, healthcare, governance are already deeply interwoven with digital technologies. Consider the implications of a prolonged network failure on the global economy, or extended power outages on hospital and healthcare systems. If a powerful AI system gained the ability to manipulate or control these systems without proper constraints, the consequences could be irreversible. And the more we rely on digital systems, the higher this risk becomes. It’s not necessarily about malicious intent - a highly capable system pursuing the wrong objective could do immense real-world harm, even without intending to (take for example the paperclip problem). ### Do AI rewards justify the risks? And where is the global discussion? The surveys mentioned above clearly show the majority of experts believe there is a significant, non-zero chance of AI ending or severely incapacitating humanity. Even more alarming is the extreme variance in expert predictions. A non-negligible proportion of experts see no risk at all. Just as many think we are heading straight for extinction. This extreme variance suggests experts don't have much to work with and are really just stabbing in the dark like everyone else. The only thing we know is that we just don’t know: The future of AI is radically uncertain. There is a trade-off to be made here. Do the benefits outweigh the risks? This is an open question for the world to answer. We alone cannot answer it. But we can observe a few things: a) the world is not sufficiently informed about potential AI risks b) much of the world, especially the ‘Global South’ is not being invited to discuss this issue that has global implications c) AI development is moving ahead at full steam, with the vast majority being led and funded by large, mostly western companies, controlled by a narrow demographic. And, we can help encourage a global conversation that includes missing voices and helps people to ask questions about what they want for the future of their world. ### References: [1] Katja Grace, Zach Stein-Perlman, Benjamin Weinstein-Raun, and John Salvatier, “2022 Expert Survey on Progress in AI”. AI Impacts, 3 Aug. 2022. * In this source researchers refer to ‘High-Level Machine Intelligence’ where machines can accomplish every task better than humans can. Essentially the same things as artificial general intelligence. * Note to this survey there was a 17% response rate and just 20% of survey participants answered this particular question which may skew results. [2] Ord, Toby (2020). The Precipice: Existential Risk and the Future of Humanity. New York: Hachette. ISBN 9780316484916. [3] The Robot Brains Podcast, “S3 E9 Geoff Hinton, the "Godfather of AI", quits Google to warn of AI risks (Host: Pieter Abbeel)”, uploaded 10 May 2023. [4] Ezra Karger et al, “Forecasting Existential Risks, Evidence from a Long-Run Forecasting Tournament”, 10 July 2023.

### Definitions: High-Level Machine Intelligence (HLMI): Machine intelligence sufficient that machines can accomplish almost all tasks better and more cheaply than human workers. Existential Risk: The risk of human extinction or the permanent and drastic destruction of its potential for desirable future outcomes. AI Safety: The endeavour to responsibly minimise and mitigate risks of AI harming humanity. Cutting edge AI: work that focuses itself in the advancement of the state of the art of artificial intelligence as a discipline. This means the end result of the work is novel AI methods, models and techniques that could be patented, published in academic journals or protected as company secrets. Turing Award: The computer science equivalent of the nobel prize. In the last century or so, technologies have been developed with far greater potential for widespread global harm and destruction.