There is more debate than any other issue among true experts on whether AI constitutes a threat to humanity. You'll probably get range of ten different responses if you ask ten leading researchers whether the chances of a university project succeeding are "negligible" or "nearly certain. It's not a settled scientific issue, and that range itself is worth paying attention to, whatever their view is and whatever their confidence level is, whether that is expressed as 1600 or 3000.
The reasons why there is so much variation in expert opinion
The huge variations in the predictions of AI risks are due to the fact that they are based on very different assumptions on when the technology will progress, whether systems will have goals that conflict with human interests, and how well current safety measures will actually succeed. There are some experts who talk about the fast pace of AI development and say there's no way to protect against it. Others note that claims of transformative or threatening AI have been exaggerated in the past, and assert that existing AI systems are still much more constrained than the doom and gloom scenarios presume.
Both are not without merit; they are simply playing the odds differently, depending on their technical opinions on the direction the technology is going.
The Trouble With Putting a Number on It
One common grievance from more cynical researchers isn't that AI risks are imagined, but that they can be given a real probability, giving the false impression of scientific accuracy. What is claimed is that it is a “10% chance of catastrophe”, which sounds tough, but has assumptions that cannot be independently tested or verified, unlike the probability of weather prediction or actuarial risk.
The difference is significant, as the findings of these people may influence public opinion and policy, even at moments of greater clarity when the researchers behind them realize that the probability of any particular result is virtually impossible to know.
Even if a specific number of people was not given, real risks exist.
The notion of doom percentages doesn't discredit the risk of AI.But doom percentages is not the same as dismissing the risk of AI. But there are a number of issues that are discussed seriously by researchers and institutions without resorting to any alarming statistics:
The potential for misuse by bad actors, such as the use of AI to reduce the cost of creating harmful technologies and/or committing large-scale cyberattacks.The threat of misuse by bad actors, e.g. AI making it easier to build harmful technologies and/or to conduct widespread cyberattacks.
The inability to effectively take control of any meaningful human supervision, so reliant on AI that stepping away becomes nearly impossible.
FAQs
1. Why is there so much disagreement among AI experts on existential risk?
Predictions are based on various changes and expectations about the rate at which AI will develop and whether safety measures will follow, which can vary from person to person and are not definite.
2. Is it realistic to set a percentage on the possibility of an existential risk?
This is debated. Critics say such percentages give false precision because the assumptions cannot be tested or verified independently as with normal probability estimates.
3. Are doom percentages not to be taken as seriously?
No. Many critics still have concerns about taking real risks seriously, such as risks from misuse by bad actors or lack of oversight, as well as specific concerns about overconfident predictions of numbers.
4. What are some tangible risks of AI that aren't ‘extinction’ related?
Other dangers, such as those related to cybersecurity, to disinformation on a large scale, and to relying heavily on AI systems for important decisions, are deemed serious but not existential dangers.
5. What are the chances that this debate will have any impact on the future of AI in real-world applications?
Most sit in the middle ground: that there is real risk, but not a great deal of certainty when making predictions, and not the most extreme dismissive or alarmist interpretation.
