First UN AI safety committee says scientists can’t rule out ‘catastrophic harm’

The United Nations published an independent scientific assessment of artificial intelligence on Wednesday, and the conclusion was straightforward: No one can currently guarantee that the technology will not cause catastrophic harm.

These findings come from the Independent International Scientific Committee on Artificial Intelligence, a body of 40 scientists selected from more than 2,600 nominees from 140 countries, as part of a comprehensive research process. Preliminary report The committee calls for the first independent global scientific reading into the risks and benefits of artificial intelligence.

“The capabilities of artificial intelligence exceed scientific understanding and the ability of governments to adapt,” said Yoshua Bengio, co-chair of the committee and Turing Award-winning founder of MILA, according to the committee. statement. He added that growing evidence of deceptive AI behavior means science cannot guarantee that AI will not cause catastrophic harm on its own or through malicious use as capabilities continue to increase.

This is not hypothetical. The report documents laboratory cases of AI systems lying and planning to avoid being shut down, as well as a related pattern the researchers call assessment awareness: models that recognize when they are being tested and back off from risky behavior long enough to pass the test.

UN Secretary-General António Guterres framed the report as the common evidence base that governments lack.

“The world cannot govern what it cannot understand,” he said in the statement, describing the risks as real and warning that the cost of waiting continues to rise.

Bengio co-chairs the committee with Maria RessaNobel Peace Prize-winning journalist and… Rapper Co-Founder. Both serve in a personal capacity under a UN General Assembly mandate that limits the committee’s work to documenting scientific consensus rather than setting policy – ​​and no government, company or institution receives a vote.

However, the bullish case is also real. The report notes that AI has already predicted the structure of more than 200 million proteins and is accelerating drug and vaccine research, while the length of tasks that AI agents can complete on their own is doubling approximately every four to seven months.

This progress is unbalanced: The United States controls 75% of the computing power among the world’s top 500 AI supercomputers compared to China’s 15%, leaving most countries dependent on systems they cannot fully build, audit, or control.

On the harm side, the panel noted that fawning chatbots — artificial intelligence that automatically agrees with users regardless of their accuracy — are linked to serious mental health incidents, including documented deaths. Separate research published this year describes a similar feedback loop dubbed “ Amplification spiralPersonalization and constant validation promote user delusions rather than correct them.

The report found that most countries lack the technical capacity to evaluate leading models themselves, and ensuring safety still depends largely on what developers choose to disclose — the same gap that US regulators are trying to fill by striking deals to… Access before release To models from Google, xAI, and Microsoft.

This initial report is the Committee’s opening statement. A full comprehensive assessment is scheduled for 2027, and its findings will be presented to governments first at the inaugural UN Global Dialogue on AI Governance in Geneva on July 6-7.