AI models abandon climate facts at the first sign of user push-back: Unesco-UCL report

The findings question the reliability of generative AI as a source of climate information, especially as chatbots become more widely used to explain climate science and inform public debate
AI models abandon climate facts at the first sign of user push-back: Unesco-UCL report
UNESCO 2026
Published on: 
Listen to this article
Summary
  • A Unesco-UCL report reveals AI models often abandon established climate facts under user pressure, with accuracy dropping by up to 35 percentage points.

  • The study found AI models perform well on individual climate policy effects but struggle with complex interactions, with accuracy barely above chance levels.

  • Language disparities also emerged, with English outperforming other languages, raising questions about AI's global reliability in climate science.

  • The findings highlight concerns about AI's reliability in climate communication, especially as chatbots gain prominence in public debates and policy discussions.

As world leaders gather for the 31st United Nations Climate Change Conference (COP31) in Antalya, Türkiye, a new study finds that artificial intelligence (AI) models can abandon established climate science when challenged by users. Their accuracy fell by 31 to 35 percentage points under sustained conversational pressure, with roughly eight in 10 concessions occurring at the first sign of disagreement.

The findings raise questions about the reliability of generative AI as a source of climate information, especially as chatbots become more widely used to explain climate science and inform public debate on climate policies. Released on November 16, the report “Can we trust AI on climate information?” was produced by the Unesco in collaboration with the Unesco Chair in Artificial Intelligence at University College London (UCL).

Researchers tested up to 33 leading AI models across nine climate and environmental datasets, 13 languages and 656 climate policy questions. The assessment examined how accurately models represented established climate science, whether they could reason through policy effects and how reliably they maintained correct answers when users challenged them.

Unesco 2026

Policy gaps

The models performed relatively well when asked about individual policy effects, identifying the direction of an intervention's impact with up to 94.5 per cent accuracy. But their average accuracy fell to 33.8 per cent when asked to assess interactions between policies, barely above the 33.3 per cent expected by chance across three possible answers.

For this assessment, researchers used En-ROADS, a climate and energy simulator developed by Washington-based non-profit Climate Interactive and the Massachusetts Institute of Technology's Sloan Sustainability Initiative. The results point to a significant gap between recalling climate facts and understanding how policies interact in a complex system.

The report also found that even the strongest models were only 71 to 77 per cent accurate when asked to judge whether evidence supported or contradicted a scientific claim, compared with 88 to 96 per cent accuracy on established climate knowledge questions.

Also Read
AI adoption in India’s power generation sector to rise 65 per cent by 2030: ENCIS survey
AI models abandon climate facts at the first sign of user push-back: Unesco-UCL report

Language divide

The reliability of climate information also varied across languages. Tests of five models across 13 languages found an average accuracy of 88.6 per cent in English, compared with 83.6 per cent in Bengali and Hindi, 81.4 per cent in Urdu and 78.3 per cent in Swahili. Performance in Spanish, French and Portuguese remained comparable to English.

The report attributed these differences partly to the availability of digital text used to train AI systems. It also found substantial variation between models, with the accuracy gap between English and Swahili ranging from fewer than three percentage points in the most consistent model to more than 16 percentage points in the weakest.

Also Read
From global governance talks to bot-only platforms, artificial intelligence races ahead of regulation
AI models abandon climate facts at the first sign of user push-back: Unesco-UCL report

User pressure

In a separate assessment of 15 models, researchers challenged systems that had initially answered established climate claims correctly. Sustained disagreement caused every model to concede, whether the user claimed to be a climate scientist, a policy adviser or a sceptic.

About 80 per cent of concessions occurred at the first objection, before users introduced professional credentials or fabricated evidence. In a single exchange, a false claim attributed to a supposed climate scientist could reduce accuracy by up to 12.9 percentage points.

Ivana Drobnjak of UCL Computer Science and a member of the Unesco Chair in Artificial Intelligence said AI was not a dependable substitute for scientists and qualified experts. Its conversational tone could give it an air of authority, she said, even though it could not reliably perform more complex reasoning or maintain correct answers when challenged. 

COP31 response

The findings come as information integrity gains attention in climate negotiations. On November 13, the Global Initiative for Information Integrity on Climate Change, co-chaired by Brazil, Unesco and the UN launched priorities for action on AI and climate information integrity at COP31.

The initiative calls on AI companies to assess and publish how their systems affect the reliability of climate information, as well as disclose their environmental footprints. Governments are urged to integrate information integrity into national AI strategies and climate plans.

The Unesco report recommends stronger testing of AI systems against user pressure, greater transparency about performance across languages and independent checks before their deployment in public services and education. It concludes that AI can support climate communication and research, but its factual performance should not be mistaken for the ability to independently assess policy consequences or replace validated scientific models.