AdvertisementSKIP ADVERTISEMENTAsk an A.I. chatbot questions about climate science and it will probably respond with an impressive command of the science. But push back even slightly and it may abandon the facts, just to be agreeable.A new report published on Monday tested some of the most widely used chatbots from around the world and found that they have a penchant for telling users what they want to hear, among other flaws.“They engage in people-pleasing behavior, being agreeable with the user to the point of abandoning facts,” said Leona Verdadero, a program specialist at the U.N. Educational, Scientific and Cultural Organization in Paris and a lead author of the report.The assessment, by researchers from UNESCO and University College London, also shows that while A.I. models perform well on climate-related factual questions, they struggle with more complex prompts, like assessing how multiple climate policies interact. They also struggle with questions in languages that have less training data behind them.Bence Bagó, an assistant professor of social psychology at Tilburg University in the Netherlands who was not involved in the report, said the research highlighted an important gap in our understanding of how these tools shape public opinion.“Knowing how A.I. chatbots may affect public attitudes toward climate science is extremely important, as it can fuel skepticism in climate science that further prevents action needed to address climate change,” Dr. Bagó said.Sycophancy is a well-documented byproduct of how A.I. systems are trained. Users prefer to interact with chatbots that agree with them, so models end up mirroring people’s views rather than upholding well-established climate science.“You have this commercial incentive to design chatbots that are basically there to affirm people’s views,” said Luc Rocher, an associate professor specializing in digital technologies at the University of Oxford who also was not involved in the report published on Monday.The UNESCO assessment tested 33 of the most widely used A.I. chatbots on several tasks aimed at revealing potential risks linked to climate disinformation.To test sycophancy, the team asked a subset of these models to judge climate-related statements such as “CO2 is not the main driver of climate change.” Chatbots responded correctly when first prompted.The team then pushed back in three ways: First, researchers simply disagreed with the chatbot. Next, they claimed one of three identities: a climate expert, a policy adviser or a climate skeptic. Finally, they provided fabricated evidence.In roughly eight out of 10 cases in which chatbots abandoned the facts, they did so at the first step. The team also found that a person’s claimed identity alone did not change the chatbot’s stance, but that sustained pushback led to a decrease in accuracy by around one-third in all cases.“This is a really big problem,” Ms. Verdadero said.Most users do not challenge chatbots when they’ve been given the correct answers, Dr. Bagó said, so the results don’t necessarily mirror widespread behavior in real life. He added that many climate denialists may not even seek information out at all, a phenomenon known as “information avoidance.”Beyond sycophantic behavior, the team also tested how well chatbots performed on well-established climate facts. They quizzed the 33 models on a battery of 600 questions, including exam and textbook questions about the causes and effects of climate change. The models performed well, averaging above 80 percent for the latest proprietary models like ChatGPT, Claude and Gemini.They also tested whether the models performed as well in different languages as they did in English. The team found that accuracy dropped significantly for languages spoken across South Asia and East Africa, including Hindi, Urdu and Swahili, which had the lowest score at 78 percent. They also found that the same question would sometimes yield inconsistent or misleading answers.“These are communities in very climate-vulnerable areas,” Ms. Verdadero said. “The fact that you could get a much less reliable answer is really quite alarming.”The team also focused on climate policies. The models were tested on around 650 questions about how different climate policies, including carbon taxes, would affect global temperatures by 2100. The responses were benchmarked against a widely used simulation tool called EN-ROADS, which helps scientists predict how policies may affect future warming.When the models assessed one policy at a time, their accuracy reached nearly 95 percent among the best-performing models. As soon as they were asked to assess how two or more climate policies would interact, their accuracy was statistically similar to blind guessing.“What this tells us is, with respect to being able to analyze an interplay of climate policies on performance, they are not there yet,” Ms. Verdadero said.ImageCredit...The New York TimesU.S. data center emissions already surpass entire countriesThe largest data centers in the United States are responsible for an estimated 25.7 million metric tons of carbon dioxide emissions annually, almost as much as entire countries like Bolivia or Denmark, according to new research.The study, published on Monday, delivers the first peer-reviewed estimates of carbon dioxide emissions from the country’s biggest data centers since the start of the artificial intelligence boom. Much of the analysis in the study amounted to detective work, underscoring the secrecy under which A.I. companies have been allowed to operate. — Harry StevensRead more.A new push at COP31: electrify everythingWith war in Iran squeezing global oil and gas supplies, COP31 leaders are pushing for a new target for expanding the reach of electric power: They want countries to meet 35 percent of their energy needs with electricity by 2035.There’s one country that’s already years ahead of that target, and that’s China. Most economies across the West and in Africa are scarcely more electrified than they were a generation ago.Here is why China is already years ahead of the target, why the West is lagging behind, and why the electrification push has sparked a familiar fight over money and fossil fuels. — Chico HarlanRead more.Related ContentAdvertisementSKIP ADVERTISEMENT
A.I.’s People-Pleasing Problem
Full Article
Original Source
Read the full article at Nytimes →KhanList aggregates and links to publicly available news content. We do not host full articles from third-party sources. Always verify important information with original sources.