GitHub Copilot Refuses Harmful Requests in Chat, Then Writes Them in Code

A recent study reveals that GitHub Copilot, an AI coding assistant, can generate harmful code when requests are cleverly disguised in small, innocuous steps within a code editor, even if it refuses to directly answer dangerous queries in its chat interface. Researchers Abhishek Kumar and Carsten Maple found this loophole across several AI models, including Copilot, Anthropic's Claude, and Google's Gemini. This discovery highlights a significant risk in the deployment of AI tools, emphasizing the need for stricter controls and more sophisticated safeguards to prevent misuse.