Can LLMs Audit Smart Contracts? Benchmarking Claude Opus 4.7, GPT-5.5, and Gemini 3.1 Pro

I gave 56 known-vulnerable Solidity smart contracts to three frontier LLMs — Claude Opus 4.7, GPT-5.5, and Gemini 3.1 Pro — and asked each one to find the bugs. 168 API calls, ~$5, and a couple of surprises later, here is what the data says. Claude finds the most bugs (98.2%). GPT-5.5 localizes them most precisely (92.9% strict recall). Gemini sits in the middle at 89.3% — but only after I caught a benchmarking gotcha that was silently costing it 20 points. This article walks through how th...