I tested ChatGPT, Claude, and Gemini on topics I actually know, and the results were embarrassing

Published Oct 10, 2026, 6:00 AM EDT Megan is a Software Writer at XDA Developers who has been writing about consumer technology since 2016. With a postgraduate degree in New Media journalism, she has always been passionate about diving deep into topics and writing about them in a way that makes them easier to understand for everyday consumers. She runs her own blog called Tech Valkyrie and has contributed to Tom's Guide and Android Authority. She also worked at MakeUseOf as a Lead Editor. In her spare time she plays video games and geeks out about the latest tech. Even though I'm relatively wary of AI answers, I still occasionally turn to AI tools to quickly understand certain topics. This is especially true when I use Brave Search to get an answer to a question. But there's a reason I take these answers with a pinch of salt. Often when I ask AI tools about something I already have a deep understanding of, I start to notice flaws in their answers — and sometimes, outright hallucinations or bad advice. AI answers sound good until you start picking up the mistakes Sometimes the advice is just downright bad AI chatbots have a way of being confidently incorrect — presenting dubious information as completely true. I've even had to argue with tools like Gemini when I know an answer is wrong. Eventually, the chatbot relents, but I could've walked away having taken the first response as the truth. This happens across topics, from low-stakes image identification to queries about subjects I know well. Recently, I decided to replicate this by asking ChatGPT, Claude, and Gemini about whether they would recommend a phone with a low PWM dimming rate to someone with migraines. For context, low PWM dimming rates are associated with screen flicker at lower brightness, which may affect certain sensitive individuals and cause headaches, migraines, nausea, and eye strain. All the AI tools correctly noted that while PWM dimming rates affect people differently, someone with chronic migraines should err on the side of caution and choose a phone with DC dimming or a high PWM dimming rate. But when it came to product suggestions, the cracks started to show. ChatGPT overlooked key specs based on its own suggestions, only adjusting its ranking when I pointed out that a lower-ranked phone had a top-tier trait: DC dimming. It then corrected its ranking. But its shortlist also included the Samsung Galaxy S26 Ultra, which it actually recommended against buying later in its own response. Quizzing Gemini similarly revealed a decline in reliable answers. Gemini suggested that someone with chronic migraines turn their phone up to full brightness and then use third-party software or built-in features to lower the brightness. While this solution could eliminate flicker to some extent, it overlooks a key consideration: bright screens worsen and trigger migraines. Even with a feature like Extra Dim or Reduce White Point enabled, 100% brightness is still too bright. I even tested this on my Samsung Galaxy S23 Ultra to confirm it. Third-party screen-dimming apps can also pose a security risk, as they let the app apply an overlay on your screen. This is something the AI failed to mention. For product recommendations, Gemini suggested smartphones that were a few generations old, despite the prompt including no price range or time frame. For example, it suggested the Honor 200 series, even though Honor's latest number series is the Honor 600. In fact, it recommended only phones from 2023 to 2024 for no apparent reason. Meanwhile, Claude just suggested brands rather than specific phone models. However, at least the information was accurate, even though it was vague. The answer depends on the chatbot For questions with objective answers, you'd expect AI tools to reach similar conclusions. But sometimes AI tools will steer you in completely different directions. I noticed this recently when trying to propagate some new plants from my succulents. I asked whether I could use a flower stem to propagate new pups. Gemini confidently said I could do this and gave me instructions. A few days later, when I wanted to remember the steps, I asked Brave Search how to propagate succulents from the flower stem. Instead, it told me this was not possible. This contradiction is not a one-off quirk either. Today, when I quizzed ChatGPT, Claude, Gemini, and Brave Search about this propagation method, the chatbots gave conflicting answers. Claude and Brave Search took the stance that, generally, you can't propagate succulents this way. But Gemini and ChatGPT said this was possible. Gemini didn't seem to doubt it, but ChatGPT noted it's only sometimes achievable. Sources and models matter Some tools perform better than others Whenever I prod AI tools for answers, Claude and Brave Search tend to perform the best. Part of this is because the tools can express uncertainty, but also because of the general quality of sources. That said, they don't avoid faltering. Brave Search, especially, can prioritize sources that aren't necessarily authoritative. For example, if I ask the search engine a really niche question, it often cites Reddit as its source. I've had responses that essentially rely on one Reddit comment as the citation. This highlights how sources and model behavior can shape answers. Even NotebookLM, which tends to hallucinate much less than other AI tools, can go awry with the wrong sources. To give the tools credit, however, they did explain a few topics well when there were many sources available. But it's when you get into the nitty-gritty that they often seem to falter. Claude, though, remains the tool that impresses me the most. It's a reminder to fact-check the answers AI gives you There are constant disclaimers that AI can get things wrong, yet we still see people use AI answers as a single source of truth. But when you really start testing their knowledge, it becomes clear how often these answers overlook key factors, contradict themselves, or just provide low-quality information. Using AI answers for low-stakes queries won't backfire too much. You might just waste some time trying to grow a succulent the wrong way. However, if you want reliable information on topics you're not sure about, check sources and get answers from authoritative outlets.