Everyone Is Selling AI at You — Here’s How to Keep Your Judgement

If you work in product or tech, you have probably sat through this meeting. Someone forwards a vendor demo or a viral post, and suddenly the roadmap needs an agent, an MCP app, or a harness. We see the same scene in client meetings all the time, usually starting with “we need an agent for this.” Often, after decomposing the request from first principles, it turns out the client needs something completely different: a custom predictive model, a better use of the LLMs they already have, or no AI at all. Someone has to say “not so fast”, without sounding defensive.That’s not easy, because the AI conversation is loud and convincing. This article gives you three moves to stay grounded:Understand how the AI value chain works, and where you realistically sit in it.Build a foundation of knowledge that lets you structure and reuse what you learn about AI.Consume information intentionally, keeping your own perspective.Together, these practices make you a stronger sparring partner when the next hype wave hits your team. At the end, you’ll find six common vendor claims and the question that deflates each one.1. Understand your place in the AI value chainBefore you can judge an AI claim, you need to know who is making it and why. This section maps the AI ecosystem, shows where you likely sit in it, and explains how that position might work against you.Who sells what to whom?The AI value chain can be modeled in five main layers, from the chips at the top to the companies that put AI to work at the bottom:Figure 1: Mapping the AI value chainAt the top, returns are close to banked: the silicon is sold and paid for on delivery. As you move down, value gets less certain. The cloud providers and model labs are betting on future demand, so their real returns are harder to pin down. A lot of that “demand” is actually the same money circulating inside the ecosystem: chipmakers fund the labs, labs commit to the clouds, and the clouds buy chips. Real end-user demand is only decided at the end of the chain.The odds are stacked against AI consumersIf you are reading my work, chances are you sit at that receiving end, as an enterprise AI adopter or application developer consuming models, tools, and platforms from the layers above. You are part of the end-user demand, and the whole ecosystem is working hard to maximize it. That puts you in a weak position:Everyone is selling at you. AI companies have perfected the art of marketing. Their message is that AI is cheap, easy, works out of the box, and will transform your life and business. I like to call this the “accessibility illusion.” The more uncertain the actual product, the heavier the marketing behind it.You know less. Vendors know the limits of their products, but you often discover them only when you are already struggling with the last mile: the dangerous stretch between a demo and a system that delivers value to real users.Your payoff comes later. Vendors get paid when you buy; you get paid only when the system works and you can prove it. Getting from raw ingredients like models, APIs, and agent frameworks to measurable value takes a mature mix of conviction, technical skill, and business knowledge.The bottom line: AI’s last mile is still mostly undone. In McKinsey’s 2025 survey, more than 80% of companies using generative AI said they had not yet seen a clear impact on their overall profits. RAND reports a failure rate above 80% for AI projects, although this covers more than just projects that never reach production. And even a deployed system doesn’t guarantee value. Many companies have no reliable way to measure whether AI actually improves business results, so the loop stays open (cf. Dataiku’s 2026 CIO survey).Developing good judgment about AI and learning to apply it in your company’s context is your main defense against falling for the hype.2. Build a foundation of knowledgeGood judgment comes from knowing the basics well enough to see through the noise.Peeling off the emotional layerMost AI content mixes facts with emotions that were added on purpose: excitement, urgency, fear of missing out. Emotion works even on experienced people because it exploits three blind spots:Demos over workflows. A demo shows the best of one run. Production means the same task a thousand times, edge cases included.Benchmarks over your data. Benchmarks measure models on clean, often public datasets, not on your messy internal ones.Survivorship in case studies. Vendor case studies feature the projects that worked, not the ones that were quietly shelved.To peel off this subjectivity, you need to understand the basics of how AI works, and in particular its limitations and risks. Without that foundation, you are mentally building a house of cards. Each card is a headline, a demo, or a vendor claim, propped up by the others. The structure can grow impressively tall, but one sharp question can bring it down.Structuring your AI knowledgeA solid foundation grows more slowly, but everything you learn later has a designated place to rest. In my view, it has two essential components:The math, first-hand or second-hand. AI is rooted in linear algebra, probability theory, and calculus. That’s how you get to know its intrinsic limitations — like the fact that today’s language models regularly fail by design because they estimate probabilities. Learning the math takes years; if that isn’t realistic, borrow it from a few experts whom you trust.Systems thinking. To uncover the value of AI in your specific business context, you need to understand how they connect and interact. Good starting points are Donella Meadows’ Thinking in Systems or Shane Parrish’s The Great Mental Models; for AI, our AI Strategy Playbook maps some of the mental models that we use across our work with clients.With this foundation, claims start to sound different. On a house of cards, a vendor promising “zero hallucinations” sounds great. On a solid foundation, it sounds like a question for the next call: zero, measured how, and on which data?3. Consume information intentionallyHow do you find credible sources and real insight in an overwhelming sea of AI content? In this section, I share the mental habits that help me recognize content that will actually teach me something new.Understand who benefitsBehind most sources sits someone who benefits when you follow their call to action, and their incentives are likely different from yours. To keep your own perspective, it helps to invert the typical flow of content creation:Facts: a statistic, a benchmark, a survey result.Story: examples, customer quotes, and emotions wrapped around the facts.Action: the step the story moves you toward, like booking a demo or buying a platform.Read backwards from the action, and it becomes clear which facts were selected and why. The Dataiku survey I cited above is a good example. The data is useful, but it was commissioned by a company that sells agent management software, and the story of CIOs losing control nudges readers straight toward that product. That doesn’t make it wrong: use the numbers, but discount the conclusion.Pay attention to languageHow a piece is written often tells you more than what it claims. Watch out for these red flags:AI slop. Saying something genuinely new about AI is hard. People who make that intellectual effort tend to put their thoughts in their own words, edit heavily, and disclose when they used AI. Polished, generic language signals recycled content.Anthropomorphic framing. Enterprise vendors increasingly frame AI products as “digital workers” that join your team. This invites us to think in headcount rather than outcomes, which makes ROI promises feel intuitive before they can actually be measured.Inflated emotions. Headlines about a looming job apocalypse or AI-induced threats to humanity are rarely just journalism; often, there is a marketing machine behind them. Fear cuts both ways: if a technology is powerful enough to end the world, surely it is also powerful enough to transform your business (see Lee Vinsel’s Notes on Criticism and Technology Hype). When you break these claims down to first principles, they rarely hold up as stated.Heavy jargon. Jargon often dresses up an existing concept as something new. Gartner, for example, describes “agent washing” as the rebranding of existing products, such as AI assistants, robotic process automation (RPA), and chatbots, without substantial agentic capabilities. Many use cases positioned as agentic today don’t actually require agentic implementations.Also, look at how a product reaches you. When the value is obvious, a company can afford to let the product speak for itself. Cursor’s founders did no outbound sales until late 2025: the product was useful from day one, and users did the marketing. When the value is uncertain, the marketing gets louder instead. Builder.ai promised to make software creation “as easy as ordering pizza” and marketed its AI assistant Natasha as a breakthrough. In 2025, the company filed for bankruptcy amid financial scandals and accusations of AI washing (Wikipedia).Putting it together: my filter for new AI conceptsAs much as I love exploring new AI innovations, running two companies leaves me limited time for experimentation. Here is the filter I use whenever a new AI idea or concept hits the headlines:Triage. I ask two questions. Is it genuinely new, or a rebrand of something that already exists? And is it strategically relevant for our work?If both answers are yes, go deep. I read the primary sources and try it out myself, ideally on real data.If not, park it. I check back once third-party data and opinions from experts I trust become available.Revisit with evidence. If the data shows promise, the idea goes back to step 2.Figure 2: A new AI idea earns attention through relevance or evidenceInternally, we also develop and use the AI Radar as a quantitative overview of the AI landscape. Looking at numbers and trend curves is another great way to make your decisions more objective.Conclusion: Pushing back without being “against AI”You don’t need to win a heated argument about whether we are in a bubble. Rather, you need a decision logic that holds up either way. Next time an AI claim reaches you, decode it first:Table 1: Decoding AI claimsUse questions like these to break an AI idea down to what can be verified. Over time, you will learn to uncover gaps and recognize those ideas that are feasible and can deliver true value in your business.Which hype claims are you pushing back on right now? Leave a comment, and I may pick them apart in a future article!Note: All images are by the author.