Your Website’s New Audience: AI Agents

Web performance has traditionally always been about human user experience. In recent time, that has evolved to include AI agents as well. AI agents can be thought of as anonymous and agentic systems that browse, consume and interact the web on behalf of users. As more and more people continue to use AI agents or search engines pushing AI mode, it is recommended to optimize for agentic browsing as well. Google has recently included an agentic browsing category for both lighthouse and pagespeed insights, although they are currently experimental. How do we do that? Broadly speaking, there are a few basic steps that can be covered here: 1. Accessibility Tree: When agents browse your page, they do not typically "see" the page as a human would. They parse the DOM structure to analyze element roles, names and relationships. If any part of the tree is incomplete, it may lead to broken data and checkout flows. Google Lighthouse today measures this by the following methods: Every interactive element (button, input, link) needs to have a text node or aria-label assigned to it so they are "visible" to agents. Parent - child relationships or sibling relations must be clear in the tree. For example, a label must be clearly tied to an input and so on. This is for better context of the general page structure and executing workflows. Interactive elements should NOT be hidden - A clickable Submit button hidden via CSS or aria-label confuses agents. 2. Proper Metadata: A llms.txt present at the root of your domain summarizes key pages and sections for the agent to browse for any given task without having to hit every page unnecessarily. There are more advanced techniques here like schema.org and llms-full.txt for a more enhanced summary. 3. Cumulative Layout Shift (CLS): Cumulative Layout Shift is already a part of traditional web performance metrics but for agents, it is even more crucial as layout stability ensures that the agent does not misclick or misfire anything in a given workflow or task. 4. WebMCP Adoption: With WebMCP, agents are provided with APIs that enable them to directly take any action based on the user prompt. There are currently two ways to implement this - a declarative API, defined directly with HTML forms or imperative API, with JavaScript. This is currently available under an origin trial - meaning you can actually try this out today in production with real users. This has practical use cases - like ecommerce, travel and general automation of day-to-day tasks. Closing thoughts I'm not sure the future of the web will be fully autonomous (or if we would have evolved to some new interface) but for the moment, the best strategy is: build your page for humans and optimize it for agents.