I was burning context feeding HTML to LLMs, so I built a Markdown converter for agents

The article highlights the inefficiencies of feeding raw HTML to large language models (LLMs) due to the unnecessary data embedded in it, like div tags and other non-content elements. The author discovered these inefficiencies after conducting an audit on GitHub's documentation and realized that converting HTML to Markdown could save significant tokens. This conversion not only improves efficiency but also has broader implications for reducing computational costs and improving the performance of AI agents, scrapers, and retrieval-augmented generation (RAG) pipelines.