Shadow Web cuts LLM web-page tokens 64–97% with open-source tool

Web pages contain far more HTML than AI agents need to see: inline stylesheets, JavaScript bundles, tracking pixels, and dynamically-rendered shadow DOM elements buried in Web Components. Most of this noise becomes tokens sent to an LLM, inflating API bills without adding semantic value. Shadow Web sits between the browser and the LLM, reading the live DOM (including shadow trees), stripping decorative markup, and organizing what remains into a sparse Action Map—a list of clickable elements with labels and semantic groups (like "Login", "Cart", "Navigation"). This cuts raw token load by 64–97% depending on the page.

The tool also solves a secondary pain point: Playwright and other browser automation libraries struggle to reach elements inside shadow DOM or closed Web Components, and selectors break when the DOM changes. Shadow Web provides self-healing selectors that first try fuzzy local recovery, then fall back to LLM verification if needed—no cloud roundtrip required for most failures. For teams building AI agents that navigate the web, this combination of compression, structural parsing (SchemaSnap), and local selector repair reduces both API costs and error rates.

The package is pip-installable and includes turnkey integrations for Cursor and Claude via an MCP (Model Context Protocol) server. It works offline and produces no side effects on the original page, making it safe to use with any framework (React, Vue, Svelte, etc.). Benchmarks on real sites show token reductions of 6× on Wikipedia and 4.4× on GitHub Trending.