Meta built an AI system to map 4,100+ files of hidden engineering knowledge, cutting AI agent errors by 40%

Meta created a swarm of 50+ specialized AI agents that systematically read every file in a massive data pipeline (4,100+ files across Python, C++, and Hack code) and distilled tribal knowledge—conventions and design patterns that only existed in engineers' heads—into 59 concise context files. Before, AI agents could navigate only ~5% of the codebase; now they can navigate 100%.

The context files follow a 'compass, not encyclopedia' design: each is only 25–35 lines (~1,000 tokens), containing quick copy-paste commands, the 3–5 files you actually need, and non-obvious patterns (like hidden naming conventions where one pipeline stage outputs a temporary field name that a downstream stage renames). AI agents using these files used roughly 40% fewer tool calls and tokens per task, and complex workflows that previously required ~two days of engineer consultation now complete in ~30 minutes.

For any team with a large proprietary codebase, this matters because undocumented domain-specific conventions and cross-module dependencies are where AI coding assistants fail most. Meta documented 50+ of these 'non-obvious patterns' that cause silent failures—like deprecated enum values that must never be removed due to serialization compatibility. Engineers can now ask natural language questions ('Add a new data field') and get multi-phase validated output instead of having to map the codebase manually.