AI coding assistants are powerful, but they are only as good as their understanding of your codebase. When we alerted AI agents to one of Meta’s large data processing pipelines – spanning four repositories, three languages ​​and over 4,100 arquivos – we quickly discovered this They didn’t make useful changes quickly enough.

We solved this problem by developing a pre-computing engine: a swarm of more than 50 specialized AI agents that systematically read each file and create 59 concise context files that encode tribal knowledge that previously lived only in the minds of engineers. The result: AI agents now have structured navigation guides for 100% of our code modules (up from 5%, all covering over 4,100 files across three repositories). We have also documented more than 50 “non-obvious patterns.” underlying design decisions and relationships that are not immediately apparent from the codeand preliminary testing shows 40% fewer AI agent tool calls per task. The system works with most leading models because the knowledge layer is model independent.

The system also maintains itself. Every few weeks, automated jobs regularly validate file paths, detect coverage gaps, rerun quality critics, and automatically repair outdated references. AI is not a consumer of this infrastructure, but rather the engine that runs it.

The problem: AI tools without a map

Our pipeline is Config-as-Code: Python configurations, C++ services, and hack automation scripts that work together across multiple repositories. Onboarding a single data field affects configuration registers, routing logic, DAG composition, validation rules, C++ code generation, and automation scripts – six subsystems that must remain in sync.

We had already developed AI-powered systems for operational tasks that scan dashboards, match patterns with historical incidents, and suggest remedial actions. But when we tried to extend it to development tasks, it failed. The AI ​​had no map. It was not known that two configuration modes use different field names for the same operation (swap them and you get silent wrong output) or that dozens of “deprecated” enum values ​​must never be removed because serialization compatibility depends on them.

Without this context, agents would guess, explore, guess again, often producing code that compiled but was subtly wrong.

The approach: Teach agents before they explore the environment

We used a large context window model and task orchestration to structure the work into phases:

• Two Explorer agents mapped the code base,
• 11 module analysts read each file and answered five key questions:
• Two authors created context files and
• 10+ critics passed three rounds of independent quality review,
• Four fixers made corrections,
• Eight upgraders have refined the routing layer,
• Three prompt testers validated more than 55 queries across five personas.
• Four gap fillers covered the remaining directories and
• Three final reviewers conducted integration tests – over 50 specialized tasks orchestrated in a single session.

The five questions each analyst answered per module:

1. What configures this module?
2. What are the most common modification patterns?
3. What non-obvious patterns cause build errors?
4. What cross-module dependencies are there?
5. What tribal knowledge is contained in code comments?

Question five was where the most profound insights were gained. We found more than 50 non-obvious patterns, such as hidden intermediate naming conventions, where a pipeline stage emits a temporary field name that a downstream stage renames (point to the wrong one and code generation silently fails), or append-only identifier rules where removing a “deprecated” value breaks backwards compatibility. None of this had been written down before.

What we built: A compass, not an encyclopedia

Each context file follows what we call the “compass, not encyclopedia” principle – 25-35 lines (~1,000 tokens) with four sections:

1. Quick commands (copy-paste operations).
2. Key files (o 3-5 files you actually need).
3. Non-obvious patterns.
4. See also (cross-references).

No frills, every line deserves its place. All 59 files combined take up less than 0.1% of the context window of a modern model.

Adicionalmente, we built an orchestration layer that automatically directs engineers to the right tool based on natural language. Type “Is the pipeline OK?” a. Dashboards are also scanned and matched to more than 85 historical incident patterns. Enter “Add new data field” and the configuration will be generated with multi-step validation. Engineers describe their problem; The system takes care of the rest.

The system updates itself every few weeks, validating file paths, identifying coverage gaps, re-running critical agents, and resolving issues automatically. A context that decays is worse than no context at all.

Beyond individual context files, we created a cross-repository dependency index and data flow maps that show how changes propagate across repositories. This yields “What depends on X?” from a multi-file exploration (~6000 tokens) to a single graph search (~200 tokens) – in config-as-code, where a field change propagates across six subsystems.

Results

Metric	Before	After
AI context coverage	~5% (5 arquivos)	100% (59 arquivos)
Codebase files with AI navigation	~50	4,100+
Tribal knowledge documented	0	Sobre 50 non-obvious patterns
Tested Prompts (Core Success Rate)	0	55+ (100%)

In preliminary tests on six tasks in our pipeline, agents with precomputed context used approximately 40% fewer tool calls and tokens per task. Complex workflow instructions that previously took about two days Research and consultation with engineers is now completed in about 30 minutes.

Quality was non-negotiable: three rounds of independent review agents improved the score from 3.65 to 4.20 out of 5.0, and all referenced file paths were checked without hallucinations.

Challenging the conventional wisdom about AI context files

Youngest academic research found that AI-generated context files actually reduced agent success rates in well-known open source Python repositories. This result deserves serious consideration, but it has one limitation: it was evaluated against codebases such as Django and Matplotlib, which models already “know” from pre-training. In this scenario, context files are redundant noise.

Our codebase is the opposite: proprietary configuration as code with tribal knowledge that is not present anywhere in a model’s training data. Three design decisions help us avoid the pitfalls that research has identified: files are concise (about 1,000 tokens, no encyclopedic summaries), opt-in (loads only when relevant, not always active), and quality-driven (multi-level critical review plus automatic self-update).

The strongest argument: Without context, agents waste 15 to 25 tool calls exploring, miss naming patterns, and produce subtly incorrect code. The cost of not providing context is measurably higher.

Here’s how to apply this to your codebase

This approach is not specific to our pipeline. Any team with a large, proprietary codebase can benefit:

1. Identify your tribal knowledge gaps. Where do AI agents fail the most? The answer is usually domain-specific conventions and cross-module dependencies that are not documented anywhere.
2. Use the “Five Questions” framework.. Let agents (or engineers) answer: What does it do, how do you change it, what breaks, what depends on it, and what is not documented?
3. Follow Compass, Not Encyclopedia..“ Limit context files to 25-35 lines. Actionable navigation surpasses comprehensive documentation.
4. Build quality goals. Use independent critique agents to evaluate and improve the generated context. Don’t trust unaudited AI output.
5. Automate freshness. An outdated context causes more damage than no context. Regular validation and self-repair of the build.

What’s next?

We extend context coverage to additional pipelines in Meta’s data infrastructure and explore tighter integration between context files and code generation workflows. We also investigate whether the automatic update mechanism can detect not only stale context, but also emerging patterns and new root knowledge formed in recent code reviews and commits.

This approach transformed undocumented tribal knowledge into a structured, AI-readable context that connects to each subsequent task.