When the system writes its own architecture docs
Heads-up before we start: Atalaya is still WIP — I'll open-source it once it stabilizes. What follows is the working design, not a finished product. Architecture documentation is the file you open when you land in a new repo and pray it's up to date. It almost never is. In Atalaya the system writes its own ARCHITECTURE.md, LAYOUT.md and PROJECT.md on every commit. Not because it's elegant — because the alternative is that they lie.
I've spent enough years opening README files that begin with "this section is out of date" to know which kind of doc humans are good at and which kind they aren't. The distinction is what made this worth building, not the automation itself.
The Unpopular Hypothesis: Manual Docs Are Dead
Let me be precise about which docs. Not vision docs. Not ADRs. Not runbooks. Those stay human because their content is something only a human can supply — a reason, a tradeoff considered and rejected, a piece of context that lives in someone's head.
The docs I'm talking about are the descriptive ones. What's in this repo. How is it organized. What depends on what. Which directory holds which concern. These are the files that lie systematically, and they lie because the code underneath them changes faster than the disciplined human updates the description. Every commit is a small invalidation. Most commits go unanswered.
What I've noticed — and this isn't comfortable to write — is that the more senior the team, the more embarrassing the docs get. Junior teams either write good ones because they're being graded on it, or write none at all and that's honest. Senior teams write them once, ship the project, and then leave them there knowing they're wrong. The doc says "this module handles X" and everyone on the team knows it actually handles X, Y, and half of Z now. Nobody updates it. The shame compounds.
The deeper issue is that the cost of updating a descriptive doc is high (you have to read your own code, distill it, write English) and the benefit is invisible (nobody thanks you for an accurate LAYOUT.md). The economics are wrong for humans. They're correct for an agent.
There's a second-order point worth naming. A stale descriptive doc is not just useless — it actively damages the team. New engineers anchor on it. Conversations refer to it. Architecture diagrams get drawn from it. The wrongness propagates outward into other documents, into onboarding videos, into RFC templates. By the time someone notices, the misinformation has taken root in five places. Removing a wrong doc is harder than removing a missing one, because the wrongness has already been laundered through other artifacts that look authoritative.
What RepoDigestAgent Does
The component is called RepoDigestAgent. It runs on every commit — pre-push git hook for fast feedback, redundant CI run for the guarantee. It reads the repo tree, the import graph, the entry points, the test structure, the package metadata. It builds a snapshot.
Then it calls the LLM with that snapshot plus three fixed templates, and produces three files inside .atalaya/:
ARCHITECTURE.md— what the module does, how it breaks down, the relationships between its partsLAYOUT.md— a directory map with a one-line description for each directoryPROJECT.md— purpose, owners, basic runbook, external integrations
The three files are small and bounded. They don't try to be exhaustive. They try to be true. A new developer who opens .atalaya/LAYOUT.md should get oriented in a minute and a half — what's where, what to read first, what not to bother with on day one.
The Decisions That Keep This From Becoming a Nightmare
LLM-generated documentation is the kind of idea that looks great in a demo and turns into noise in production. The decisions below are what kept it from being noise.
The LLM doesn't get free rein. It gets a schema and fixed templates. The output is predictable in shape: same headings, same field order, same length budget. We're not asking the model to compose. We're asking it to fill in a structured form. That makes the output diff-able commit over commit, which is half the value.
If the result doesn't parse against the schema, the commit is rejected. That's the load-bearing rule. Without it, the system slowly degrades into corrupt docs nobody trusts. With it, the docs are either present and valid or absent — never wrong-shaped.
The generated files are first-class citizens. The rest of the ecosystem reads them. Akopia indexes them, other agents consult them when they need to understand a repo they haven't seen. They're not a courtesy artifact for humans — they're an input to other systems.
And they live in .atalaya/, clearly marked. There's no "auto-generated" pretense that the file might be hand-edited. The folder name announces what it is. In repos where the cost isn't worth it, .atalaya/ is gitignored and the agent doesn't run. Opt-out is one line.
One more rule, learned the hard way: never let the model invent structure that isn't in the code. The first version of RepoDigestAgent occasionally hallucinated a clean architectural story when the code was actually a mess. That was worse than no docs — it was confident lies about a state that didn't exist. The fix was to constrain the prompt to describe what's in the snapshot, not what would be elegant. Generation has to be honest about ugly code. If the directory tree is chaotic, the LAYOUT.md should read as chaotic. The doc is a mirror, not a beautification.
The Cost (Which Is What People Ask About First)
Nobody believes me on the cost until I show numbers, so let me show them.
On small repos (one module), about 30 seconds per commit. On big repos (Atalaya itself), about 2 minutes. The local hook is optional — developers who don't want the latency skip it. CI absorbs it on the next push, which is the actual guard.
API costs run about $0.02 to $0.05 per commit. Claude Haiku or an equivalent class model is plenty — the task is structured enough that you don't need the heavy model. Across my 8 active repos, monthly spend lands between $5 and $15. Orders of magnitude less than the cost of stale docs onboarding the next engineer.
The non-monetary cost is the one to watch: discipline. The schema has to be maintained. When the templates evolve, all the existing .atalaya/ files get regenerated on next commit, and the diffs are loud. That noise is real and you absorb it the same way you absorb a linter version bump.
What Changed in How I Run the Team
The behavior changes were what surprised me. Onboarding a new developer became "open .atalaya/ before you read the code" with the actual guarantee that the file isn't lying. That changes the first-day experience.
In code review, structural changes started surfacing through the docs. When someone adds a new module or rewires an existing one, the diff of .atalaya/ARCHITECTURE.md shows up in the PR alongside the code diff. That diff is a forcing function for discussion — "you're claiming this is now what the module does; is that what we want it to be?" It catches scope creep that pure code review misses.
External agents — other Atalaya agents working in unfamiliar repos, or client agents operating on repos they've never seen — consult these files to orient themselves. The descriptive layer became a kind of API for tooling, not just for humans.
The most convincing data point came from an experiment. I turned RepoDigest off on one repo to see what would happen. The docs were stale within three weeks. Not catastrophically — just enough that the LAYOUT.md was wrong about two directories and ARCHITECTURE.md described an old module boundary. Three weeks. That's the half-life of accurate descriptive docs in an active repo.
What It Doesn't Do (and Why That's Fine)
It doesn't replace ADRs. Decisions stay human documents — the rationale for choosing one approach over another isn't in the code, and no model can infer it from the artifact.
It doesn't replace operational runbooks. Those need context the code doesn't expose: pager rotations, escalation paths, "we tried this before and it made things worse."
It doesn't replace product vision. No LLM is going to infer why the project exists, who it's for, what success looks like. That's upstream of code.
What it replaces is the descriptive-of-state class — the docs no human should be hand-writing because the cost is high, the benefit is invisible, and the failure mode is silent rot.
Transferable Principle
Docs that drift are docs that lie — and they lie with confidence, which is worse than not having them. A confident wrong doc actively misleads. A missing doc at least announces the gap.
The only descriptive docs that survive are the ones the system regenerates. Everything else decays.
The mental rule I use now: if the document describes what's there (structure, layout, dependencies), automate it or assume it will lie. If it describes why or how (decisions, vision, runbooks), keep it human and accept the cost of maintaining it. The two are different categories of work, and they shouldn't be governed by the same process.
Which doc in your repo do you no longer trust? And why is it still there with no asterisk saying "last updated: 14 months ago"? Send me a DM or reach out via the contact channels at rlabs.cl.
#MetaSoftware #ExecutableKnowledge #DeveloperExperience #Architecture #Engineering