The Only Spec That Never Lies

There’s a particular thing AI coding agents are astonishingly good at, and a particular thing they’re astonishingly bad at, and I think they’re the same thing viewed from opposite ends.

Jeff Dean was on the Y Combinator Startup Podcast recently - the episode titled The 1% Rule for Building in AI - talking about where agents genuinely work today. His example was translating software from one language to another. Ask a model to turn a Python implementation into Go and it will do a remarkable job: port the tests, run them, compare behaviour between the two implementations, and keep going until the differences are gone. His explanation was one line, and it’s the line that made me want to write this:

Because in that case, you actually have an incredibly detailed specification.

That’s it. That’s the whole thing. The reason translation works isn’t that translation is an easy programming task. It’s that translation is the only software task where the specification is already finished.

Zoom All The Way In #

I’ve written before about the coastline paradox of software development: measure a coastline from orbit and you get one number, measure it with a metre stick and you get a bigger one, measure every grain of sand and it approaches infinity. Software refines the same way. A conceptual vision becomes a high-level outline, becomes a detailed spec, becomes code.

The conclusion I landed on then was that code is just the most detailed version of the spec - the one that happens to be executable. Not a separate artefact. Not the output of the spec. The spec itself, zoomed all the way in.

Dean’s translation example is that argument running in reverse, and passing.

When you hand an agent a working codebase and say “make this Go,” you have handed it a specification with properties no written document has ever had:

• It is complete. Every branch, every edge case, every off-by-one someone fixed in 2019 at 2am - it’s all in there. There are no gaps, because a gap in code is a crash, and the thing runs.
• It is unambiguous. Nobody has to interpret it. There’s no illusion of communication to fall into, because the code isn’t communicating - it’s behaving.
• It is verifiable. This is the part people skip past. The tests aren’t documentation about the spec, they’re an executable oracle. The agent doesn’t have to believe it got it right. It can check, get told no, and go again.

Complete, unambiguous, verifiable. Give a human developer those three things and they’ll do great work too. We’ve just never actually been able to give them to anyone.

What The Nuance Actually Is #

Here’s the bit I keep coming back to. A high-level spec doesn’t just have fewer details than the code. It has none of the interesting ones.

The written spec says “users can reset their password.” The code says: the token expires after thirty minutes, except for accounts flagged as enterprise where it’s twenty-four hours, and the email goes out on a delay because the transactional provider used to rate-limit us, and we deliberately return the same message whether or not the address exists, and there’s a comment above that last bit saying don’t change this, see INC-4471.

None of that is in the spec. All of it is load-bearing. Most of it was learned by getting it wrong first.

A human developer, handed the thin version, closes that gap with things a model simply doesn’t have. They’ve watched real users get confused. They know which stakeholder always changes their mind at the demo. They’ve been on call for this system. And crucially, when the gap is too wide to close on experience alone, they stop and ask a question - which is the single most valuable thing a senior developer does and the thing agents are worst at.

An agent, handed the thin version, doesn’t stop. It fills the gap with whatever is most plausible. Plausible is not the same as correct, and the failure is silent. That’s the mechanism behind every story of an agent wandering off, building an elaborate solution to a problem nobody had, or confidently going down a dead end for two hours. It isn’t that the model is stupid. It’s that it was never given enough spec to know it had gone wrong, and had nothing to check itself against.

Translation removes both failure modes at once. There’s no gap to fill, and there’s a red test if you fill one anyway.

Better Specs, Not Better Prompts #

The useful takeaway isn’t “AI is good at translation.” It’s what the translation case tells you about every other case.

Dean’s broader point in that conversation is that specs were always how developers handed work off to each other, and that with agents it matters more, not less - a very good engineer will fill in your gaps and ask sharp follow-up questions, and an agent mostly won’t. The upfront work you used to be able to skip is now the work.

Which reframes what “writing a spec for an AI” should mean. Not a longer prompt. Not a more elaborate document. Moving your spec further down the coastline, towards the executable end.

• Tests are spec. They’re acceptance criteria that can fail.
• Types and schemas are spec. They’re constraints the machine enforces rather than constraints you hope someone read.
• A worked example with real input and expected output is worth a page of prose describing the transformation.
• Existing code in the repo is spec. It’s why agents do better work in a codebase with strong patterns than in an empty directory.

Every one of those does the same job: it takes something that lived in your head as a intention and makes it something that can be checked. That’s the whole difference between the translation case and the greenfield case. Not detail. Falsifiability.

If you can’t give an agent a complete spec - and for genuinely new work, you can’t, because the spec doesn’t exist yet - then give it the shortest possible loop to find out it’s wrong.

The Catch #

Two, actually.

Translation preserves everything, and everything includes your mistakes. A faithful Python-to-Go port carries across the bad data model, the class that should have been three classes, the workaround for a vendor you stopped using in 2022. The spec is perfect at describing what the system currently does, which is not the same as what it should do. You’re re-measuring the same coastline with a different ruler, not choosing a better coastline.

And the reason none of this contradicts what I wrote about tools changing while thinking doesn’t: if code is the most detailed spec, then translating between languages isn’t AI writing software at all. It’s a change of notation. It’s an extraordinarily valuable change of notation - I’d take it, today, for a legacy system I needed off a dying runtime. But the thinking was already done. Someone did it, years ago, badly documented and mostly by accident, and it survives in the only place it ever really lived.

The hard part of software was never the typing. It was never going to be the thing that got automated first.

Final Thought #

Agents are brilliant when the spec is complete and checkable, and lost when it isn’t. That’s not a limitation to wait out. It’s a fairly precise instruction about where to put your effort.

Stop trying to describe the system more thoroughly in English. Start moving more of what you know into a form that can fail.

Because the only spec that never lies is the one you can run.