AI output is not the bottleneck anymore

Everybody has tried AI by now, but not everyone has gotten outputs they can actually use.

Most people use AI in a simple loop: they write a prompt, run a workflow, use a tool, look at the output, and then manually judge whether it is good or bad. If the output is wrong, they correct it. If it is useful, they move forward.

But this reveals the next major bottleneck in AI adoption: human review.

The bottleneck is not just building better agents. It is not just writing better prompts. The real bottleneck is that human judgment is still trapped inside people’s heads.

Every useful AI system eventually depends on judgment. What is good? What is bad? What is acceptable? What is on-brand? What is legally risky? What is factually wrong? What should be rejected? What should be improved?

Today, most of this judgment is applied manually. A human looks at the AI output and decides whether it works. But if we want AI systems to improve, repeat work reliably, and operate with less supervision, we need to codify that judgment.

That is where evals come in.

Evals are not just tests. They are a way of converting human judgment into a repeatable system. They define what good looks like. Once you can define what good looks like, you can measure it. Once you can measure it, you can improve it. And once you can improve it, AI can start handling more of the workflow without requiring constant human review.

But there is one problem: you can only codify judgment if you have captured it first.

Most people do not have their thinking documented. Their decisions, preferences, corrections, edge cases, reasoning, and context live informally in conversations, calls, Slack messages, notes, and memory. Even if you ask someone, “Do you have everything you do documented in one place?”, the answer is usually no.

The larger thesis is this: the future of AI work is not just humans prompting machines. It is humans continuously externalizing their judgment so machines can learn how to work according to their standards.