What running a team of AI coding agents taught me

I build and run Bifröst, my own team of AI coding agents. It works on real software, around the clock. The first version stalled, so I stopped it, measured what had gone wrong and rebuilt it from scratch.

Most writing about AI agents is about what they might do. This is about what happened when I relied on them.

One agent is a tool. Several agents are a system.

A single agent in a terminal is easy to reason about. You watch it, you correct it, you stop it.

Several agents working at once is a different kind of problem. It stops being about prompts and becomes distributed systems: work gets lost, two workers collide, one waits forever on another. None of those failures look dramatic. The dashboard stays green while nothing moves.

If you're planning to run more than one agent, plan for that. The model is rarely the weak part. The system around it usually is.

What actually broke

Three failures did most of the damage in my first version:

• Agents nobody could really see into. When something went wrong, I couldn't tell what an agent had done or why without digging. If you can't see it, you can't trust it.
• Messages that vanished. One agent told another something, and it never arrived. Nothing raised an error. The work just quietly went wrong later.
• Jobs that sat still. Work ended up in places where nothing would ever move it again. It didn't fail, so nothing flagged it. It simply stopped.

The common thread is that all three were silent. A crash gets fixed in an hour. A silent stall can cost you days before you notice.

Stopping was the right call

The hardest decision was to stop the whole thing rather than keep patching it. Every patch made sense on its own, and together they were making the system harder to understand.

So I paused it, went through what had really happened and rebuilt it in phases. Each phase had its tests written first and had to be proven live before the next one started. That's slower per step and much faster overall, because I stopped building on things I only believed were working.

What I'd tell anyone starting now

Ask for evidence, not reports. An agent saying "done, all tests pass" is a claim. It's usually honest and sometimes wrong. Make the proof the output: the real test run, the real response from the live system. This one habit catches more problems than any other.

Give agents walls, not instructions. "Please don't touch production" is a request. A permission the agent doesn't have is a guarantee. Decide what each agent can't do, and enforce it outside the model.

Keep humans on the decisions that matter. Agents are good at implementation. Scope, direction and anything hard to undo should stay with a person. Where you draw that line is the most important design decision you'll make.

Assume context runs out. Long-running agents forget. Anything that has to survive belongs somewhere durable, not in a conversation that might be summarised away.

Review with fresh eyes. An agent reviewing its own work tends to agree with itself. A separate reviewer that never saw the code being written finds far more.

Does this replace engineers?

Not from what I've seen. It changes what the job is. I spend less time typing and more time deciding what should exist, checking whether it's right and designing the system that does the work. You need more engineering judgement to run this well, not less, because mistakes now arrive faster and sound more confident.

What it does change is scale. One engineer who runs agents well can now ship what used to take a small team, which is how I can build Helm, run it for 10+ paying businesses and still ship client work.

If you want the longer version of how I work with models and harnesses, I've written it up on my AI engineering page.