← All posts

We let three flows run our backlog. Here is one issue, start to merge, for 76 cents

Dimitris Moraitis · ·5 min read
  • software factory
  • AI agents
  • flows
  • AI code review
  • Open source

The video is the 90 second cut. Below is the whole run, with the numbers.

Every engineering team we know carries the same kind of debt. Not the big rewrite. The small stuff: a misaligned column, a missing test, a label that says the wrong thing. Nobody disagrees that it should be fixed. It just never wins a planning meeting.

We stopped trying to schedule it. On the Preloop repository, small issues now move through a chain of agents. One plans the work, one writes it, one reviews it, and the writer fixes what the reviewer finds. A person files the issue at the start and merges at the end.

We call this the software factory. Each stage is a Preloop flow, so it runs with the agent, model, policies and keys we chose, and it leaves a full record. The point is simple: the cheap work gets done, and our engineers spend their attention on the work that actually needs a human.

One run, step by step

On 4 October 2026 one of our developers filed issue #1250: "Running rows on the Flow executions page overlap the model column". That was the whole report.

Triage: 4m42s, $0.21. The Issue Triage Assistant read the relevant code and wrote a plan into the issue, marked "Ready for development". The plan covered the cause, how to reproduce and check it, the change to make, and acceptance criteria. It then labelled the issue complexity:low, risk:low and readiness:ready, and added the dispatch label for the next stage.

Implementation: 9m15s, $0.19. Two seconds after the labels landed, the Automated Issue Implementation flow started. Its agent worked from the plan in a worktree, wrote a test and committed. The agent never touched a git credential. The flow held it, and the flow opened pull request #1252.

Review: $0.10. The Pull Request Reviewer ran on #1252 and left two findings.

Repair: 7m46s, $0.20. That review woke the implementer back up on the same branch. It fixed both points and replied under the heading "Addressed both review items". In the same reply it pointed out that the reviewer had got some arithmetic wrong in one finding. We like that. A review loop is only useful if both sides are allowed to push back.

Re-review: $0.06. Second pass: everything addressed, ready to merge.

Then one of us merged it. From the moment the issue was opened to the merge, nobody touched it. Total model cost was about $0.76.

The controls around it

We would not run this on our own repository if we could not see inside it. Every stage uses the same governance as any other Preloop agent.

  • Policies decide the tools. An agent gets what the account's policies allow, and each flow can narrow that with its own overrides.
  • The back and forth has limits. Repair has a cost budget and stops after 5 turns, so two agents cannot keep arguing on our bill.
  • Not every comment counts. Only trusted reviewers can resume the implementer. A comment from a random account does nothing.
  • The flow keeps the keys. The agent edits code. The git credential and the pull request belong to the flow.
  • You can replay any stage. Every model and tool call passes through the gateway. Each step can be stepped through request by request, and cost is metered per issue against our own keys.

Setting it up yourself

All three flows ship as presets in github.com/preloop/preloop under backend/preloop/presets/:

  1. Issue Triage Assistant: turns a report into a plan with acceptance criteria and tags complexity, risk and readiness.
  2. Automated Issue Implementation: takes issues with the dispatch label, implements them and opens a pull request.
  3. Pull Request Reviewer: reviews each pull request, and reviews again after every fix.

The steps:

  1. Install with curl -fsSL https://preloop.ai/install/cli | sh. Preloop is Apache 2.0 and you can host it yourself. If you would rather not, there is Preloop Cloud.
  2. Connect GitHub or GitLab.
  3. Add the presets as flows. Choose an agent and a model for each one. Model calls go through the Preloop gateway using your own keys.
  4. Route on the complexity tag. Triage already set it, so the implementation flow can send low complexity issues to an affordable model and harder ones to a more powerful model. You only pay for extra reasoning where it helps.

Tag routing and per-flow policy overrides are new this week (#1212, #1214, #1246).

Filming it taught us more than using it

None of this was set up for the video. The Pull Request Reviewer already reviews every Preloop pull request, and our backlog already goes through the factory. But using something every day and watching one run end to end, console open, no cuts, are different tests.

The second one found four bugs. We fixed each on the day:

  • concurrent webhooks could create duplicate issue rows (#1199);
  • resumed runs had a budget bug (#1204);
  • the executions list query took 8 to 15 seconds (#1205);
  • two console screens showed different cost figures for the same thing (#1305).

You do not find bugs like these by describing a system. You find them by watching it work.

That is the real trade. A factory does not mean you stop checking. It means checking is cheap, because each step is logged, priced and can be replayed.

Where to start

Do not switch on all three at once. Begin with the Pull Request Reviewer: install, connect a repository, and your next pull request gets reviewed. Add triage once the reviews earn your trust. Add implementation once the plans do.

Your small issues have waited long enough.