A software factory is the idea of a system where you push work in one end, agents create software, test it, review it and deploy it at varying levels of automation.
Lots of people are experimenting with these patterns. Including me.
v0.1 with Cursor
I really got into this using Cursor and Linear. This isn’t Cursor the AI-connected IDE, which was fun but not the game changing software factory. Cursor’s real power is its AI agent framework.
Here’s how the workflow happened:
- Linear to track issues and assign work
- To Cursor which uses different models to do the work
- Then pushes PRs to Github
- Github Actions build and run automated tests
- Github Actions run reviews of the code
- Agents keep working until the tests pass and the reviews are green
- Human merges the PR
- Github Actions builds and deploys
This worked really well and once I discovered Cursor’s Composer 2.5 model, chunking maintenance and minor changes into tickets became super cheap tasks to do. Composer 2.5 is a reinforcement on top of Kimi’s 2.5 model. Fast, effective and with the guardrails of tests and reviews, very good at what it does.
The reviews are really quite fun. I’ve created a bunch of different reviewers for different angles. Security, design, usability, analytics tracking, performance, cost, test coverage. So much so that I’ve got a triage agent that decides which review agents to kick off.
Then Cursor got bought by the Space Nazi. Dammit. Gotta leave.
Looking at alternatives
I spent some time disentangling from Cursor and wasn’t really satisfied with any alternatives. I tried Factory (couldn’t really work out what it’s supposed to be), Kilo (good but seemed to crap out on tasks from Linear), Cyrus hosted in a VM on my home server (really good but very token-heavy and with hindsight a bit inflexible), Capy (couldn't make it work with the models I use).
v0.2 playing with Symphony
Then I stumbled over OpenAI Symphony, which is an open spec for a software factory. It’s meant to be built around Codex but being open that’s really not necessary.
While on a work trip to Korea I’d spent the flight over playing with the pi agent harness. I rebuilt my personal site using it while in the air, using Z.ai’s amazing GLM-5.2 model.
Once in Korea, I pointed the agent at the Symphony spec, with some changes to substitute in the cheap models I wanted to use. And I wanted it built in Go. Cos why not!
I left the laptop running while we went out for team activities, came back, made some tweaks and had a working software factory running on my laptop, driven by Linear. Amazing!
Speed isn’t that important and local models need to be running 24x7
While thinking this all through I had a realisation: with a software factory like this, speed isn’t all that important. You assign something and can have faith it’ll build and deliver an object that passes your tests and reviews. If your tests and reviews enforce what you need, you can just assign the task and go to bed.
So now local LLMs get interesting. Let’s say you decided to buy a $10k Mac Studio to run local LLMs. You really want that investment working 24x7 to maximise what you’re getting from it. Same if you were to buy any of the approaches to running local LLMs. I feel like this is going to be a trend as the local models get Good Enough™️.
Marry those two things together: a software factory that just crunches through your backlog, plus a mixture of expensive frontier models doing planning and coordination of cheaper models (potentially local, though OpenRouter hosted cheap models are pretty damn good and cheap enough to avoid dropping $10k on a space heater) doing most of the work. Again, speed isn’t so important if it’s crunching through things 24x7.
This approach is also interesting from a cost perspective. If you’re a cloud inference provider, bursty traffic is hard to plan for. People sitting at their desks in work hours and you run out of capacity, then they go home and you’ve got idle hardware sitting around not making you money. Z.ai already have a Peak Hours pricing approach. Provisioning a consistent throughput could be an interesting pricing option designed for software factories and aligns with the inference provider's interests.
v0.3 Symphony for the Devil
So my latest approach is an iteration on the OpenAI Symphony approach. It's called Symphony for the Devil because that's the first thing that came to mind with the name "Symphony".
Workflow
- You assign a Linear task to the agent ("devil").
- The coordinator spins up a VM that contains all the tools you need for the task, customised to that task.
- The VM runs a pi agent harness and pulls the repo out, works on it
- Agent pushes in a PR
- Agent waits for the tests and reviews to run, then works on those until all resolved
- (optionally) merges the PR.
- The VM is then put into sleep state.
- After 7 days the VM's disk is deleted.
NixOS VMs
I'm using nixOS for the VMs. That's a Linux distro with composable config. My son has been using it for ages and it feels like a great fit for this project. You write a configuration and can build a new machine (he uses it for his desktop) from the config with no interaction and it ends up just as you want it.
For VMs, I've got a base VM image with the core coding harness in it and access to the LLM keys. Then for different tasks I add to the base. A web version with nodeJS and Playright browser so it can test it. An app version with Android and iOS libraries to build that stuff (I do those builds in Github Actions cos you need a Mac to build iOS). Thinking maybe a test version with the appropriate bits for that.
Hosting
This all runs in a VM on my home server in 16 gigs RAM, so I can host 3 or so concurrent tasks. For some reason I can’t afford to upgrade the RAM in my server to get headroom. I wonder why? But there’s no reason this couldn’t be in a cloud machine or spin up cloud VMs for much more throughput.
Future optimisations
There’s a lot of tweaking to do. I’ve got a bit of a roadmap forming.
- Coding plan maximisation: get the pacing just right to use as much of the coding plans as possible (I currently have Z.ai and Kimi but you could just as easily use OpenAI or other providers)
- Cheapest model optimisation: work out which models are the cheapest for the subagents doing most of the work. Prompt compaction looks like the signal that the model isn’t up to the task. Break the task down into smaller tasks or use a chunkier model?
- Testing mechanisms: bigger VMs specifically for running automated tests, maybe for builds too?
- Devil Dashboard: what tasks are being worked on, tokens per hour, cost per hour, VMs running, RAM and disk consumption