Writing
What actually broke in an agency run by agents
Three failures from a delivery stack that runs on AI. None of them were the model's fault, and all three reported success while producing nothing.
Writing
Three failures from a delivery stack that runs on AI. None of them were the model's fault, and all three reported success while producing nothing.
Notes
When people talk about AI going wrong they usually mean a model saying something false. In a company whose delivery actually runs on this stuff, that is the easy category. A wrong answer is visible. You read it, you catch it, you move on.
The expensive failures are the opposite shape. The system reports success, the dashboard is green, the job exits cleanly, and nothing happened. Three of those cost me real time this year. None of them were the model's fault.
Our content pipeline ends in a self-hosted scheduler that posts to eight channels. It was running on the workstation, a month of client content was queued, and the jobs moved through the states they were supposed to move through.
Twenty-nine posts scheduled. Zero delivered.
The cause was architectural and entirely mine. Uploaded media was being served on a machine-local address. Instagram and Facebook do not accept an image you hand them; they take the URL and fetch it themselves. A local address means nothing to a server in another country. LinkedIn pulls the file from inside the container, which could not reach it either. Every fetch failed on the far side of a boundary my logs did not cross, so the job state stayed clean.
Moving the scheduler to a real host with a public base URL fixed it. Then a second layer appeared underneath. The build running on the server was old enough that it shipped without its orchestrator process, so its queues were never registered with the workflow engine at all. Sixteen posts had been sitting in that queue since 11 May. No error, no retry, no alert. Just a queue nobody was reading.
A job status is the system's opinion of itself. Delivery is a fact about the world. Those are different data, and only one of them is worth counting.
The pipeline now counts published items by reading the returned public URL of each post, not the job state. It is a smaller number and it is true.
An asset factory runs Blender headless overnight: blender --background --python build.py, a few hundred procedural assets, a queue that checks each job before starting the next.
Measured on 8 September 2026: a script inside that command threw a Python traceback, failed to produce anything, and the process still returned exit code 0. The traceback was printed to standard output where nothing was reading it. The queue took the exit code at face value, marked the job complete, and carried on.
A night shift built on exit codes will happily report that two hundred assets were generated, and in the morning the folder is empty.
The fix is unglamorous. Every headless script now ends inside a try and except pair that prints the traceback and calls a hard exit with a non-zero code. Anything that looks up a node or a socket by name prints the names it can actually see and then dies, rather than quietly binding to the wrong one, which was the other half of the same evening.
The third one is the most uncomfortable, because it is the failure mode of the thing I am selling.
A sub-agent was launched to do a batch of file work. It ran, it reported DONE, and it had written nothing. Its tool calls needed approval that it could not request from where it was running, so each one was refused, and the refusals never became errors it recognised. It summarised its own inability as completion.
That is worth sitting with. An agent's report of its own work is generated text. It is not a log, it is not a receipt, and it carries no privileged access to what actually happened.
All three are the same failure with different clothes on. Something in the chain reported success without checking the world, and automation had already removed the person who would have noticed the discrepancy by accident.
The rule I work by now is short enough to apply while tired:
A task does not get automated until it has a machine-checkable success criterion, and the check reads the result, not the job. No check, no automation. It stays manual.
For the scheduler that means counting public URLs. For the render farm it means counting files on disk and their sizes. For the agent it means reading the folder it claimed to write to. In every case the check is boring, takes ten lines, and is the only part of the system I actually trust.
The scheduler problem cost roughly three weeks of a client calendar that looked delivered and was not. The renderer problem cost one night. The agent problem cost an afternoon and a small amount of confidence that I have not fully got back.
None of that is an argument against building this way. It is an argument that the interesting engineering in an AI company is not the prompting. It is the plumbing that decides whether you are allowed to believe your own dashboard.
More