What breaks when the loop works alone (part 2)

An agent loop that works on its own for hours doesn’t fail like software with a human in front of it: it fails silently, and silence looks a lot like success. In the first part of this series I laid out the architecture of an agent box — outbound daemon, your own worktree engine, zero open ports — and promised to tell the other half. This is it: what broke with the box on for weeks on end and agents working inside around the clock.

That an unsupervised loop can deliver big work is no longer the question: Bun’s rewrite from Zig to Rust shipped in 11 days of agentic work against the 1-2 years estimated by hand. The question is the other one: when nobody’s watching, what degrades first? On my box, the answer wasn’t the model or the code it writes. It was four more boring things: a credential, an error channel, two defaults, and my own way of fixing machines. All four reproduced and diagnosed; none of them announced itself when it broke.

Two legitimate processes are enough to corrupt a credential

The worst incident so far announced itself as a restart every 15 seconds in a process that stays on 24 hours a day: the bridge — the server daemon that holds the outbound connection to Anthropic and sustains the sessions — was stuck in a start-and-die loop with a single message, You must be logged in. No attacker, no expired token. The cause: the bridge authenticates with its owner’s seat credential — the subscription seat — and a nightly cron was using that same credential for its batch process.

The mechanism is a race over a file. Both clients refresh the OAuth token when it expires: the batch job rotates it first, the bridge — which keeps its copy in memory — writes the old version back on top, and the credential on disk ends up corrupted for both. It’s the kind of failure that only shows up with time: everything works, for days, until the refresh clocks cross. And the one who suffers isn’t the cron that caused it, but the session you try to open from your phone the next day.

In practice: one credential per process, no exceptions. The bridge with its owner’s login; each cron with its own token from claude setup-token in its .env with 600 permissions. Audit yours with this test: if you can’t revoke one process without taking down another, two things are sharing an identity they shouldn’t share.

Your phone doesn’t show you the errors of the system that sustains it

Since Claude Code shipped remote mode — sessions run on your own server and you steer them from the phone app — in February 2026, steering agents remotely has a structural consequence: the human channel shows the session — messages, diffs, permissions — but not the process that sustains it. 4 configuration flags separate a working bridge from one that hangs silently, and none of the four produces an error visible from the app. If the bridge dies, you don’t see an error; you see that your server has stopped appearing in the environment list, which is exactly the same symptom as “it hasn’t started yet.”

Two measured examples. One: without starting the daemon with -v --debug-file, its errors go nowhere — the process lives under systemd, with no terminal, and whoever steers from the phone will never see them. Two: claude auth login regenerates the configuration file and deletes the flags that allow running without an interface (remoteEnabled, hasCompletedOnboarding and friends); the symptom shows up later, as a Workspace not trusted or a silent hang after acquiring the workspace lock. Modal’s CTO put it for code in his agent experience thesis: when the agent is the one writing, observability matters more than reading the code. Running the box teaches you the corollary: it also applies to the infrastructure that sustains the agent.

In practice: configure logging to a file from the very first start, and keep two health commands at hand for when you administer the box — one grep of the connection registration to know the bridge is alive, one grep of errors to see what the phone never shows. After every re-login, re-seed the flags with a merge over the configuration, not an overwrite. And check health in pull mode — you look it up yourself when you log in, at a glance — not with alerts nobody reads.

Defaults operate when nobody’s watching

4 GB of swap — disk-backed memory that stands in for RAM — and one checkbox in a web console separate a healthy box from an unreachable one. The first default that bit me was memory: a modern frontend build plus a Chromium with no graphical interface trigger the OOM-killer — the kernel process that kills processes when RAM runs out — even on a 16 GB box, and the victim isn’t the build: it’s the other sessions working next to it. With 4 GB of swap the same build passes without drama. Swap isn’t optional on an agent box; it’s the difference between a slow build and all your sessions dead.

The second default is more treacherous because it takes half a year to fire: the mesh VPN’s device key expires on its own — in Tailscale, at ~180 days unless you disable it per node; other VPNs have their own version of the same timer. On a laptop that’s reasonable hygiene; on a server whose only way in is that private network — the box exposes no public SSH at all — expiry means being locked out of your own machine. The general pattern: defaults are designed for the majority case with a human present, and an agent box is the opposite case. Closing the firewall has its strict order too — bring up the VPN, verify from another terminal that you can get in through it, allow its interface in the firewall and only then enable it — because doing it in any other order locks you out with the same elegance.

In practice: add swap even if RAM looks plentiful, disable key expiry on every node that’s a server, and rehearse the lockout before you suffer it — check that the provider’s console gives you an emergency access that actually works. A default you didn’t decide is still a decision: someone else made it, and it executes when you’re not there.

Cattle, not pets: recovery as the default operation

Standing up a new box from a snapshot — with the date in its name: dev-system-2026-07-10 — costs about 20 minutes of admin; diagnosing a sick box has no upper bound. From that asymmetry comes the doctrine that makes the box sustainable: boxes are cattle, not pets. The system is built in two layers with different lifecycles — the system layer lives in an idempotent script in git and rarely changes; the project layer lives in each repo’s seed (clone, dependencies, database with demo data, smoke test) and changes daily. The snapshot is just a time cache: any stale box catches up by re-running the script, and the script in git is the source of truth.

The operating rule: if a fix requires surgery on state that holds data — upgrading a database with a live data directory, dependencies compiled against another runtime — you don’t operate: you recreate the box and the project gets rebuilt from its seed. The doctrine also pays a dividend you don’t see until you need it: retiring a box is deleting it and revoking its keys — nothing else to remember. No backups to babysit, no state to migrate: the work lives in git, the demo data regenerates from the seed, and the snapshot recreates the machine in minutes.

In practice: write your project’s seed — a script in the repo itself that takes it from zero to passing the smoke test — even if you don’t have a single box: it’s useful the first day someone gets a new laptop. And adopt the recreation criterion: the moment diagnosing a machine costs more than recreating it, stop diagnosing.

The architecture in part one was the easy half: you design it once and it stays put. This half doesn’t — credentials refresh, defaults expire, memory runs out — and no failure on the list throws an exception the loop can catch: they all degrade the system underneath the agent, which looks asleep instead of broken. The operational half isn’t about designing better; it’s about deciding what to measure when nobody’s watching.

How do you find out that your agent has been stalled for six hours: from a log you were already watching, or because the work you expected never shows up?