Cron'd Claude Agents: A Maintenance Log

Cron'd Claude Agents: A Maintenance Log

Running a fleet of twenty Claude agents on a launchd schedule teaches you fast that scheduler guarantees are weaker than you expect. Each agent wakes, processes a slice of work, writes its results, and goes back to sleep until the next tick. The pattern is simple; the failure modes are not. Three months of nightly runs have turned up launchd timing quirks, session-end failures, and the quiet pressure of growing turn counts. This post is a candid maintenance log: what broke, why it broke, and what I changed. The scheduler is not a magic timer When I first set up the cron-style jobs, I used launchd because it integrates cleanly with macOS and gives fine-grained control over start times, resource limits, and restart policies. The first surprise was that launchd does not guarantee exact start times. If the system is busy, a job may be delayed by several seconds, and those seconds add up over a day. For a single agent the drift is negligible, but with a fleet of twenty the cumulative delay can push the final run past the intended window, causing overlapping executions. The overlapping runs manifested as two agents trying to write to the same SQLite file at the same time. SQLite locks the file for writes, so the second agent stalled until the lock cleared. In a tight schedule that meant a cascade of timeouts, and eventually the launchd daemon marked the job as failed. The fix was to assign each agent a distinct, fixed start minute in its launchd plist. Rather than allowing multiple agents to share the same start slot, hand-spacing the schedule across the day gave each agent a clear window with no overlap and eliminated the write-lock cascade. Another hidden quirk is launchd's handling of environment variables. The agents rely on a PATH that includes the Python interpreter and a few helper scripts. When launchd launches a job, it inherits a minimal environment that does not include the user's shell profile. The first few runs failed with "command not found" errors because the interpreter could not be located. The solution was to define the full PATH inside the launchd plist and to reference the interpreter with an absolute path. This made the jobs independent of any interactive shell configuration. Turn budgets are counts, not tokens Claude agents in this fleet run under per-ticket turn-count ceilings, not token budgets. The ceilings live in a central configuration file and vary by the type of work: a build ticket gets a higher ceiling than a review. The numbers were calibrated against historical run data at roughly twice the measured mean, so legitimate work should rarely approach the limit. The monitoring piece came first: a turn monitor runs alongside each agent session and fires warnings at the ceiling. What came later was enforcement. The practical lesson is that instrumentation has to precede enforcement. Without knowing which ticket types run long, any ceiling you set will cut legitimate work short on some types and leave the door open on others. Instrument first. Calibrate against real data. Enforce later. Session-end handling and data ownership Claude agents automatically save a session summary at the end of each run. The most common failure mode was a sudden termination of the Python process due to an unhandled exception. When the process died, the SQLite transaction was left open, and the next run attempted to write to a locked database. The lock persisted until the operating system reclaimed the file handle, which could take minutes. During that window the entire fleet stalled. Ensuring that every database connection is explicitly closed on exit -- whether the session ends cleanly or not -- is the fix. A connection left open by a crashed process holds the write lock until the OS reclaims the file descriptor. Adding explicit close calls in the error path, rather than relying on garbage collection, keeps the lock window short and the next scheduled run clean. I wrote earlier about the cost of context loss between agent sessions -- that post focuses on the token waste. Here the problem is the structural consequence: lost writes corrupt downstream data. Another subtle issue was the handling of open questions. The agents try to capture any unanswered items that arise during a run. The capture is best-effort: if the session does not contain enough signal, the question is not recorded. Early on I assumed every open question would be saved and built downstream alerts on missing rows. When the capture failed silently, the alerts generated noise and eroded trust in the monitoring system. Treat the open-question log as a helpful hint rather than a strict contract, and design downstream processes to tolerate missing entries. What three months taught me Running a fleet of Claude agents on a schedule is not a set-and-forget exercise. Schedule reliability matters more than raw speed. Turn budgets need instrumentation before enforcement. Robust session handling prevents cascading failures that can bring the whole fleet to a halt. If you are a data engineer or AI practitioner building an autonomous agent fleet, these are your maintenance checklist items. Start with a well-defined schedule, monitor turn counts from day one, and make your data persistence resilient to crashes. If you are considering building an agentic data pipeline from scratch, the LangGraph implementation guide covers the architectural decisions that precede the operational ones covered here. This post is the first in an ongoing maintenance log. Read the full post on the Labyrinth Analytics blog, or reach out if you already have a fleet running and want a second pair of eyes on the design. PS -- Get posts like this weekly: subscribe to Dispatches from the Labyrinth

Original Source

Read the full article at Dev →

KhanList aggregates and links to publicly available news content. We do not host full articles from third-party sources. Always verify important information with original sources.