Skip to content

ADR-0005 — Execution-workspace lifecycle is a named domain language

ADR-0005 — Execution-workspace lifecycle is a named domain language

Section titled “ADR-0005 — Execution-workspace lifecycle is a named domain language”

Status: Accepted (2026-07-07)

A project execution workspace is the project-owned lease for one shared Daytona sandbox. Agent profiles run isolated provider processes inside it; profiles never own or fork sandboxes. Its workspace_status lifecycle (pending → materializing → ready/active/idle_warm → suspended/failed) was never modeled — every consumer re-encoded the predicates it needed as inline set-literals:

  • the registry’s deferred-ensure path ({"ready","active","idle_warm"}),
  • the warm-pool candidate scan (same set, second copy),
  • the dispatch loop’s warmup gate ({"ready","active","idle_warm","connected"} — a different set, silently),
  • the convergence sweep’s stuck-row policy (materializing × provider-binding × deferred-flag combinations),
  • the repo’s stuck-row fetch (its own prose re-statement of the above).

Two production livelocks hid inside those unnamed conditionals and cost days:

  1. Convergence starvation (#2792): the stuck-row fetch returned oldest-first; sixteen terminal-litter rows (dead failed/suspended devpod workspaces) saturated the limit=10 window, so convergeable materializing rows were never fetched. The convergence predicate was correct — it just never saw the rows.
  2. Ready-lease demotion seesaw (#2796): every execute pass re-ran the deferred ensure, which unconditionally re-marked an already-ready workspace materializing. The convergence lane repaired; the next dispatch tick demoted; sessions parked at waiting_execution_workspace forever while both sides “worked correctly.”

Both fixes were one-line predicates. Neither could be SEEN because the lifecycle had no single place to look, and both would have been caught by a reviewer reading preserves_ready_lease(...) next to mark_workspace_materializing(...).

  1. One domain module owns the vocabulary: contexts/access/domain/integration/execution_workspace_lifecycle.py — pure functions, no I/O. The ubiquitous names:
    • SERVING_WORKSPACE_STATUSES / DISPATCHABLE_WORKSPACE_STATUSES — the two historically different ready-sets, now named so the difference is a fact instead of a trap.
    • preserves_ready_lease(status, provider_workspace_id) — the demotion guard.
    • is_deferred_materializing(status, metadata) — deferred-provisioning rows, convergeable with or without a provider binding.
    • is_silent_materializing_hang(status, provider_workspace_id).
    • is_convergeable_without_live_run(...) — the union the convergence sweep uses.
  2. Application code imports the names; inline status set-literals in the registry and the dispatch loop are replaced, behavior-preserving (the full registry suite, 118 tests including both livelock regressions, pins this).
  3. CQRS rule for the convergence lane: the sweep is a command; the stuck-row fetch is its query. Both MUST share these predicates so the query can never drift from what the command would converge (the starvation bug was exactly this drift, expressed as fetch ordering). Additionally, per-row convergence failures must never be silently swallowed — a skipped row needs a reason a diagnosis can read (converged/skipped/failed), not a bare continue.
  • New lifecycle predicates go INTO the domain module first, with a name and a test; application code never grows a new inline status set.
  • Known follow-ups (owners follow repository authority):
    • the registry’s “resumable bound-sandbox” set ({materializing,ready,active,idle_warm,suspended,failed}) is still inline — name it when next touched (backend registry owner).
    • terminal-litter rows (failed/suspended, days old, no live run) still accumulate and only lost their sweep-starving power via fetch ordering; a litter reaper needs a backend owner (#2792 follow-up).
    • the metering endpoint is built in TWO places (coding_agent_auth_env — meter-aware — and agent_provider_profiles — raw); consolidate behind one foundation helper (backend runtime configuration owner; see runtime-config-debt #11).
  • Unify the two ready-sets into one: tempting, but connected in the dispatch gate is load-bearing (transient daytona-probe status); narrowing or widening either set changes live behavior. Naming both preserves behavior and makes the difference reviewable.
  • A full state-machine class over workspace_status: heavier than the problem; the failures were unnamed predicates, not missing transitions. Pure functions keep the registry diff minimal and codex’s convergence lane unblocked.
  • Leave the fixes inline with comments: comments were already there (both livelocks had adjacent prose) and did not prevent the bugs — names + a single module + colocated tests do.