From 9188d53ac02202623ec5bef17e4fe705e2833f0c Mon Sep 17 00:00:00 2001 From: Codex Date: Wed, 22 Jul 2026 10:53:08 +0000 Subject: [PATCH] feat(spot): default spotInterruption on; notify shell sessions; document config dir MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The EC2 Spot interruption handler (issue #20) shipped behind an mkEnableOption, so it defaulted off. A box could then run on Spot with the feature present in the module but no monitor unit instantiated — exactly what bit a live box today: an interruption stopped it on ~2 min notice with no agent prompted to persist context, losing an in-progress session. - modules/agent-box.nix: make spotInterruption.enable a bool option defaulting to true (was mkEnableOption, forced-off). The monitor self-gates on instance-life-cycle and exits at once on non-Spot/dev boxes, so on-by-default is safe. Already-deployed boxes whose configuration.nix doesn't set the flag inherit the new default on the next agent-box-update.service run — no config edit needed. - aws/template.yaml: set spotInterruption.enable = ${UseSpot} so new installs tie it to the CFN Spot parameter; on-demand boxes omit the monitor unit outright. - Notify shell sessions too, instead of skipping them: a shell executes typed text as a command, so the notice is prefixed with '#' to land as a harmless comment the operator still sees in scrollback (no Escape — nothing to interrupt, and Escape+'#' hits readline meta bindings). - Document the config location in the embedded AGENTS.md: name ~/.config/agent-box/ as the config dir and note that its `env` file ignores blank/`#`-comment lines, so it can be annotated. Magic filenames aren't discoverable; a commented, documented config is. Co-Authored-By: Claude Opus 4.8 (1M context) Claude-Session: https://claude.ai/code/session_01JkQjyhaueHL7GbnA6obZNQ --- aws/template.yaml | 12 ++++-- modules/agent-box.nix | 91 +++++++++++++++++++++++++++---------------- 2 files changed, 65 insertions(+), 38 deletions(-) diff --git a/aws/template.yaml b/aws/template.yaml index 6111456..8a755c0 100644 --- a/aws/template.yaml +++ b/aws/template.yaml @@ -660,10 +660,14 @@ Resources: }; # Handle Spot interruptions gracefully (issue #20): on the # ~2 min IMDS notice, prompt live agent sessions to persist - # their context, then cleanly stop the agent units. Safe to - # leave on when UseSpot=false — the monitor reads - # instance-life-cycle and exits at once on a non-Spot box. - spotInterruption.enable = true; + # their context, then cleanly stop the agent units. Tied to + # the UseSpot parameter — 'true'/'false' splice verbatim as + # Nix booleans — so on-demand boxes omit the monitor unit + # outright. (The module already defaults this on and the + # monitor self-gates on instance-life-cycle, so leaving it + # enabled on a non-Spot box would also be harmless; wiring it + # to UseSpot just keeps the deployed config honest.) + spotInterruption.enable = ${UseSpot}; }; ${Nat64Config} diff --git a/modules/agent-box.nix b/modules/agent-box.nix index 52ada24..48ffee5 100644 --- a/modules/agent-box.nix +++ b/modules/agent-box.nix @@ -64,9 +64,12 @@ let - Install extra tools with nix, e.g. `nix profile add nixpkgs#awscli2` (no sudo needed; tools land in ~/.nix-profile/bin, already on PATH). - - Secrets go in ~/.config/agent-box/env (KEY=value, one per line) or the - settings page, and load on the next session (re)start, eg. GH_TOKEN is read - automatically, so `git clone https://github.com/...` just works. + - Your config lives in the directory ~/.config/agent-box/. Secrets and + environment variables go in the file `env` there (KEY=value, one per + line; blank lines and `#` comment lines are ignored, so annotate freely) + or via the settings page, and load on the next session (re)start — e.g. + GH_TOKEN is read automatically, so `git clone https://github.com/...` + just works. - Manage your own sessions without a rebuild: `agent-box-session ls|add|rm|restart`. `add` takes an optional name plus `--agent claude|codex|shell` and `--cwd DIR` — handy for fanning out @@ -1034,28 +1037,41 @@ in }; spotInterruption = { - enable = lib.mkEnableOption '' - graceful handling of EC2 Spot interruptions (issue #20). A small - root monitor polls the instance metadata service (IMDS) for a - scheduled interruption; when AWS posts one (~2 min notice) it - injects a "save your context now" prompt into every live agent - session (via tmux send-keys, so it works for any agent CLI — - claude, codex, …), waits spotInterruption.gracePeriod for the - agents to write continuity notes to disk, then cleanly stops the - per-user agent units so a cleanly-exited session is NOT respawned - during the shutdown window. - - It deliberately does NOT power the box off itself. This deployment - runs as a PERSISTENT Spot request with interruptionBehavior=stop, - and ONLY an AWS-initiated interruption-stop is auto-restarted when - capacity returns — a guest-initiated shutdown/stop counts as a - MANUAL stop, which AWS leaves stopped until someone starts it by - hand. So we let AWS perform the stop (its stop drives a graceful - systemd shutdown that flushes buffers anyway) and only prepare for - it. Harmless to leave enabled on a non-Spot (on-demand) box: the - monitor reads instance-life-cycle at startup and exits immediately - when it is not "spot" (or when IMDS is unreachable, e.g. a dev rig). - ''; + # Defaults to true (not mkEnableOption): the monitor is self-gating and + # harmless on a non-Spot box, so the safe default is "on" — a Spot box + # that forgets to set it must not silently lose agent context (issue #20, + # the failure that motivated this). Set false to omit the unit entirely; + # aws/template.yaml ties it to the UseSpot parameter for that reason. + enable = lib.mkOption { + type = lib.types.bool; + default = true; + example = false; + description = '' + Whether to enable graceful handling of EC2 Spot interruptions + (issue #20). A small root monitor polls the instance metadata + service (IMDS) for a scheduled interruption; when AWS posts one + (~2 min notice) it injects a "save your context now" prompt into + every live agent session (via tmux send-keys, so it works for any + agent CLI — claude, codex, …), waits spotInterruption.gracePeriod + for the agents to write continuity notes to disk, then cleanly + stops the per-user agent units so a cleanly-exited session is NOT + respawned during the shutdown window. + + It deliberately does NOT power the box off itself. This deployment + runs as a PERSISTENT Spot request with interruptionBehavior=stop, + and ONLY an AWS-initiated interruption-stop is auto-restarted when + capacity returns — a guest-initiated shutdown/stop counts as a + MANUAL stop, which AWS leaves stopped until someone starts it by + hand. So we let AWS perform the stop (its stop drives a graceful + systemd shutdown that flushes buffers anyway) and only prepare for + it. + + Defaults to true and is harmless to leave enabled on a non-Spot + (on-demand) box: the monitor reads instance-life-cycle at startup + and exits immediately when it is not "spot" (or when IMDS is + unreachable, e.g. a dev rig). + ''; + }; gracePeriod = lib.mkOption { type = lib.types.ints.unsigned; @@ -3902,21 +3918,28 @@ in sf="/home/$u/.config/agent-box/sessions.json" user_tmux "$u" list-sessions -F '#{session_name}' 2>/dev/null \ | while IFS= read -r s; do - # Skip "shell" pseudo-agent sessions: text typed into a - # shell would run as a command, not read as a note. agent=$($JQ -r --arg s "$s" \ '.sessions[$s].agent // ""' "$sf" 2>/dev/null) - [ "$agent" = shell ] && continue - # A single Escape interrupts any in-flight generation/tool - # call so the notice is handled NOW instead of queued behind - # a long operation that may outlast the ~2 min window. Only - # one — a double Escape opens claude's history picker. Then a - # short settle before typing: sent too fast, the TUI (still - # returning to its prompt) swallows the first characters. # Target "=NAME:" — an EXACT session match resolved to its # active pane. Bare "=NAME" is a session target that send-keys # rejects as a pane ("can't find pane"); plain "NAME" would # prefix-match (session "dev" could hit "devs"). + if [ "$agent" = shell ]; then + # A shell EXECUTES typed text as a command, so prefix '#' to + # make the notice a harmless no-op comment the operator still + # sees in scrollback. No Escape: there is no in-flight + # generation to interrupt in a shell, and Escape+'#' would + # trigger readline's meta bindings. + user_tmux "$u" send-keys -t "=$s:" -l -- "# $MSG" \ + && user_tmux "$u" send-keys -t "=$s:" Enter + continue + fi + # Agent session. A single Escape interrupts any in-flight + # generation/tool call so the notice is handled NOW instead of + # queued behind a long operation that may outlast the ~2 min + # window. Only one — a double Escape opens claude's history + # picker. Then a short settle before typing: sent too fast, the + # TUI (still returning to its prompt) swallows the first chars. user_tmux "$u" send-keys -t "=$s:" Escape sleep 1 # -l types MSG literally; a separate Enter submits it.