Skip to content

Rung-1 GPU handoff: frontier bands + BC→R4 for RESTRAINT/RECOVERY (+ rung-0 honesty) #18

Description

@mabry1985

Handoff of the GPU-gated rung-1 work. Everything CPU-side is staged and on mythxengine-sdk PR #330 (feat/zoo-rung1-restraint-recovery); these steps need the GPU + the lab BC/R4 machine. The engine machine is world-agnostic (M1 proved it), so each world plugs into scenario_survive → zoo_traces → train → R4 with zero re-plumbing.

Queued work (priority order)

1. Finish the frontier bands — do this FIRST (cheap go/no-go per world).
The greedy floor is already built + swept (CPU, MYTHX_POLICY=greedy). Missing arms need the model:

  • Strong teacher: thinking-on Ornith-35B sweep (frontier top).
  • Weak arm: a served base Qwen3.5-2B sweep.
    Decisive question: does RECOVERY have any room above greedy? If greedy ≈ teacher, RECOVERY does not earn training spend (per-world gate) — skip it, save the GPU.

2. Train the worlds with headroom (M-R1 / task 9).
Teacher traces → zoo_traces --survive (SFT fold, already fixed for both new worlds) → masked-BC Qwen3.5-2B → NVFP4 → R4: claw ×3 + protoAgent/evals, BC vs base. RESTRAINT looks worth it; RECOVERY gated on step 1.

3. Close rung-0 honesty (task 1).
Re-run M1's BC arm ×3 (was single-trial vs base ×3) + the proper base-Qwen cold-R4 → BC → re-measure ×3 both arms — the still-owed falsifier.

Greedy-floor findings (the headroom read so far)

World greedy floor (seeds 0–7) median Ornith thinking-off (seed 1) read
RESTRAINT 16–32 24 16 (below floor) weak arm below a trivial floor → real trainable deficiency (can't time the regrow pulse). Healthy headroom — worth training.
RECOVERY 60–120 72 120 (≈/above floor) weak arm at/above floor → little deficiency to train. Yellow flag: possibly shallow (re-selection ≈ what greedy does). Confirm with the teacher sweep before spending.

The gate (please honor)

  • Kill-criterion (ENDSTATE.md): abandon the sim-as-RLVR-factory line if R4 hasn't moved > 2×pooled-SE (≈ +0.06–0.09 on claw, or a clear move on protoAgent/evals tool+abstention) on ≥1 instrument, attributable to sim training, by end of rung 2. Rung 0–1 flat is NOT falsifying.
  • Per-world spend gate: a world with greedy ≈ strong-teacher (no learnable band) does not earn BC/RL spend (the COURIER-v0 rule).

What's ready to run (pointers)

  • Engine: ~/dev/mythxengine-sdk (PR #330 branch; merge to main first if you prefer). Rebuild: cargo build -p mythx_host.
  • Runtime for the harness: ~/dev/mythxengine-sdk/.venv-mmo/bin/python (other venvs have a protobuf/gencode mismatch).
  • Worlds: builtin:story_restraint[:<seed>], builtin:story_recovery[:<seed>].
  • Harness: examples/scenario_survive.py — env: MYTHX_POLICY=greedy|llm, MYTHX_LLM_THINKING=on|off, MYTHX_WINDOWS, MYTHX_LLM_MODEL, MYTHX_LLM_BASE_URL.
  • Frontier sweep driver: examples/survive_frontier.py (preset-agnostic).
  • SFT fold: examples/zoo_traces.py --survive <frontier.jsonl> --model <id> → OpenAI chat rows w/ reward+orient+view stamp (both new worlds carry their own briefing now).
  • Datasets land under /mnt/data/datasets/agent-zoo/.
  • Lab BC/R4 machine: ~/dev/lab/experiments/mythxengine-bridge/ (the M1 pipeline: train_lora → merge → NVFP4 → claw ×3).

What to provision

  • Serve a base Qwen3.5-2B (and ideally 4B) for the weak arm + BC base + R4 base comparison. Keep Ornith-35B on :8000 as the teacher. If both GPUs are free, serve base-2B alongside Ornith to avoid swaps.

Reference

  • Roadmap: ~/dev/lab/experiments/mythxengine-bridge/ROADMAP.md (open forks resolved).
  • ENDSTATE + kill-criterion: ~/dev/lab/experiments/mythxengine-bridge/ENDSTATE.md.
  • World designs + greedy-floor detail: mythxengine-sdk/docs/plan/rung1-worlds.md.
  • M1 baseline: ~/dev/lab/experiments/mythxengine-bridge/MILESTONE1-RESULTS.md.

FOREMAN (rung-1 world #3) is intentionally held until this measurement is in.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions