Hive Exit stability hardening

[Deadbytes101]

Purpose

Move RIGOS from a functional Alpha8 appliance toward failure-oriented mining-appliance stability. This PR does not claim HiveOS parity and does not bump the release version.

First hardening batch

• bound XMRig restart behavior with a ten-minute start-limit window and five-restart ceiling
• increase restart delay to avoid rapid crash loops
• define controlled stop/kill timeouts and resource ceilings
• add an observe-only miner health authority sampled every minute
• classify miner state as ready, warming_up, waiting_external, blocked, degraded, or failed
• distinguish pool/network outage markers from missing speed evidence
• record systemd restart count, PID state, runtime revision, share counters, and recent evidence without exposing mining identity
• never restart or kill the miner from the heuristic observer
• make rigosd and rigosctl default to /run/rigos/xmrig-public.json
• keep explicit --xmrig-config available for authorized diagnostics
• expose safe algorithm/thread/huge-page compatibility fields in the redacted public view
• add regression fixtures for restart bounds, outage classification, revision mismatch, redaction, and image wiring

Stability doctrine

• process death recovery is systemd policy
• pool/network outage is not a reason to reboot or kill a healthy miner
• heuristic observation remains read-only until physical evidence proves remediation thresholds
• runtime policy mismatch blocks mining truthfully
• private wallet, worker password, and API token never enter the public inspector view

Remaining blockers from issue #12

• persistent SSH host identity across reboot and power loss
• controlled miner-crash physical test
• network unplug/replug recovery test
• abrupt power-cut/state recovery test
• last-known-good activation rollback proof
• A/B update and rollback
• 24-hour single-node and seven-day three-node soak evidence

This PR must remain draft until authoritative verification and exact physical tests pass.

[Deadbytes101]

Authoritative source verification: PASS

Verified locally on exact head 53efcb826f35eef72ef14924c6201e65f7d68217.

Passed gates:

• cargo fmt --all -- --check
• targeted miner stability tests: 2/2
• observer classifications: ready, waiting_external, degraded, unknown, blocked
• bounded restart policy and observe-only contract
• redacted public inspector/runtime truth fixture
• full workspace unit/integration/doc tests
• schema generation check
• systemd ordering verification
• optimized release build
• final RIGOS verification passed

This is source verification only. PR remains draft. No image or physical stability claim is authorized until the crash, network outage, power-cut, rollback, and soak gates in issue #12 pass.

[Deadbytes101]

Hostile review blockers before image build

Authoritative source verification passed on 53efcb826f35eef72ef14924c6201e65f7d68217, but review found two release-blocking issues:

1. XmrigBackend::discover currently selects the running process --config path before explicit_config. The rigosd/rigosctl wrapper therefore does not yet guarantee that /run/rigos/xmrig-public.json wins over the identity-bearing runtime path.
2. The public runtime view is currently produced by copying the private configuration and removing known fields. This is not a stable security boundary because a future upstream secret field could be copied unintentionally.

Required fix contract:

• explicit inspector configuration takes precedence over process command-line discovery
• regression test proves the public path wins over a synthetic private process path
• public view is built from a strict allowlist
• unknown top-level, CPU, pool, and HTTP sentinel fields never enter the public JSON
• Rust formatting, rigos-xmrig tests, runtime redaction fixture, and git diff --check all pass before commit

A temporary exact-text applicator and guarded workflow are staged. PR remains draft and image construction remains blocked until these fixes are committed and full verification is rerun.

[Deadbytes101]

Hostile review implementation ready for local verification

GitHub Actions are disabled for this account (HTTP 422: Actions has been disabled for this user), so the temporary workflow/applicator path was removed rather than bypassed.

Current exact head: 9ee224f7373ff82cad560ac88eeceaff91478b8d

Implemented directly on the stability branch:

• private 0700 staging for the legacy runtime renderer
• jq allowlist construction of xmrig-public.json
• serialized publication through an executable rigos-runtime-authority using a 30-second flock bound
• stale staging/temp cleanup before and after publication
• status publication after private/public documents
• managed XMRig uses -c /run/rigos/xmrig.json, while inspector wrappers explicitly select the public view
• integration fixture proves unknown top/CPU/pool/HTTP sentinel fields remain private
• integration fixture proves explicit public inspection wins with the managed short config option
• source fixture checks shell syntax and executable authority mode
• appliance verifier now extracts and checks the runtime authority, allowlist publisher, miner drop-ins, health observer, wrappers, restart bounds, and exact XMRig command from squashfs bytes

Temporary workflow files are absent from the stability branch and the obsolete manual workflow was removed from main.

PR remains draft. No image build or physical stability claim is authorized until local targeted tests and the full ./scripts/verify.sh gate pass on this exact head.

[Deadbytes101]

Targeted Hive Exit verification: PASS

Verified locally on exact head ebbca7dfd01de82c9bf3aea2481be38a3aac7b52.

Passed targeted gates:

• Rust formatting
• explicit public inspector configuration boundary
• LF-only shell authority contract and shell parsing
• staged allowlist runtime publication
• runtime gate allowed valid exact-thread config
• runtime gate denied a one-thread profile mismatch
• image verifier coverage for runtime authority/stability bytes
• appliance wiring and recovery regression suite

Observed final marker:

HIVE_EXIT_TARGETED_VERIFICATION=PASS

This proves the new source contracts only. PR remains draft. Full ./scripts/verify.sh on the same head is still required before image construction.

[Deadbytes101]

Authoritative source verification: PASS

Exact head: 004eaebeef9b55297c38284aad7842fc0277e881

Local WSL verification completed successfully with a clean tracked tree.

Passed:

• workspace formatting
• clippy with warnings denied
• full workspace tests and doc-tests
• schema generation check
• locked release build
• systemd ordering verification
• firstboot/runtime/miner policy checks
• forbidden command-path scan
• Alpha8 SSH and recovery hotfix verification

Observed final markers:

• RIGOS systemd ordering verification passed
• RIGOS verification passed
• RIGOS Alpha8 SSH and recovery hotfix verification passed
• HIVE_EXIT_AUTHORITATIVE_VERIFICATION=PASS

This authorizes exact image construction from this head only. It does not authorize merge, release, or a HiveOS-parity claim. PR remains draft until image-byte verification and physical failure testing pass.

[Deadbytes101]

Exact image construction and byte verification: PASS

Source commit: 004eaebeef9b55297c38284aad7842fc0277e881

Built with rootful Podman after a successful privileged device-node preflight.

Exact artifact identity:

• image: rigos-usb-amd64-0.0.4-alpha.8.img
• SHA-256: ce87871e55b346e495b08d8f57411db99863aef843a82890850a0291975de746
• size: 2685403136 bytes
• manifest: rigos-usb-amd64-0.0.4-alpha.8.build-manifest.json
• builder image: 61d9acdc8292daa04191ec328237c90da472b2fc65d6ce8b4cf2a4724598b78e

Passed:

• rootful Podman check
• privileged mknod preflight
• locked release build
• pinned XMRig archive and binary SHA verification
• Debian live-build
• MBR image construction
• BIOS and removable UEFI GRUB installation
• systemd ordering verification from the built image
• USB appliance verifier
• recovery ISO verifier
• external SHA-256 verification
• manifest source/version/artifact/hash/size/layout verification

Final marker: HIVE_EXIT_EXACT_IMAGE_BUILD=PASS

This artifact is now authorized for controlled physical flashing and failure testing only. PR remains draft and unmerged. Persistent SSH host identity, miner crash, network outage, power-loss, rollback/A-B, and soak gates remain open.

[Deadbytes101]

Windows preflash gate: PASS

The final image was re-hashed and size-checked from the Windows filesystem immediately before flashing.

• image: rigos-usb-amd64-0.0.4-alpha.8.img
• SHA-256: ce87871e55b346e495b08d8f57411db99863aef843a82890850a0291975de746
• size: 2685403136 bytes
• result: PREFLASH_GATE=PASS

This confirms the file selected for flashing still matches the exact verified build artifact. Controlled physical flashing remains authorized for this exact hash only.

[Deadbytes101]

First physical boot on exact image: FAIL-CLOSED

Artifact under test:

• source 004eaebeef9b55297c38284aad7842fc0277e881
• image SHA-256 ce87871e55b346e495b08d8f57411db99863aef843a82890850a0291975de746

Confirmed on hardware:

• RIGOS 0.0.4-alpha.8 booted from /dev/sdb
• persistent state mounted from /dev/sdb4 and expanded to the USB capacity
• internal SATA disk remained unmounted
• SSH password access worked
• state and state-ready units were active
• SSH and miner-health timer were active

Blocking failures:

• rigos-firstboot.service failed
• rigos-runtime-render.service failed
• rigos-miner.service remained inactive
• /run/rigos/runtime-config-status.json was absent

This is a valid fail-closed result, not a passed physical baseline. Crash/network/power-loss testing is paused until runtime publication root cause is identified and fixed. PR remains draft.

[Deadbytes101]

Physical root cause confirmed: missing jq in appliance

Exact failed image:

• source 004eaebeef9b55297c38284aad7842fc0277e881
• SHA-256 ce87871e55b346e495b08d8f57411db99863aef843a82890850a0291975de746

Physical journal proved the runtime authority failed with exit 127:

/usr/lib/rigos/rigos-runtime-publish: 24: jq: not found

State, current revision, profile apply, shell syntax, script modes, and SSH were valid. The firstboot failure was a cascade from activation retry after runtime publication failed. Miner remained fail-closed.

Fix staged on head 732164021d450f5ab7986bfa3a1f3307a81b67e8:

• add jq to the live appliance package list
• make runtime publisher use absolute /usr/bin/jq
• emit an explicit exit-127 diagnostic before staging if jq is absent
• require executable usr/bin/jq inside the built squashfs verifier
• source regression locks the package/verifier contract
• behavioral regression proves missing jq fails before creating runtime staging/temp files

Previous image authorization is revoked for physical stability testing. New source verification and a new exact image are required. PR remains draft.

[Deadbytes101]

jq runtime fix authoritative verification: PASS

Exact head: 223794509e82d5dedda79da87a11ed2941c16c34

Verified locally from the Windows checkout through WSL with a clean tracked tree.

Passed:

• package manifest LF byte contract
• package manifest contains jq
• runtime publisher uses absolute /usr/bin/jq
• missing jq exits 127 before staging or temp-file creation
• staged allowlist publication remains green
• image verifier requires executable usr/bin/jq inside squashfs
• shell authority LF/syntax contract
• complete workspace tests and doc-tests
• schema check
• release build
• systemd ordering verification
• Alpha8 SSH/recovery verification

Observed final marker:

JQ_RUNTIME_FIX_AUTHORITATIVE_VERIFICATION=PASS

The old image SHA ce87871e55b346e495b08d8f57411db99863aef843a82890850a0291975de746 remains revoked for physical stability testing. A new exact image must be built and byte-verified from this head. PR remains draft.

[Deadbytes101]

jq-fixed exact image construction: PASS

Exact source head: 223794509e82d5dedda79da87a11ed2941c16c34

Built with rootful Podman after successful device-node preflight.

Exact artifact identity:

• image: rigos-usb-amd64-0.0.4-alpha.8.img
• SHA-256: 736a265eec2d27c1dc746fb9d5847e76be2e954f0d7d9a041bab1c7bf30963d8
• size: 2685403136 bytes
• manifest: rigos-usb-amd64-0.0.4-alpha.8.build-manifest.json
• builder image: 61d9acdc8292daa04191ec328237c90da472b2fc65d6ce8b4cf2a4724598b78e

Passed:

• rootful Podman check
• privileged mknod preflight
• release build
• pinned XMRig archive/binary SHA verification
• Debian live-build
• BIOS and removable UEFI installation
• systemd ordering verification
• USB appliance verifier, including embedded executable /usr/bin/jq
• recovery image verifier
• external image and ISO SHA verification
• manifest source/version/artifact/hash/size/layout checks

Final marker: JQ_FIXED_EXACT_IMAGE_BUILD=PASS

The previous image SHA ce87871e55b346e495b08d8f57411db99863aef843a82890850a0291975de746 remains revoked. This new image is authorized only for Windows preflash verification and a fresh controlled physical baseline. PR remains draft and unmerged.

[Deadbytes101]

jq-fixed Windows preflash gate: PASS

Final Windows-side verification immediately before flashing confirmed:

• image: rigos-usb-amd64-0.0.4-alpha.8.img
• SHA-256: 736a265eec2d27c1dc746fb9d5847e76be2e954f0d7d9a041bab1c7bf30963d8
• size: 2685403136 bytes
• result: JQ_FIXED_PREFLASH_GATE=PASS

This confirms the file selected for flashing matches the exact jq-fixed artifact built from source 223794509e82d5dedda79da87a11ed2941c16c34. Controlled physical flashing is authorized for this exact SHA only. PR remains draft and unmerged.