feat(idea-spark): quality gauntlet upgrades + run-state navigator + SKILL.md slimdown#12
Open
fistyee wants to merge 1 commit into
Open
feat(idea-spark): quality gauntlet upgrades + run-state navigator + SKILL.md slimdown#12fistyee wants to merge 1 commit into
fistyee wants to merge 1 commit into
Conversation
…KILL.md slimdown Effectiveness: - Dual-channel collision (3.1): signature_terms @10mo (was 6mo) for contemporaneous scoops + NEW alias_terms @48mo — Phase 2.2 emits other communities' names for the same mechanism from parametric knowledge, closing the lexical blind spot windows can't (renamed ancestors, e.g. "goal-conditioned success detector" vs "goal-image conditioned scorer"). Hits carry a collision_channel tag; audit told not to age-discount alias threats. Backstop: audit's parametric_family_concern soft signal (family name + query vocabulary, never paper cites) lifts into Phase 4 reviewer_concerns as a "scoop-check X first" flag. - 5th audit check falsification_structure_check + the single audited kill-switch exception: scope=falsification revision targets apply via a dedicated rewrite_falsification op — merger verifies audit authorization (--critique), stamps falsification_rewritten, requires a bounded single-check re-audit; kill_switch_integrity re-bases the byte-identity anchor at 3.3 on that path; compute_budget stays sealed. - Bounded internal retry before abandon: archive attempt_1/, re-run Phase 2.1+2.2 once with the failed audit as negative constraints (one-shot guarantee bars asking the user, not internal regeneration). Efficiency: - NEW `next` subcommand (scripts/next_step.py): read-only run-state navigator that prints exactly the next step (bash command or sub-agent spec), encoding the full phase graph incl. gates and retry/re-audit branches; host loop becomes next -> do -> next. - SKILL.md 69KB -> 29KB: design rationale moved to references/ design-notes.md, install steps to references/setup.md; operational runbook only. - Default Phase 0 flow passes --queries up front (skips the rc=10 intent sentinel round-trip); Phase 2.1+2.2 run in ONE isolated context (both generation-side; adversarial pairs stay separate); pattern tagging delegated to a cheap/fast model; timeout guidance (>=600s) documented. Usability: - NEW add_user_ref subcommand: deterministic dedup-merge of title/ID user refs into user_refs.json — avoids the host Write-tool read-before-overwrite failure observed in a live run. Tested: 43 checks across merger authorization paths, kill-switch validator chains (advance/revise/rewrite/tamper), dual-channel stub retrieval, next state-machine walkthrough on a real run dir + synthetic branches, add_user_ref edge cases; validators regression-pass on a pre-change run. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
@fistyee please read the following Contributor License Agreement(CLA). If you agree with the CLA, please reply with the following information.
Contributor License AgreementContribution License AgreementThis Contribution License Agreement (“Agreement”) is agreed to by the party signing below (“You”),
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
One squashed commit upgrading the idea-spark skill on three fronts, validated on real runs.
Effectiveness
signature_terms[]@ 10-month window (was 6). NEW alias channel:alias_terms[]— other communities' names for the same mechanism, produced from parametric knowledge at Phase 2.2 — @ 48-month window. This closes a lexical blind spot window-widening can't: a real run's "goal-image conditioned scorer" had 2023-24 near-ancestors published as "goal-conditioned success detectors", invisible to both the topical Phase 0 queries and the candidate-vocabulary signature terms. Hits carry acollision_channeltag; the audit is instructed not to age-discount alias-channel threats. Backstop: a newparametric_family_concernsoft signal (family name + query vocabulary, never paper cites) flows into Phase 4 reviewer_concerns as a "run a scoop-check on X first" flag.falsification_structure_checkaudits the falsification paragraph's structure (minimal experiment / outcome metric + direction / ONE load-bearing variable / non-tautological negative control) — previously demanded at generation but never audited, and unrepairable due to the kill-switch. Now repairable through one gated door: ascope=falsificationrevision target applies via a dedicatedrewrite_falsificationop; the merger verifies audit authorization (--critique), stampsfalsification_rewritten, and a bounded single-check re-audit must pass before Phase 4.kill_switch_integrityre-bases the byte-identity anchor at Phase 3.3 on that path;compute_budgetstays sealed under every op and scope.attempt_1/, marks.retry_used, and regenerates Phase 2.1+2.2 once with the failed audit + selection as negative constraints. The one-shot guarantee bars asking the user, not internal regeneration.Efficiency
nextsubcommand (scripts/next_step.py): a read-only run-state navigator that inspects the run dir and prints exactly one next step (a verbatim bash command, or an LLM sub-agent spec with prompt/inputs/output paths). It encodes the full phase graph — fulltext gate, citation gate (run inline), abandon-retry and falsification re-audit branches, per-path Phase 4 flags — so the host loop degenerates tonext→ do it →next.references/design-notes.md, installation toreferences/setup.md; SKILL.md is now the operational runbook only.--queriesup front (skips the rc=10 intent sentinel); Phase 2.1+2.2 run in ONE isolated context writing both output files (both generation-side; the adversarial pairs 3.2/3.3 and 4.fill/4.1.5 stay separate); per-paper pattern tagging is delegated to a cheap/fast model; ≥600s timeout guidance documented for the retrieval steps.Usability
add_user_refsubcommand: deterministic dedup-merge of title/ID user references intouser_refs.json, avoiding a host Write-tool read-before-overwrite failure observed in a live run.Testing
43 checks: merger authorization paths (unauthorized/authorized/double-rewrite/compute_budget refusal), kill-switch validator chains (advance / revise / audited-rewrite / tamper detection), stub dual-channel retrieval (windows, per-channel queries, cross-channel dedup + tagging),
nextstate-machine walkthrough on a real completed run plus synthetic retry/re-audit/terminal branches,add_user_refedge cases; all 5 validators regression-pass on a pre-change run.🤖 Generated with Claude Code