Skip to content

Auto-run the matching term-info query for count questions (v3.3.1 follow-up)#6

Merged
Robbie1977 merged 3 commits into
mainfrom
feature/answer-quality-recipes
Jul 15, 2026
Merged

Auto-run the matching term-info query for count questions (v3.3.1 follow-up)#6
Robbie1977 merged 3 commits into
mainfrom
feature/answer-quality-recipes

Conversation

@Robbie1977

Copy link
Copy Markdown
Contributor

Follow-up to the merged PR #5 (count: -1 digest fix). Brings the remaining fix on feature/answer-quality-recipes into main.

What & why

After the -1 fix, a count question like "how many images of neurons with a part in the medulla are available?" correctly saw the data but only told the user to run the query — the synthesiser is evidence-only, so it can't call a tool mid-answer. This auto-runs the matching query so the answer gives the number.

  • maybeInjectCountQueryStep (lib/orchestrator.mjs, called after the other maybeInject* steps): for a count question ("how many … of <term>") with a single resolved term, pickBestQueryForQuestion picks the term-info query whose label uniquely best-matches the question's distinctive words (ignoring the term's own name and stopwords like "part"), and injects a vfb_run_query step. runStep's extractAnswer turns the resulting count into evidence, so the synthesiser reports it.
  • Fires only on an unambiguous match (strict top score, e.g. ImagesNeurons beats NeuronsPartHere for "images of neurons"); skips if a connectivity/scRNA-seq/neuron-count/run_query step is already planned.
  • Tests: orchestrator suite 23/23 (added 3); next lint clean.

Known limitation / follow-up (not in this PR)

The chat still doesn't understand the individual-image queries vs class queries distinction, which caused a wrong answer (PartsOf = 28 subparts reported as "28 images", when ImagesNeurons = 226,524 is the real images count). Rows tell them apart: individual-image queries return VFB_* rows (count = images — ImagesNeurons, ListAllAvailableImages), class queries return FBbt_* rows whose thumbnails are just examples (count = number of classes — NeuronsPartHere, PartsOf, SubclassesOf). We only expose terse one-line labels, no count semantics. Proposed follow-up: a per-query_type semantics map (kind + count-noun + blurb) surfaced in the digest, plus using kind: individual-images to route "how many images …". Happy to do this next.

After merge

Tag v3.3.1 on main for the versioned image build.

Note: the large diff is the intentionally-tracked test-results/task-battery/ snapshots; the code delta is lib/orchestrator.mjs + its test (~112 lines). The latest.json merge was pre-resolved (kept the newer run) — this generated file conflicts on every feature → main PR and is worth a structural fix (stop tracking it or a merge rule).

Robbie1977 and others added 3 commits July 14, 2026 22:24
After the count -1 fix, "how many images of neurons with a part in the medulla
are available?" correctly saw the data but only told the user to run the query.
Auto-run it instead so the answer gives the number.

Add maybeInjectCountQueryStep: for a count question ("how many … of <term>")
with a single resolved term, pick the term-info query whose label best and
UNAMBIGUOUSLY matches the question's distinctive words (ignoring the term's own
name), and inject a vfb_run_query step for it. The step result (e.g.
ImagesNeurons count 226,524) becomes evidence, so the synthesiser reports the
count rather than deferring to the user. Skipped when a specialised
count/graph/expression step already covers it or the match is ambiguous.
@Robbie1977 Robbie1977 merged commit 76f9581 into main Jul 15, 2026
2 of 3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant