Skip to content

fix(backfill): chunk mv_replay from the MV source so empty targets bootstrap (#187)#188

Merged
alvarogar4 merged 1 commit into
mainfrom
alvaro/num-7482-chkit-backfill-mv_replay-chunk-planning-fails-on-empty
Jul 8, 2026
Merged

fix(backfill): chunk mv_replay from the MV source so empty targets bootstrap (#187)#188
alvarogar4 merged 1 commit into
mainfrom
alvaro/num-7482-chkit-backfill-mv_replay-chunk-planning-fails-on-empty

Conversation

@alvarogar4

Copy link
Copy Markdown
Member

Summary

Fixes #187. A mv_replay backfill of a from-scratch empty aggregate target failed with No partitions found for <target>. The table may be empty. Chunk planning introspected the target's system.parts to size chunks and empty-checked it — but for mv_replay the target is legitimately empty (you're bootstrapping it), and the data actually read/sized is the view's source (its FROM table), not the target.

The fix detects the execution strategy before chunk planning and, for mv_replay with a single shared source, runs generateChunkPlan against that source table instead of the target:

  • chunking/sql.tsextractSourceTableRef() parses the first top-level FROM table out of a view's SELECT (handles db.table, bare table, backticks; rejects subqueries/table functions), reusing the existing quote/paren-aware scanner.
  • detect.tsresolveMvReplaySource() collapses the MVs feeding a target to a single shared source (unqualified FROM defaults to the view's database). Returns undefined — falling back to target introspection — when a source is unparseable or the views fan in from different sources, preserving the multi-source replay behaviour from fix(backfill): replay every MV feeding the target, not just the first (NUM-7480) #185.
  • planner.tsbuildBackfillPlan now detects the strategy first, chunks the resolved source for single-source mv_replay, and the empty-check guards the introspected table (an empty source still fails fast). The copy path is unchanged.

Behaviour preserved

Test plan

  • Unit: extractSourceTableRef (qualified/bare/backtick/alias/subquery/table-function/string-literal FROM), resolveMvReplaySource (single, shared, unqualified, differing sources, subquery).
  • Unit regression: an empty target with a single-source MV no longer throws (source introspected); an empty source still throws with the source name.
  • E2E (mv-replay-plan.e2e.test.ts, live ObsessionDB): populated source + empty SummingMergeTree target + MV → buildBackfillPlan plans one chunk per source partition, executeBackfill populates the target, per-bucket values match the forward MV output.
  • typecheck + lint pass; existing planner/detect/sql/copy-path e2e tests green.

Note: pre-existing chunking/e2e/smart-chunking.e2e.test.ts failures in this environment are unrelated — they depend on a separately-seeded chkit_e2e_chunking_* dataset (UNKNOWN_TABLE) and fail identically with these changes stashed.

🤖 Generated with Claude Code

https://claude.ai/code/session_013tzRKHhZETTZtQtDe999Fi

…otstrap (#187)

An mv_replay backfill of a from-scratch empty aggregate target failed with
"No partitions found for <target>". Chunk planning introspected the target's
system.parts to size chunks and empty-checked it — but for mv_replay the target
is legitimately empty (you're bootstrapping it), and the data actually
read/sized is the view's source (its FROM table).

Detect the strategy before chunk planning and, for a single shared MV source,
run generateChunkPlan against that source instead of the target. The empty-check
now guards the source; the copy path and multi-source fan-in (#185) are
unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013tzRKHhZETTZtQtDe999Fi
@alvarogar4 alvarogar4 merged commit f85f568 into main Jul 8, 2026
6 checks passed
@alvarogar4 alvarogar4 deleted the alvaro/num-7482-chkit-backfill-mv_replay-chunk-planning-fails-on-empty branch July 8, 2026 10:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug: backfill mv_replay chunk planning fails on empty target (should chunk the source)

1 participant