Skip to content

serve: accept modified filepaths via POST to scope cold hashing#424

Merged
tinder-maxwellelliott merged 1 commit into
masterfrom
claude/modified-filepaths-serve-8b8a2a
Jul 11, 2026
Merged

serve: accept modified filepaths via POST to scope cold hashing#424
tinder-maxwellelliott merged 1 commit into
masterfrom
claude/modified-filepaths-serve-8b8a2a

Conversation

@tinder-maxwellelliott

Copy link
Copy Markdown
Collaborator

What

Adds per-request modified filepaths support to the bazel-diff serve query service — the same content-scoping optimization as generate-hashes --modified-filepaths, exposed over HTTP.

When a request carries modifiedFilepaths, the server reads and hashes the content of only those files at both from and to, treating every other source file as unchanged. On a large monorepo that turns an O(all source files) content read into O(changed files) — the expensive part of a cold hash. A CI client already has the list cheaply from git diff --name-only from...to.

API

A changed-file list is too large for a URL, so it rides in a POST body:

curl -X POST http://localhost:8080/impacted_targets \
  -H 'Content-Type: application/json' \
  -d '{"from":"main","to":"my-feature-branch","modifiedFilepaths":["foo/BUILD.bazel","foo/bar.py"]}'
  • POST /impacted_targets and POST /impacted_targets_with_distances accept {"from","to","targetType"?:[…],"modifiedFilepaths"?:[…]}.
  • GET is unchanged and always the full-content hash. An absent/empty modifiedFilepaths on POST behaves identically to GET. Other verbs still 405.

Correctness & caching (the tricky part)

  • The same set must be applied to both revisions: an unchanged, unlisted file is content-skipped on both sides and so hashes identically; a listed file is read on both. The set must be a superset of what actually changed or impacted targets are missed — the same caveat that makes the CLI flag experimental.
  • A scoped hash of a SHA is only comparable to another SHA scoped with the same set, so scoped cache entries are keyed <sha>.<fingerprint>.<digest-of-set> — never mixed with, or served in place of, the full <sha>.<fingerprint> entry. The digest is order-independent (sorted) and hex, so the <key>.json on-disk mapping stays filename-safe.
  • The empty-set path keeps today's cache key, so GET and warmup keep sharing existing entries. Trade-off: on the scoped path the shared-base full-hash cache isn't reused (each distinct changed-set re-hashes the base), but each such hash is cheaper. Existing LRU --cacheMax* pruning bounds the extra entries.

No new CLI flag — this is per-request, and it is not part of the config fingerprint.

Changed

  • HashService — scoped cacheKey + thread the set into hashAllBazelTargetsAndSourcefiles
  • ImpactedTargetsService — pass the same set to both revisions
  • BazelDiffServer — GET+POST handling, JSON body parsing/validation
  • Unit tests across the four serve suites
  • Docs: tools/readme_template.md + README.md (endpoint + a caching-implications note)
  • tools/serve_harness.py — POST-body support in the http() helper, retargeted the old POST → 405 assertion to PUT, and a new scoped case

Testing

  • Unit: HashServiceTest, ImpactedTargetsServiceTest, BazelDiffServerTest, ServeCommandTest (+ storage/metrics) all green; ktfmt clean.
  • End-to-end (real Bazel + real git), via tools/serve_harness.py:
    • --only scoped → 3/3: scoping to the changed file matched the full GET set (//:core, //:core.txt, //:mid); a negative control that omits the changed file correctly misses it, proving content-skipping takes effect.
    • --only full/subprocess → 28/28, including the new PUT → 405, POST body parity with GET, and malformed POST body → 400.

🤖 Generated with Claude Code

Threads the experimental `generate-hashes --modified-filepaths` content-
scoping optimization into the `serve` query service. A CI client already
knows the changed set cheaply (`git diff --name-only from...to`); passing
it lets the server read+hash the content of only those files at both
revisions and treat every other source file as unchanged, turning an
O(all source files) content read into O(changed files) — the expensive
part of a cold hash on a large monorepo.

Because a changed-file list is too large for a URL, it rides in a POST
body: `POST /impacted_targets` (and `/impacted_targets_with_distances`)
with `{"from","to","targetType"?,"modifiedFilepaths"?}`. GET is unchanged
and always the full-content hash; an absent/empty `modifiedFilepaths`
behaves identically to GET.

Correctness: the same set must be applied to both revisions (an unchanged,
unlisted file is content-skipped on both sides and so hashes identically),
and it must be a superset of what actually changed or impacted targets are
missed — the same caveat as the CLI flag. A scoped hash of a SHA is only
comparable to another SHA scoped with the same set, so scoped cache entries
are keyed `<sha>.<fingerprint>.<digest-of-set>`, never mixed with or served
in place of the full `<sha>.<fingerprint>` entry. The empty-set path keeps
today's key so GET/warmup keep sharing entries; existing LRU `--cacheMax*`
pruning bounds the extra entries.

Covered by unit tests (scoped cache key, both-revision threading, POST
parsing/validation) and the serve integration harness — a new `scoped`
case proves scoped-POST == full-GET parity plus a negative control where
an incomplete set correctly misses the change.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@tinder-maxwellelliott tinder-maxwellelliott force-pushed the claude/modified-filepaths-serve-8b8a2a branch from be5093a to a8bf2f2 Compare July 10, 2026 21:00
@tinder-maxwellelliott tinder-maxwellelliott marked this pull request as ready for review July 10, 2026 23:58
@tinder-maxwellelliott tinder-maxwellelliott merged commit 16670fc into master Jul 11, 2026
15 checks passed
@tinder-maxwellelliott tinder-maxwellelliott deleted the claude/modified-filepaths-serve-8b8a2a branch July 11, 2026 00:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant