Skip to content

Interleave rollout requests across groups - #81

Draft
nightlessbaron wants to merge 1 commit into
prodfrom
agent/stratified-rollout-dispatch
Draft

Interleave rollout requests across groups#81
nightlessbaron wants to merge 1 commit into
prodfrom
agent/stratified-rollout-dispatch

Conversation

@nightlessbaron

Copy link
Copy Markdown

What

  • Dispatch each candidate batch in a seeded, stratified order: shuffle groups and member positions, then submit one member per group per pass.
  • Apply --rolling-start-size to exact sample-level waves instead of truncating the batch to whole groups.
  • Reassemble each group in its original member order before group RM, filtering, tail cancellation, and partial-rollout handling.
  • Preserve deterministic sampling seeds by original within-group position.

Why

The existing rolling start submits complete prompt groups together. With 256 groups × 16 samples and --rolling-start-size 256, that clusters all 16 siblings from a small set of groups into the same burst. This change produces 16 waves of 256 requests, with one sample from every group in each wave and 15 intervals between waves.

Impact

Only the inference rollout scheduler changes. No arguments, configuration, or test files are added.

Validation

  • 256 × 16 behavioral check: 4,096 unique samples, 16 waves, one sample per group per wave, deterministic ordering.
  • Existing agentic rollout tests: 9 passed together; the socket-sensitive concurrency case passed separately.
  • Ruff, Python byte-compilation, and git diff --check passed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant