perf(lingbot): decouple condition prefetch from control ingress

[ActivePeter]

LingBot Condition Prefetch PR Description

Core design

/mnt/ceph/zyc/TeleFuser/docs/zh/design_lingbot_condition_prefetch.md

Proposed PR Title

[PERF] Decouple LingBot condition prefetch from control ingress

English

Description

Reduce LingBot-World-Fast control-to-output latency by moving condition encoding off the interactive control critical
path.

The runtime now prefetches a bounded number of condition chunks before control arrival and refills the window after
denoising. This preserves the original model computation, output ordering, and numerical path.

The PR also fixes two issues discovered by the example regression:

• asynchronous denoising did not enter torch.inference_mode();
• examples/run_examples.py did not explicitly close pipeline-owned workers.

Motivation

Condition encoding depends on the session image and chunk position, but not on the interactive control input.

Previously, each condition request was submitted together with its control, serializing:

1. control arrival;
2. condition encoding;
3. DiT denoising;
4. VAE decoding.

Prefetching conditions allows control processing to join an already prepared condition while retaining strict sequence
ordering and bounded memory usage.

Type of Change

• Bug fix
• Performance improvement
• Documentation update
• Breaking change
• New feature

Changes Made

• Initialize the LingBot runtime and streaming session before waiting for the first control.
• Prefetch at most two condition chunks per session.
• Track independent condition and control cursors.
• Reject duplicate, skipped, or out-of-order controls.
• Refill condition lookahead after denoising to overlap with current-chunk decoding.
• Add latency_anchor_artifact so control latency starts at control acceptance rather than condition prefetch.
• Run asynchronous denoising under torch.inference_mode().
• Explicitly close pipeline-owned workers in the example regression runner.
• Add scheduler, streaming, cleanup, ordering, and inference-mode regression tests.
• Document the final pipeline and the before/after Mermaid sequence diagrams.

No model weights, diffusion steps, dtype, attention implementation, VAE numerical path, service schema, environment
variable, or user-facing configuration is changed.

Testing

• LingBot-World-Fast unit tests
• CI-scoped unit and server tests
• GPU example regression
• Four-GPU AIPerf benchmark

Commands:

python -m ruff check \
  examples/run_examples.py \
  telefuser/pipelines/lingbot_world_fast/streaming.py \
  tests/unit/pipelines/lingbot_world_fast/test_streaming.py \
  tests/unit/test_run_examples.py

python -m pytest -q \
  tests/unit/pipelines/lingbot_world_fast \
  tests/unit/test_run_examples.py

python -m pytest -q tests/unit tests/server \
  -m "not gpu and not distributed and not slow and not quant"

CUDA_VISIBLE_DEVICES=4 \
python examples/run_examples.py \
  --pipeline lingbot_world_fast_i2v \
  --verbose

Results:

• LingBot-focused tests: 103 passed
• CI-scoped tests: 813 passed, 2 skipped, 307 deselected
• Example regression: PASS
  • frames: 9
  • resolution: 832x464
  • peak VRAM: 50.38 GiB
  • repeated fixed-seed comparison: PSNR=inf, SSIM=1.0000
• Workers exited normally with no remaining GPU processes.

The PSNR/SSIM comparison validates fixed-seed determinism against the first post-fix output; no historical example
baseline was available.

Plain pytest tests/ is not used as the completion criterion because the existing
tests/integration/test_pp_forward_consistency.py requires two distributed ranks but initializes only rank 0 when
invoked directly by pytest. The repository's CI marker scope shown above was used instead.

Performance Impact

Measured with 4x H100, chunk_size=3, and SageAttention SM90:

Metric	Before	After
Chunk mean	1.8010 s	1.4476 s
Chunk P90	-	1.4903 s
Chunk P99	-	1.5110 s
Weighted compute FPS	-	8.2898

Mean chunk latency decreased by approximately 19.6%.

AIPerf run ID:

0710537a-6302-41b2-a3b5-d1944ed7991f

Trade-offs

A session now allocates its runtime caches and up to two bounded condition slots before the first control arrives.
These resources remain session-owned and are released through the existing reverse-topological cleanup path.

Checklist

• Code follows project formatting and lint rules
• New regression tests added
• Documentation updated
• GPU regression completed
• Performance benchmark completed
• Full pre-commit suite executed

Related Issues

N/A

GPU Architecture Support

No kernel implementation is changed. Performance validation was performed on NVIDIA H100 (SM90).

中文

变更说明

本 PR 将 condition 编码移出交互控制的关键路径，以降低 LingBot-World-Fast 从控制输入到视频输出的延迟。

运行时会在 control 到达前有界预取 condition chunk，并在 denoise 完成后补充预取窗口。模型计算、输出顺序和数值
路径均保持不变。

本 PR 同时修复了 example 回归测试发现的两个问题：

• 异步 denoise 线程未进入 torch.inference_mode()；
• examples/run_examples.py 未显式关闭 pipeline 持有的 worker。

变更动机

Condition 编码仅依赖 session 图像和 chunk 位置，并不依赖用户的交互控制输入。

优化前，每个 condition 请求必须与 control 一起提交，导致以下阶段容易串行执行：

1. 等待 control；
2. 编码 condition；
3. DiT denoise；
4. VAE decode。

提前预取 condition 后，control 可以直接与已准备好的 condition 汇合，同时继续保证严格的 chunk 顺序和有界内存。

变更类型

• Bug 修复
• 性能优化
• 文档更新
• 破坏性变更
• 新功能

主要改动

• 在等待第一个 control 前初始化 LingBot runtime 和 streaming session。
• 每个 session 最多提前预取两个 condition chunk。
• 分别维护 condition cursor 和 control cursor。
• 拒绝重复、跳号或乱序的 control。
• 在 denoise 完成后补充 condition 预取窗口，与当前 chunk 的 decode 重叠。
• 增加 latency_anchor_artifact，从 control 被接受时开始计算控制延迟，而不是从 condition 预取开始计时。
• 让异步 denoise 在 torch.inference_mode() 中运行。
• 在 example 回归 runner 中显式关闭 pipeline 持有的 worker。
• 补充 scheduler、streaming、cleanup、顺序和 inference-mode 回归测试。
• 增加最终流水及优化前后 Mermaid 时序图文档。

本次改动不改变模型权重、扩散步数、dtype、attention 实现、VAE 数值路径、服务协议、环境变量或用户配置。

测试验证

• LingBot-World-Fast 单元测试
• CI 范围 unit/server 测试
• GPU example 回归测试
• 四卡 AIPerf 性能测试

执行命令：

python -m ruff check \
  examples/run_examples.py \
  telefuser/pipelines/lingbot_world_fast/streaming.py \
  tests/unit/pipelines/lingbot_world_fast/test_streaming.py \
  tests/unit/test_run_examples.py

python -m pytest -q \
  tests/unit/pipelines/lingbot_world_fast \
  tests/unit/test_run_examples.py

python -m pytest -q tests/unit tests/server \
  -m "not gpu and not distributed and not slow and not quant"

CUDA_VISIBLE_DEVICES=4 \
python examples/run_examples.py \
  --pipeline lingbot_world_fast_i2v \
  --verbose

测试结果：

• LingBot 相关测试：103 passed
• CI 范围测试：813 passed, 2 skipped, 307 deselected
• Example 回归：PASS
  • 帧数：9
  • 分辨率：832x464
  • 峰值显存：50.38 GiB
  • 固定 seed 重复生成对比：PSNR=inf、SSIM=1.0000
• Worker 均正常退出，未残留 GPU 进程。

PSNR/SSIM 用于验证修复后固定 seed 重复运行的确定性；测试环境中没有可用的历史 example 基线，因此该结果不表示
与历史提交的画质对比。

未将直接执行 pytest tests/ 作为完成标准，因为仓库现有的
tests/integration/test_pp_forward_consistency.py 需要两个分布式 rank，但由 pytest 直接执行时只初始化 rank 0。
因此采用了上面与仓库 CI 一致的 marker 范围。

性能影响

测试环境为 4x H100、chunk_size=3、SageAttention SM90：

指标	优化前	优化后
Chunk mean	1.8010 s	1.4476 s
Chunk P90	-	1.4903 s
Chunk P99	-	1.5110 s
加权 compute FPS	-	8.2898

Chunk 平均延迟降低约 19.6%。

AIPerf Run ID：

0710537a-6302-41b2-a3b5-d1944ed7991f

代价与影响

Session 现在会在第一个 control 到达前分配 runtime cache，并持有最多两个有界 condition slot。这些资源仍属于各自
session，并通过现有的逆拓扑 cleanup 流程释放。

检查清单

• 代码通过格式和 lint 检查
• 已增加回归测试
• 已更新设计文档
• 已完成 GPU 回归
• 已完成性能测试
• 已执行完整 pre-commit

关联 Issue

无。

GPU 架构支持

本 PR 未修改 kernel 实现，性能验证在 NVIDIA H100（SM90）上完成。

Suggested Commit Summary

The follow-up fix commit can use:

fix(lingbot): preserve inference mode and close regression workers