feat(pi05): complete the native C++ runtime interface#144
Open
LiangSu8899 wants to merge 66 commits into
Open
Conversation
This was referenced Jul 11, 2026
Member
Author
|
Shared FA2 ownership and packaging has been split into prerequisite PR #145. The model runtime continues to consume the same |
6 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
frt_model_runtime_open_v1Dependency and merge order
This PR depends on #145 for the shared Python-free FA2 runtime used by the SM120 backend. Merge #145 first, then rebase this branch before merging so the shared kernel change lands once through its owner review.
Design boundaries
runtime/andexec/remain mechanism-only and contain no model, backend, dataset, or calibration policyfrt_model_runtime_v1face; consumers do not branch on model or backend kindcpp/models/pi05/; dataset traversal, decoding, sampling, and camera synchronization remain host policySM110 calibration contract
frt_pi05_calibration_create_v1,observe_v1, andfinalize_v1accept complete observations and publish one atomic safetensors artifact. Calls may be repeated for dataset calibration. Camera arrays are canonicalized by the Pi0.5 namesimage,wrist_image, andwrist_image_right; missing, duplicate, unknown, or non-RGB inputs reject the observation without changing the successful sample count.Artifacts bind schema/reducer version, observed hardware, checkpoint and tokenizer SHA-256, precision, fixed model shapes, successful sample count, and percentile. Runtime open requires a compatible artifact on SM110, verifies the checkpoint digest again, and includes the artifact SHA-256 in producer identity. Different calibration bytes therefore produce a different fingerprint and cannot silently restore an old capsule.
Loading and execution
The native loader accepts F32, BF16, and F16 safetensors sources, including valid unaligned payload offsets. It emits final BF16 or F16/FP8 layouts directly without a checkpoint-sized float dictionary or a second implicit cache format. Full checkpoint hashing runs concurrently with independent materialization, and the runtime is published only after both succeed.
precision="auto"resolves to BF16 on SM120 and FP8 E4M3 on SM110. Explicit hardware/precision mismatches fail setup. Native C++ NVFP4 is not advertised by this producer; existing Python NVFP4 paths are unchanged.The hot model face declares real STAGED verbs for prompt, state, images, and logical actions, with SWAP windows for noise and raw actions. Capture produces one fixed graph variant. The measured service-loop range includes input staging, replay, and output readback.
Compatibility
runtime/includeorexec/includeFLASHRT_CPP_WITH_THOR_FP8; existing defaults remain unchangedValidation
git diff --check, and static analysis passThe public build, calibration, artifact, runtime-port, identity, and validation procedures are documented in
docs/pi05_thor_native_fp8.mdand linked fromUSAGE.md.