Add MMPose wholebody estimator#231
Open
bricksdont wants to merge 9 commits into
Open
Conversation
Adds mmposewholebody.py wrapping MMPoseInferencer('wholebody') to load a video
into a Pose using the canonical COCO Wholebody 133 header from
cocowholebody133_header.py. MMPose is an optional dependency; importing the
module without it installed raises ImportError immediately (same pattern as
holistic.py / mediapipe).
Frames where no person is detected are not skipped — a zeroed, fully-masked row
is inserted instead so the output frame count stays aligned with the video.
Downstream code can distinguish "no detection" from a real keypoint via the mask.
Tests cover: header/components (no MMPose required), output shape and metadata,
empty-frame masking, all-empty video, and version default. MMPoseInferencer is
mocked via sys.modules stubs so the test suite runs without MMPose installed.
Co-Authored-By: catherine-o-brien <catherine-o-brien@users.noreply.github.com>
Co-Authored-By: GerrySant <GerrySant@users.noreply.github.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…imators Restructures "Integration with External Data Sources" to label OpenPose support as core and AlphaPose / MMPose as experimental. Adds MMPose wholebody loader example with install instructions. Co-Authored-By: catherine-o-brien <catherine-o-brien@users.noreply.github.com> Co-Authored-By: GerrySant <GerrySant@users.noreply.github.com> Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds a [project.optional-dependencies] mmpose group listing mmcv, mmengine, mmdet, and mmpose (all >=their first stable 1.x/2.x releases), installable via pip install pose_format[mmpose]. Co-Authored-By: catherine-o-brien <catherine-o-brien@users.noreply.github.com> Co-Authored-By: GerrySant <GerrySant@users.noreply.github.com> Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Version floors tightened to match a known-good OpenMMLab 2.x combination (mmcv 2.1.0 / mmengine 0.10.7 / mmdet 3.3.0 / mmpose 1.3.2) verified by ZurichNLP's install script. Also clarifies that mmcv for GPU requires the OpenMMLab CUDA-specific index, not plain pip install. Co-Authored-By: catherine-o-brien <catherine-o-brien@users.noreply.github.com> Co-Authored-By: GerrySant <GerrySant@users.noreply.github.com> Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
MMPose runs inference on raw video rather than loading a pre-existing keypoint format, so it does not belong in this section. Mediapipe Holistic is handled via the CLI (section 2) and also does not fit here. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
"load" implies ingesting a pre-existing keypoint file (like OpenPose/AlphaPose loaders); this function runs inference on raw video, so "estimate" is more accurate. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Collaborator
Author
|
mmpose dependencies are rather heavy, so they are excluded from the tests / CI |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR adds support for running MMPose wholebody inference on a video and converting the result into the library's
.poseformat.pose_format/utils/mmposewholebody.py— new module withestimate_mmpose_wholebody(input_path, fps, width, height). RunsMMPoseInferencer('wholebody')on a video file and returns aPoseobject using the canonical COCO Wholebody 133 header (fromcocowholebody133_header.py, introduced in Add COCO Wholebody 133 keypoint schema and generic.py support #226). Frames with no detected person are represented as fully-masked zero rows, preserving temporal alignment with the source video.pose_format/utils/mmposewholebody_test.py— 8 unit tests covering header structure, output shape, masked empty frames, and the all-empty-video case. MMPose is stubbed viasys.modulesso tests run without an actual MMPose installation.pyproject.toml— newmmposeoptional install target (pip install pose-format[mmpose]) with version floors derived from a known-good OpenMMLab 2.x combination.Design notes
Not wired into the CLI —
video_to_pose/videos_to_posescurrently only support mediapipe. MMPose could be added to the CLI later; this PR intentionally keeps that scope separate.Format vs. estimator — MMPose wholebody outputs COCO Wholebody 133 keypoints. A
.posefile produced by this function is structurally identical to one from any other COCO Wholebody 133 estimator (OpenPifPaf, SDPose, etc.).detect_known_pose_formatcorrectly identifies such files as"coco_wholebody_133". MMPose is therefore an estimator that produces an existing named format, not a new format in its own right.mmcvGPU install caveat — plainpip install mmcvinstalls a CPU-only build; GPU users need the OpenMMLab CUDA-specific index. TheImportErrormessage in the module links to the mmcv install guide.Relationship to other PRs
This builds on #226 (COCO Wholebody 133 canonical header). Future PRs will add OpenPifPaf and SDPose estimators using the same header.
Test plan
pytest src/python/pose_format/utils/mmposewholebody_test.py -vpasses without MMPose installedpytest src/python/pose_format/utils/generic_test.py -vpasses (no regressions)estimate_mmpose_wholebody("video.mp4", fps=25, width=1920, height=1080)returns aPosewithdetect_known_pose_format→"coco_wholebody_133"🤖 Generated with Claude Code