Add COCO keypoint and BVH motion capture I/O support#930
Open
iftiquar wants to merge 8 commits intoneuroinformatics-unit:mainfrom
Open
Add COCO keypoint and BVH motion capture I/O support#930iftiquar wants to merge 8 commits intoneuroinformatics-unit:mainfrom
iftiquar wants to merge 8 commits intoneuroinformatics-unit:mainfrom
Conversation
Add loaders for two popular human motion tracking formats: COCO Keypoint Format: - Add ValidCOCOJSON validator with JSON schema validation - Add from_coco_file() loader registered as 'COCO' source software - Parse COCO keypoint annotations with visibility-to-confidence mapping - Support track_id for consistent individual tracking across frames - Handle invisible keypoints (v=0) as NaN positions BVH (Biovision Hierarchy) Format: - Add ValidBVHFile validator for BVH structure validation - Add from_bvh_file() loader registered as 'BVH' source software - Parse skeleton hierarchy and motion data - Compute 3D joint positions via forward kinematics - Derive fps from Frame Time field when not provided Also includes: - COCO keypoints JSON schema in _json_schemas.py - Comprehensive test suite for both loaders and validators - Gallery examples demonstrating COCO and BVH data loading Closes neuroinformatics-unit#182, relates to neuroinformatics-unit#175, neuroinformatics-unit#299, neuroinformatics-unit#581
for more information, see https://pre-commit.ci
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
… in examples - Replace tempfile.mktemp() with tempfile.NamedTemporaryFile(delete=False) in both gallery examples to resolve SonarCloud high-severity security alert - Add required COCO schema fields (width, height, num_keypoints, bbox, area, iscrowd, skeleton) to the example data so it passes ValidCOCOJSON validation
for more information, see https://pre-commit.ci
|
… comparison - Refactor _ds_from_coco_data into smaller helpers (_coco_individual_mapping, _coco_fill_arrays, _coco_fill_keypoints) to reduce cognitive complexity from 22 to well under 15 - Replace direct float equality (== 0.0) with pytest.approx in test
for more information, see https://pre-commit.ci
|
- ValidCOCOJSON: extract _check_coco_keypoint_lengths as a standalone custom check passed to _json_validator(custom_checks=...) instead of using __attrs_post_init__, matching the ValidROICollectionGeoJSON pattern - ValidBVHFile: replace __attrs_post_init__ with @file.validator decorator pattern, matching ValidAniposeCSV/ValidVIATracksCSV conventions - Add See Also cross-references to both validator docstrings - Add attribute-level docstrings following existing style
for more information, see https://pre-commit.ci
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
What is this PR
Why is this PR needed?
Closes #182. Relates to #175, #299, #581.
The
movementpackage currently supports pose estimation formats used primarily in animal behaviour research (DeepLabCut, SLEAP, LightningPose, Anipose). However, the project's roadmap calls for expanding I/O support to human motion tracking formats. Two of the most widely used are:This PR adds loaders, validators, tests, and gallery examples for both formats — with zero new dependencies.
What does this PR do?
from_coco_file()loader that parses COCO keypoint annotation JSON files intomovementposes datasets (movement/io/load_poses.py).ValidCOCOJSONfile validator with JSON schema validation and a custom check for keypoint array lengths (movement/validators/files.py).COCO_KEYPOINTS_SCHEMAJSON schema (movement/validators/_json_schemas.py).from_bvh_file()loader that parses BVH skeleton hierarchy and computes 3D joint positions via forward kinematics (movement/io/load_poses.py).ValidBVHFilefile validator that checks forHIERARCHYandMOTIONsections (movement/validators/files.py).SourceSoftwaretype alias to include"COCO"and"BVH"(movement/io/load.py).tests/test_unit/test_io/test_load_coco_and_bvh.py).load_coco_data.pyandload_bvh_data.py(examples/).COCO loader details
The COCO loader (
from_coco_file()) handles the following:Each image in the JSON is treated as one time frame (sorted by
idto establish temporal order).Each annotation per image is treated as a separate individual.
If annotations include a
track_idfield, it is used for consistent individual identity across frames; otherwise, individuals are numbered per frame (id_0,id_1, …).Visibility flag mapping to confidence scores:
v0(not labelled)NaN0.01(labelled, occluded)0.5 × score2(labelled, visible)1.0 × scoreIf the annotation has no
scorefield, a default of1.0is used.BVH loader details
The BVH loader (
from_bvh_file()) handles the following:HIERARCHYsection to build a skeleton tree (joint names, offsets, channel types, parent–child relationships).MOTIONsection for per-frame channel values.ZXY,XYZ,YZX, etc.) as specified in each joint'sCHANNELSline.fpsautomatically from the BVHFrame Timefield (overridable via thefpsparameter).End Sitenodes are excluded from the output; onlyROOTandJOINTnodes become keypoints.Design decisions
Following existing patterns. Both loaders use the
@register_loaderdecorator,from_numpy()for dataset construction, andcast()for type narrowing — identical to the existing DeepLabCut, SLEAP, and VIA-tracks loaders.Validator conventions.
ValidCOCOJSONfollows theValidROICollectionGeoJSONpattern: schema validation via_json_validator()with a standalonecustom_checkscallable (_check_coco_keypoint_lengths).ValidBVHFilefollows theValidAniposeCSVpattern: basic file validation via_file_validator()plus a@file.validatormethod for format-specific structure checks.No new dependencies. COCO parsing uses only stdlib
json(already imported by_json_validator). BVH parsing and forward kinematics use onlynumpy(already a core dependency). This keeps the installation lightweight.BVH forward kinematics from scratch. Rather than adding a dependency on a motion capture library, the FK implementation uses standard Euler-angle-to-rotation-matrix conversion. The code is factored into small, tested helpers (
_axis_rotation_matrix,_euler_to_rotation_matrix,_extract_bvh_channels,_bvh_forward_kinematics).Minimal COCO schema. The JSON schema validates structural requirements (
images,annotations,categorieswith their core fields) but does not enforce optional COCO fields likebbox,area, oriscrowd. This allows the loader to work with both full COCO annotation files and lightweight keypoint-only exports from tools like MMPose.How has this PR been tested?
tests/test_unit/test_io/test_load_coco_and_bvh.py): 30+ tests organised into four classes:TestCOCOLoader: valid loading, shape, keypoint names, track_id, invisible keypoints, missing score, fps handling, source attributesTestCOCOValidation: valid file, missing keys, wrong keypoint lengths, wrong extensionTestBVHLoader: valid loading, shape, joint names, 3D space, fps from Frame Time, fps override, root positions at frame 0 and frame 1, source attributes, NaN confidence, single individualTestBVHValidation: valid file, missing HIERARCHY, missing MOTION, wrong extensionTestLoadDatasetIntegration: both formats viaload_dataset()unified interfaceconftest.py(helpers,wrong_extension_file).Is this a breaking change?
No.
Does this PR require an update to the documentation?
See Alsocross-references between loaders and validators.docs/make_api.py(no manual changes needed).Checklist