Skip to content

feat: ✨ Add xyxy_xywh function and from_transformers method for KeyPoints class added#1788

Merged
SkalskiP merged 4 commits intodevelopfrom
feat/transformers-keypoints
Feb 17, 2025
Merged

feat: ✨ Add xyxy_xywh function and from_transformers method for KeyPoints class added#1788
SkalskiP merged 4 commits intodevelopfrom
feat/transformers-keypoints

Conversation

@onuralpszr
Copy link
Contributor

@onuralpszr onuralpszr commented Feb 17, 2025

πŸš€ from_transformers method for KeyPoints class added to supervision 🎯

Open in Colab

✨ New Functionality

  • πŸ†• Bounding Box Conversion

    • Added xyxy_xywh function to convert bounding box coordinates from (x_min, y_min, x_max, y_max) format to (x, y, width, height) format.
    • πŸ“ Location: supervision/detection/utils.py
  • πŸ†• Transformers Model Integration

    • Introduced from_transformers class method in supervision/keypoint/core.py.
    • Enables creating KeyPoints objects from Transformers inference results.
    • πŸ“ Includes detailed documentation and usage examples.

πŸ”§ Updates to __init__.py

  • βœ… Updated supervision/__init__.py to include the xyxy_to_xywh function in imports.
  • βœ… Added xyxy_to_xywh to the __all__ list to ensure accessibility when the module is imported.

πŸ’‘ Feel free to test it using the provided Colab notebook!

@onuralpszr onuralpszr force-pushed the feat/transformers-keypoints branch from aa704f2 to fb1dbfd Compare February 17, 2025 00:18
@SkalskiP
Copy link
Collaborator

Hi @onuralpszr πŸ‘‹πŸ» thanks a lot for working on this feature. Could you please:

  • Add unit tests for xyxy_xywh
  • Add xyxy_xywh to docs/detection/utils.md
  • Slightly update naming convention in from_transformers docstring example
from PIL import Image
import requests
import supervision as sv
import torch
from transformers import (
    AutoProcessor,
    RTDetrForObjectDetection,
    VitPoseForPoseEstimation,
)

device = "cuda" if torch.cuda.is_available() else "cpu"
image = Image.open(<SOURCE_IMAGE_PATH>)

DETECTION_MODEL_ID = "PekingU/rtdetr_r50vd_coco_o365"

detection_processor = AutoProcessor.from_pretrained(DETECTION_MODEL_ID, use_fast=True)
detection_model = RTDetrForObjectDetection.from_pretrained(DETECTION_MODEL_ID, device_map=DEVICE)

inputs = detection_processor(images=frame, return_tensors="pt").to(DEVICE)

with torch.no_grad():
    outputs = detection_model(**inputs)

target_size = torch.tensor([(frame.height, frame.width)])
results = detection_processor.post_process_object_detection(
    outputs, target_sizes=target_size, threshold=0.3)

detections = sv.Detections.from_transformers(results[0])
boxes = sv.xyxy_xywh(detections[detections.class_id == 0].xyxy)

POSE_ESTIMATION_MODEL_ID = "usyd-community/vitpose-base-simple"

pose_estimation_processor = AutoProcessor.from_pretrained(POSE_ESTIMATION_MODEL_ID)
pose_estimation_model = VitPoseForPoseEstimation.from_pretrained(
    POSE_ESTIMATION_MODEL_ID, device_map=DEVICE)

inputs = pose_estimation_processor(frame, boxes=[boxes], return_tensors="pt").to(DEVICE)

with torch.no_grad():
    outputs = pose_estimation_model(**inputs)

results = pose_estimation_processor.post_process_pose_estimation(outputs, boxes=[boxes])
key_point = sv.KeyPoints.from_transformers(results[0])

docs: πŸ“ Update docstring for KeyPoints.from_transformers

Signed-off-by: Onuralp SEZER <[email protected]>
@onuralpszr onuralpszr force-pushed the feat/transformers-keypoints branch from 0f75e85 to cbaa605 Compare February 17, 2025 21:54
@SkalskiP
Copy link
Collaborator

Awesome @onuralpszr ! Merging! πŸ”₯

@SkalskiP SkalskiP merged commit 63388fd into develop Feb 17, 2025
5 checks passed
@onuralpszr
Copy link
Contributor Author

Awesome @onuralpszr ! Merging! πŸ”₯

Thank you very much ! :)

@Borda Borda deleted the feat/transformers-keypoints branch January 7, 2026 17:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants