Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/CI_deprecated/generate_gh_pages.sh
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ __AUTHOR__="openpose"
# must be installed.
# - Doxygen configuration file must have the destination directory empty and
# source code directory with a $(CI_BUILD_DIR) prefix.
# - An gh-pages branch should already exist. See below for mor info on hoe to
# - An gh-pages branch should already exist. See below for more info on how to
# create a gh-pages branch.
#
# Required global variables:
Expand Down Expand Up @@ -93,7 +93,7 @@ if [ -d "html" ] && [ -f "html/index.html" ]; then
git commit -m "Deploy code docs to GitHub Pages Travis build: ${CI_BUILD_NUMBER}" -m "Commit: ${CI_COMMIT}"

# Force push to the remote gh-pages branch.
# The ouput is redirected to /dev/null to hide any sensitive credential data
# The output is redirected to /dev/null to hide any sensitive credential data
# that might otherwise be exposed.
git push --force "https://${GH_REPO_TOKEN}@${GH_REPO_REF}" > /dev/null 2>&1
else
Expand Down
2 changes: 1 addition & 1 deletion CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -265,7 +265,7 @@ endif (PROFILER_ENABLED)

# Suboptions for GPU architectures
if (${GPU_MODE} MATCHES "CUDA")
set(CUDA_ARCH Auto CACHE STRING "Select target NVIDIA GPU achitecture.")
set(CUDA_ARCH Auto CACHE STRING "Select target NVIDIA GPU architecture.")
set_property(CACHE CUDA_ARCH PROPERTY STRINGS Auto All Manual)
endif (${GPU_MODE} MATCHES "CUDA")

Expand Down
41 changes: 23 additions & 18 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,24 +70,29 @@ We show an inference time comparison between the 3 available pose estimation lib


## Features
- **Main Functionality**:
- **2D real-time multi-person keypoint detection**:
- 15, 18 or **25-keypoint body/foot keypoint estimation**, including **6 foot keypoints**. **Runtime invariant to number of detected people**.
- **2x21-keypoint hand keypoint estimation**. **Runtime depends on number of detected people**. See [**OpenPose Training**](https://github.com/CMU-Perceptual-Computing-Lab/openpose_train) for a runtime invariant alternative.
- **70-keypoint face keypoint estimation**. **Runtime depends on number of detected people**. See [**OpenPose Training**](https://github.com/CMU-Perceptual-Computing-Lab/openpose_train) for a runtime invariant alternative.
- [**3D real-time single-person keypoint detection**](doc/advanced/3d_reconstruction_module.md):
- 3D triangulation from multiple single views.
- Synchronization of Flir cameras handled.
- Compatible with Flir/Point Grey cameras.
- [**Calibration toolbox**](doc/advanced/calibration_module.md): Estimation of distortion, intrinsic, and extrinsic camera parameters.
- **Single-person tracking** for further speedup or visual smoothing.
- **Input**: Image, video, webcam, Flir/Point Grey, IP camera, and support to add your own custom input source (e.g., depth camera).
- **Output**: Basic image + keypoint display/saving (PNG, JPG, AVI, ...), keypoint saving (JSON, XML, YML, ...), keypoints as array class, and support to add your own custom output code (e.g., some fancy UI).
- **OS**: Ubuntu (20, 18, 16, 14), Windows (10, 8), Mac OSX, Nvidia TX2.
- **Hardware compatibility**: CUDA (Nvidia GPU), OpenCL (AMD GPU), and non-GPU (CPU-only) versions.
- **Usage Alternatives**:
- [**Command-line demo**](doc/01_demo.md) for built-in functionality.
- [**C++ API**](doc/04_cpp_api.md/) and [**Python API**](doc/03_python_api.md) for custom functionality. E.g., adding your custom inputs, pre-processing, post-posprocessing, and output steps.
**Main Functionality**:
- **2D real-time multi-person keypoint detection**:
- 15, 18 or **25-keypoint body/foot keypoint estimation**, including **6 foot keypoints**. **Runtime invariant to number of detected people**.
- **2x21-keypoint hand keypoint estimation**. **Runtime depends on number of detected people**. See [**OpenPose Training**](https://github.com/CMU-Perceptual-Computing-Lab/openpose_train) for a runtime invariant alternative.
- **70-keypoint face keypoint estimation**. **Runtime depends on number of detected people**. See [**OpenPose Training**](https://github.com/CMU-Perceptual-Computing-Lab/openpose_train) for a runtime invariant alternative.
- [**3D real-time single-person keypoint detection**](doc/advanced/3d_reconstruction_module.md):
- 3D triangulation from multiple single views.
- Synchronization of Flir cameras handled.
- Compatible with Flir/Point Grey cameras.
- [**Calibration toolbox**](doc/advanced/calibration_module.md): Estimation of distortion, intrinsic, and extrinsic camera parameters.
- **Single-person tracking** for further speedup or visual smoothing.

**Input**: Image, video, webcam, Flir/Point Grey, IP camera, and support to add your own custom input source (e.g., depth camera).

**Output**: Basic image + keypoint display/saving (PNG, JPG, AVI, ...), keypoint saving (JSON, XML, YML, ...), keypoints as array class, and support to add your own custom output code (e.g., some fancy UI).

**OS**: Ubuntu (20, 18, 16, 14), Windows (10, 8), Mac OSX, Nvidia TX2.

**Hardware compatibility**: CUDA (Nvidia GPU), OpenCL (AMD GPU), and non-GPU (CPU-only) versions.

**Usage Alternatives**:
- [**Command-line demo**](doc/01_demo.md) for built-in functionality.
- [**C++ API**](doc/04_cpp_api.md/) and [**Python API**](doc/03_python_api.md) for custom functionality. E.g., adding your custom inputs, pre-processing, post-posprocessing, and output steps.

For further details, check the [major released features](doc/07_major_released_features.md) and [release notes](doc/08_release_notes.md) docs.

Expand Down
6 changes: 3 additions & 3 deletions cmake/Cuda.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@ function(op_detect_installed_gpus out_variable)
if (NOT WIN32)
string(REPLACE "2.1" "2.1(2.0)" __nvcc_out "${__nvcc_out}")
endif (NOT WIN32)
set(CUDA_gpu_detect_output ${__nvcc_out} CACHE INTERNAL "Returned GPU architetures from op_detect_gpus tool" FORCE)
set(CUDA_gpu_detect_output ${__nvcc_out} CACHE INTERNAL "Returned GPU architectures from op_detect_gpus tool" FORCE)
endif ()
endif ()

Expand Down Expand Up @@ -104,14 +104,14 @@ function(op_select_nvcc_arch_flags out_variable)
endif ()

# set CUDA_ARCH strings (so it will be seen as dropbox in CMake-Gui)
# set(CUDA_ARCH ${__archs_name_default} CACHE STRING "Select target NVIDIA GPU achitecture.")
# set(CUDA_ARCH ${__archs_name_default} CACHE STRING "Select target NVIDIA GPU architecture.")
# set_property( CACHE CUDA_ARCH PROPERTY STRINGS "" ${__archs_names} )
# mark_as_advanced(CUDA_ARCH)

# verify CUDA_ARCH value
if (NOT ";${__archs_names};" MATCHES ";${CUDA_ARCH};")
string(REPLACE ";" ", " __archs_names "${__archs_names}")
message(FATAL_ERROR "Only ${__archs_names} architeture names are supported.")
message(FATAL_ERROR "Only ${__archs_names} architecture names are supported.")
endif ()

if (${CUDA_ARCH} STREQUAL "Manual")
Expand Down
4 changes: 2 additions & 2 deletions doc/01_demo.md
Original file line number Diff line number Diff line change
Expand Up @@ -233,7 +233,7 @@ bin\OpenPoseDemo.exe --flir_camera --3d --number_people_max 1 --face --hand
./build/examples/openpose/openpose.bin --flir_camera --num_gpu 0 --write_images output_folder_path/ --write_images_format jpg
```

4. Reading and processing previouly saved stereo camera images
4. Reading and processing previously saved stereo camera images
```
# Ubuntu and Mac (same flags for Windows)
# Optionally add `--face` and/or `--hand` to include face and/or hands
Expand Down Expand Up @@ -348,7 +348,7 @@ Additional notes:
### Mac OSX Additional Step
**If you are not using Mac, or you are using Mac with `CPU_only`, you can skip this section.**

If you are using a Mac and selected `OPENCL` support, and it has an AMD graphics card, that means that the machine has 2 GPUs that are not compatible with each other (AMD and Intel). Then, you will have to manually select one of them (the AMD one should be more poweful). To do that, first check which device your Graphics card is set under. Most likely, your AMD device will be device 2.
If you are using a Mac and selected `OPENCL` support, and it has an AMD graphics card, that means that the machine has 2 GPUs that are not compatible with each other (AMD and Intel). Then, you will have to manually select one of them (the AMD one should be more powerful). To do that, first check which device your Graphics card is set under. Most likely, your AMD device will be device 2.
```bash
clinfo
```
Expand Down
4 changes: 2 additions & 2 deletions doc/02_output.md
Original file line number Diff line number Diff line change
Expand Up @@ -68,9 +68,9 @@ The output of the JSON files consist of a set of keypoints, whose ordering is re


## JSON Output Format
There are 2 alternatives to save the OpenPose output. But both of them follow the keypoint ordering described in the the section [Keypoint Ordering in C++/Python](#body-keypoints-in-c-python) section (which you should read next).
There are 2 alternatives to save the OpenPose output. But both of them follow the keypoint ordering described in the section [Keypoint Ordering in C++/Python](#body-keypoints-in-c-python) section (which you should read next).

1. The `--write_json` flag saves the people pose data onto JSON files. Each file represents a frame, it has a `people` array of objects, where each object has:
1. The `--write_json` flag saves the people pose data into JSON files. Each file represents a frame, it has a `people` array of objects, where each object has:
1. `pose_keypoints_2d`: Body part locations (`x`, `y`) and detection confidence (`c`) formatted as `x0,y0,c0,x1,y1,c1,...`. The coordinates `x` and `y` can be normalized to the range [0,1], [-1,1], [0, source size], [0, output size], etc. (see the flag `--keypoint_scale` for more information), while the confidence score (`c`) in the range [0,1].
2. `face_keypoints_2d`, `hand_left_keypoints_2d`, and `hand_right_keypoints_2d` are analogous to `pose_keypoints_2d` but applied to the face and hand parts.
3. `body_keypoints_3d`, `face_keypoints_3d`, `hand_left_keypoints_2d`, and `hand_right_keypoints_2d` are analogous but applied to the 3-D parts. They are empty if `--3d` is not enabled. Their format is `x0,y0,z0,c0,x1,y1,z1,c1,...`, where `c` is 1 or 0 depending on whether the 3-D reconstruction was successful or not.
Expand Down
4 changes: 2 additions & 2 deletions doc/03_python_api.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ You should be familiar with the [**OpenPose Demo**](01_demo.md) and the main Ope
## Advance Introduction (Optional)
This module exposes a Python API for OpenPose. It is effectively a wrapper that replicates most of the functionality of the [op::Wrapper class](https://github.com/CMU-Perceptual-Computing-Lab/openpose/blob/master/include/openpose/wrapper/wrapper.hpp) and allows you to populate and retrieve data from the [op::Datum class](https://github.com/CMU-Perceptual-Computing-Lab/openpose/blob/master/include/openpose/core/datum.hpp) using standard Python and Numpy constructs.

The Python API is analagous to the C++ function calls. You may find them in [python/openpose/openpose_python.cpp#L194](https://github.com/CMU-Perceptual-Computing-Lab/openpose/blob/master/python/openpose/openpose_python.cpp#L194).
The Python API is analogous to the C++ function calls. You may find them in [python/openpose/openpose_python.cpp#L194](https://github.com/CMU-Perceptual-Computing-Lab/openpose/blob/master/python/openpose/openpose_python.cpp#L194).

The Python API is rather simple: `op::Array<float>` and `cv::Mat` objects get casted to numpy arrays automatically. Every other data structure based on the standard library is automatically converted into Python objects. For example, an `std::vector<std::vector<float>>` would become `[[item, item], [item, item]]`, etc. We also provide a casting of `op::Rectangle` and `op::Point` which simply expose setter getter for [x, y, width, height], etc.

Expand Down Expand Up @@ -90,7 +90,7 @@ For quick prototyping, you can simply duplicate and rename any of the existing s


## Exporting Python OpenPose
Note: This step is only required if you are moving the `*.py` files outside their original location, or writting new `*.py` scripts outside `build/examples/tutorial_api_python`.
Note: This step is only required if you are moving the `*.py` files outside their original location, or writing new `*.py` scripts outside `build/examples/tutorial_api_python`.

Ubuntu/OSX:

Expand Down
4 changes: 2 additions & 2 deletions doc/05_faq.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ OpenPose Doc - Frequently Asked Questions (FAQ)
## FAQ
### Errors
#### Download Server Down
**Q: The download server is down, Download Hash missmatch, Error 502: Bad Gateway, etc. I.e., I cannot download the OpenPose models and/or 3rd party libraries.**
**Q: The download server is down, Download Hash mismatch, Error 502: Bad Gateway, etc. I.e., I cannot download the OpenPose models and/or 3rd party libraries.**
**A**: The are 2 alternatives. Option 1 (recommended): Download the links from [1602#issuecomment-641653411](https://github.com/CMU-Perceptual-Computing-Lab/openpose/issues/1602#issuecomment-641653411) and download them in the places indicated by [doc/1_prerequisites.md](1_prerequisites.md). Option 2: If you downloaded the models and dependencies to some local server, you could set the advanced CMake property `DOWNLOAD_SERVER` to link OpenPose to your local mirror server instead.


Expand Down Expand Up @@ -261,7 +261,7 @@ Check the [doc/06_maximizing_openpose_speed.md#profiling-speed](06_maximizing_op
COCO model will eventually be removed. BODY_25 model is faster, more accurate, and it includes foot keypoints. However, `COCO` requires less memory on GPU (being able to fit into 2GB GPUs with the default settings) and it runs faster on CPU-only mode. `MPI` model is only meant for people requiring the MPI-keypoint structure. It is also slower than BODY_25 and far less accurate.

Key differences:
- `BODY_25` is the fastest one on the GPU, the most accurate one, and the one with the highest number of keypoints (including foot keypoints!), highly recommended. It also uses the most ammount of RAM/GPU memory.
- `BODY_25` is the fastest one on the GPU, the most accurate one, and the one with the highest number of keypoints (including foot keypoints!), highly recommended. It also uses the most amount of RAM/GPU memory.
- `COCO` is faster than `BODY_25` on the CPU but slower on the GPU and less accurate.
- `MPI` is the fastest on the CPU but the least accurate of all 3.

Expand Down
2 changes: 1 addition & 1 deletion doc/06_maximizing_openpose_speed.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ On Ubuntu (for OS versions older than 20), you can also boost CPU-only speed by
OpenPose displays the FPS in the basic GUI. However, more complex speed metrics can be obtained from the command line while running OpenPose. In order to obtain those, compile OpenPose with the `PROFILER_ENABLED` flag on CMake-gui. OpenPose will automatically display time measurements for each subthread after processing `F` frames (by default `F = 1000`, but it can be modified with the `--profile_speed` flag, e.g. `--profile_speed 100`).

- Time measurement for 1 graphic card: The FPS will be the slowest time displayed in your terminal command line (as OpenPose is multi-threaded). Times are in milliseconds, so `FPS = 1000/millisecond_measurement`.
- Time measurement for >1 graphic cards: Assuming `n` graphic cards, you will have to wait up to `n` x `F` frames to visualize each graphic card speed (as the frames are splitted among them). In addition, the FPS would be: `FPS = minFPS(speed_per_GPU/n, worst_time_measurement_other_than_GPUs)`. For < 4 GPUs, this is usually `FPS = speed_per_GPU/n`.
- Time measurement for >1 graphic cards: Assuming `n` graphic cards, you will have to wait up to `n` x `F` frames to visualize each graphic card speed (as the frames are split among them). In addition, the FPS would be: `FPS = minFPS(speed_per_GPU/n, worst_time_measurement_other_than_GPUs)`. For < 4 GPUs, this is usually `FPS = speed_per_GPU/n`.

Make sure that `wPoseExtractor` time is the slowest timing. Otherwise the input producer (video/webcam codecs issues with OpenCV, images too big, etc.) or the GUI display (use OpenGL support as detailed in the next section (`Speed Up Preserving Accuracy`) might not be optimized.

Expand Down
6 changes: 3 additions & 3 deletions doc/08_release_notes.md
Original file line number Diff line number Diff line change
Expand Up @@ -160,7 +160,7 @@ OpenPose Doc - Release Notes
5. Added `--write_json` as a new version of `--write_keypoint_json`. It includes the body part candidates (if enabled), as well as any extra information added in the future (e.g., person ID).
6. Body part candidates can be retrieved in Datum and saved with `--write_json`.
2. Functions or parameters renamed:
1. `PoseParameters` splitted into `PoseParameters` and `PoseParametersRender` and const parameters turned into functions for more clarity.
1. `PoseParameters` split into `PoseParameters` and `PoseParametersRender` and const parameters turned into functions for more clarity.
3. Main bugs fixed:
1. Render working on images > 4K (#324).
2. Cleaned redundant arguments on `getAverageScore` and `getKeypointsArea`.
Expand Down Expand Up @@ -235,7 +235,7 @@ OpenPose Doc - Release Notes
11. 3D keypoints and camera parameters in meters (instead of millimeters) in order to reduce numerical errors.
12. New `PoseExtractor` class to contain future ID and tracking algorithms as well as the current OpenPose keypoint detection algorithm.
13. Added initial alpha versions of the `tracking` and `identification` modules (for now disabled but available in the source code), including `PersonIdExtractor` and `PersonTracker`. `PersonIdExtractor` includes greedy matrix OP-LK matching.
14. Added catchs to all demos for higher debug information.
14. Added catches to all demos for higher debug information.
15. GUI includes the capability of dynamically enabling/disabling the face, hand, and 3-D rendering, as well as more clear visualization for skeleton, background, heatmap addition, and PAF addition channels.
16. When GUI changes some parameter from PoseExtractorNet, there is a log to notify the user of the change.
17. Deprecated flag `--write_keypoint_json` removed (`--write_json` is the equivalent since version 1.2.1).
Expand Down Expand Up @@ -387,7 +387,7 @@ OpenPose Doc - Release Notes
1. Main improvements:
1. Multi-camera (3D) working on Asynchronous mode.
1. Functions `WrapperT::waitAndEmplace()` and `WrapperT::tryEmplace()` improved, allowing multi-camera/3-D (`TDatums` of size > 1).
2. Added `createMultiviewTDatum()` to auto-generate a `TDatums` for multi-camera/3-D from a single cv::Mat (that is splitted) and the desired camera parameter matrices.
2. Added `createMultiviewTDatum()` to auto-generate a `TDatums` for multi-camera/3-D from a single cv::Mat (that is split) and the desired camera parameter matrices.
3. Added `examples/tutorial_api_cpp/11_asynchronous_custom_input_multi_camera.cpp` for a test example.
2. Created Matrix as container of cv::Mat, and String as container of std::string.
3. After replacing cv::Mat by Matrix, headers do not contain any 3rd-party library includes nor functions. This way, OpenPose can be exported without needing 3rd-party includes nor static library files (e.g., lib files in Windows), allowing people to use their own versions of OpenCV, Eigen, etc. without conflicting with OpenPose. Dynamic library files (e.g., `dll` files in Windows, `so` in Ubuntu) are still required.
Expand Down
2 changes: 1 addition & 1 deletion doc/09_authors_and_contributors.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ OpenPose would not be possible without the [**CMU Panoptic Studio dataset**](htt
We would also like to thank the following people, who have contributed to key components of OpenPose:
1. [Bikramjot Hanzra](https://www.linkedin.com/in/bikz05): Former OpenPose maintainer, CMake (Ubuntu and Windows) version, and initial Travis Build version for Ubuntu.
2. [Donglai Xiang](https://xiangdonglai.github.io): Camera calibration toolbox improvement, including the implementation of its bundle adjustment algorithm.
3. [Luis Fernando Fraga](https://github.com/fragalfernando): Implementation of Lukas-Kanade algorith and person ID extractor.
3. [Luis Fernando Fraga](https://github.com/fragalfernando): Implementation of Lukas-Kanade algorithm and person ID extractor.
4. [Akash Patwal](https://www.linkedin.com/in/akash-patwal-63a12012a): Speedup of the CUDA image resize and visual skeleton rendering, as well as extension that allows OpenPose to speedup linearly to more than 4 GPUs.
5. [Helen Medina](https://github.com/helen-medina): First Windows version.
6. [Matthijs van der Burgh](https://github.com/MatthijsBurgh): First GitHub Actions CI version for Ubuntu and Mac, and ported all the deprecated Travis CI tests into the new CI system.
Expand Down
Loading