Pose Proposal Network Tested + Pre-Trained Models => Google Drive + Documentation Update (#302)

* Fix PPN post-processing

* Fix PPN post-processing: re

* Clean PPN header

* Remove useless scripts

* More details on TensorRT building

* Doc Comment on PPN

* Models => Google Drive | Update Documents

* Format C++ codes

Update CI

Update Install Scripts

Update Install Scripts

Update Install Scripts

* Fix FAKE

Author: ganler

.github/workflows/ci.yml

@@ -7,16 +7,29 @@ jobs:
 
     # https://help.github.com/en/articles/virtual-environments-for-github-actions#supported-virtual-environments
     runs-on: ubuntu-18.04
+    strategy:
+      matrix:
+        python-version: [3.6, 3.7, 3.8]
 
     steps:
-    - uses: actions/checkout@v1
-    - run: sudo apt-get install libopencv-dev libgflags-dev # dependencies
-    - run: sh scripts/download-test-data.sh
-    - run: sh scripts/download-tinyvgg-model.sh
-    - run: sh scripts/download-openpose-thin-model.sh
-    - run: sh scripts/download-openpose-res50-model.sh
-    - run: sh scripts/download-openpose-coco-model.sh
-    - run: cmake . -DBUILD_TESTS=1 -DBUILD_FAKE=1 -DBUILD_EXAMPLES=1 -DBUILD_LIB=1 -DBUILD_USER_CODES=0 -DEXECUTABLE_OUTPUT_PATH=./bin
+    - uses: actions/checkout@v2
+    - name: Set up Python ${{ matrix.python-version }}
+      uses: actions/setup-python@v2
+      with:
+        python-version: ${{ matrix.python-version }}
+    - name: Initialize Python Env
+      run: python3 -m pip install --upgrade pip
+    - name: Install System Dependencies
+      run: sudo apt-get install libopencv-dev libgflags-dev # dependencies
+    - name: Check download scripts.
+      run: |
+        sh scripts/download-test-data.sh
+        sh scripts/download-tinyvgg-model.sh
+        sh scripts/download-openpose-thin-model.sh
+        sh scripts/download-openpose-res50-model.sh
+        sh scripts/download-openpose-coco-model.sh
+    - name: Build Project(NO GPU)
+      run: cmake . -DBUILD_TESTS=1 -DBUILD_FAKE=1 -DBUILD_EXAMPLES=1 -DBUILD_LIB=1 -DBUILD_USER_CODES=0 -DEXECUTABLE_OUTPUT_PATH=./bin
     - run: cmake --build . --config Release
 
     # - run: ctest -C Release

.gitignore

@@ -44,6 +44,7 @@ venv
 _build
 docs/make.bat
 examples/user_codes/*.cpp
+debug.*
 
 !docs/Makefile
 !docs/markdown/images/*

README.md

@@ -4,7 +4,7 @@
 
 HyperPose is a library for building human pose estimation systems that can efficiently operate in the wild.
 
-> **Note**: We are in the process of migrating our APIs from 1.0 to 2.0. The migration is expected to finish by July 2020.
+> **News**: The PoseProposal inference model is released! See the HyperPose models on [Google Drive](https://drive.google.com/drive/folders/1w9EjMkrjxOmMw3Rf6fXXkiv_ge7M99jR?usp=sharing).
 
 ## Features
 
@@ -19,14 +19,14 @@ You can install HyperPose and learn its APIs through [Documentation](https://hyp
 
 ## Example
 
-We provide an example to show human pose estimation achieved by HyperPose. You need to install CUDA Toolkit 10+, TensorRT 7+, OpenCV 3.2+ and gFlags (cmake version), and enable C++ 17 support. Once the prerequisite are ready, run the following script:
+We provide an example to show human pose estimation achieved by HyperPose. You need to install CUDA Toolkit 10+, TensorRT 7+, OpenCV 3.2+ and gFlags (cmake version), and enable C++ 17 support. Once the prerequisite are met, run the following script:
 
 ```bash
-sudo apt -y install git cmake build-essential subversion curl libgflags-dev # libopencv-dev # [optional] 
+sudo apt -y install git cmake build-essential subversion libgflags-dev libopencv-dev
 sh scripts/download-test-data.sh         # Install data for examples.
 sh scripts/download-tinyvgg-model.sh     # Install tiny-vgg model.
 mkdir build && cd build
-cmake .. -DCMAKE_BUILD_TYPE=RELEASE && make -j$(nproc) # Build library && examples.
+cmake .. -DCMAKE_BUILD_TYPE=RELEASE && make -j # Build library && examples.
 ./example.operator_api_batched_images_paf    # The ouput images will be in the build folder.
 ```
 

docs/markdown/design/design.md

@@ -47,7 +47,7 @@ int main() {
     using namespace hyperpose;
 
     const cv::Size network_resolution{384, 256};
-    const dnn::uff uff_model{ "../data/models/hao28-600000-256x384.uff", "image", {"outputs/conf", "outputs/paf"} };
+    const dnn::uff uff_model{ "../data/models/TinyVGG-V1-HW=256x384.uff", "image", {"outputs/conf", "outputs/paf"} };
 
     // * Input video.
     auto capture = cv::VideoCapture("../data/media/video.avi");
@@ -106,7 +106,7 @@ int main() {
     using namespace hyperpose;
 
     const cv::Size network_resolution{384, 256};
-    const dnn::uff uff_model{ "../data/models/hao28-600000-256x384.uff", "image", {"outputs/conf", "outputs/paf"} };
+    const dnn::uff uff_model{ "../data/models/TinyVGG-V1-HW=256x384.uff", "image", {"outputs/conf", "outputs/paf"} };
 
     // * Input video.
     auto capture = cv::VideoCapture("../data/media/video.avi");

docs/markdown/install/prediction.md

@@ -6,10 +6,13 @@
 * CMake 3.5+ 
 * Third-Party
     * OpenCV3.2+.
-    * [CUDA 10](https://developer.nvidia.com/cuda-downloads), [TensorRT 7](https://docs.nvidia.com/deeplearning/tensorrt/archives/tensorrt_304/tensorrt-install-guide/index.html).
+    * [CUDA 10.2](https://developer.nvidia.com/cuda-downloads), [CuDNN 7.6.5](https://docs.nvidia.com/deeplearning/cudnn/install-guide/index.html), [TensorRT 7.0](https://docs.nvidia.com/deeplearning/tensorrt/install-guide/index.html). (For Linux users, [Debian Installation](https://docs.nvidia.com/deeplearning/tensorrt/install-guide/index.html#installing-debian) is highly recommended.)
     * gFlags(optional, for examples/tests)
 
-> Older versions of the packages may also work but not tested.
+> Other versions of the packages may also work but not tested.
+
+> Different TensorRT version requires specific CUDA and CuDNN version. For specific CUDA and CuDNN requirements of TensorRT7, please refer to [this](https://docs.nvidia.com/deeplearning/tensorrt/support-matrix/index.html#platform-matrix).
+> Also, for Ubuntu 18.04 users, this [3rd party blog](https://ddkang.github.io/2020/01/02/installing-tensorrt.html) may help you. 
 
 ## Build On Ubuntu 18.04
 
@@ -18,7 +21,7 @@
 sudo apt -y install cmake libopencv-dev  
 # You may also install OpenCV from source to get best performance.
 
-# >>> Install CUDA/TensorRT
+# >>> Install CUDA/CuDNN/TensorRT
 
 # >>> Build gFlags(Optional) from source. Install it if you want to run the examples.
 wget https://github.com/gflags/gflags/archive/v2.2.2.zip
@@ -32,7 +35,7 @@ sudo make install
 git clone https://github.com/tensorlayer/hyperpose.git
 cd hyperpose
 mkdir build && cd build
-cmake .. -DCMAKE_BUILD_TYPE=RELEASE && make -j$(nproc)
+cmake .. -DCMAKE_BUILD_TYPE=Release && make -j
 ```
 
 ## Build with User Codes

docs/markdown/performance/prediction.md

@@ -13,8 +13,6 @@
 > **Environment**: [email protected], GPU@1070Ti, CPU@i7(12 logic cores). 
 >
 > **Tested Video Source**: Crazy Updown Funk(resolution@640x360, frame_count@7458, source@[YouTube](https://www.youtube.com/watch?v=2DiQUX11YaY))
->
-> **Availability**: All model above are available [here](https://github.com/tensorlayer/pretrained-models/tree/master/models/hyperpose). 
 
 > OpenPose performance is not tested with batch processing as it seems not to be implemented. (see [here](https://github.com/CMU-Perceptual-Computing-Lab/openpose/issues/100)) 
 

docs/markdown/performance/supports.md

@@ -16,11 +16,8 @@
 ### Supported Post-Processing Methods
 
 - Part Association Field(PAF)
-- Pose Proposal Networks(Coming Soon)
+- Pose Proposal Networks
 
 ### Released Prediction Models
 
-- [Tiny VGG](https://github.com/tensorlayer/pretrained-models/blob/master/models/hyperpose/hao28-600000-256x384.uff)
-- [OpenPose-COCO](https://github.com/tensorlayer/pretrained-models/blob/master/models/hyperpose/openpose_coco.onnx)
-- [OpenPose-Thin](https://github.com/tensorlayer/pretrained-models/blob/master/models/hyperpose/openpose_thin.onnx)
-- [ResNet18(for PAF)](https://github.com/tensorlayer/pretrained-models/blob/master/models/hyperpose/lopps_resnet50.onnx)
+We released the models on [Google Drive](TinyVGG-V1-HW=256x384.uff). `.onnx` and `.uff` files are for inference.

docs/markdown/quick_start/prediction.md

@@ -30,9 +30,10 @@ sh scripts/download-openpose-thin-model.sh      # ~20  MB
 sh scripts/download-tinyvgg-model.sh            # ~30  MB
 sh scripts/download-openpose-res50-model.sh     # ~45  MB
 sh scripts/download-openpose-coco-model.sh      # ~200 MB
+sh scripts/download-ppn-res50-model.sh          # ~50  MB (PoseProposal Algorithm)
 ```
 
-> You can download them manually to `${HyperPose}/data/models/` via [LINK](https://github.com/tensorlayer/pretrained-models/tree/master/models/hyperpose) **if the network is not working**.
+> You can download them manually to `${HyperPose}/data/models/` via [LINK](https://drive.google.com/drive/folders/1w9EjMkrjxOmMw3Rf6fXXkiv_ge7M99jR?usp=sharing) **if the network is not working**.
 
 ## Predict a sequence of images
 
@@ -46,17 +47,23 @@ sh scripts/download-openpose-coco-model.sh      # ~200 MB
 # Take images in ../data/media as a big batch and do prediction.
 
 ./example.operator_api_batched_images_paf
-# The same as: `./example.operator_api_batched_images_paf --model_file ../data/models/hao28-600000-256x384.uff --input_folder ../data/media --input_width 384 --input_height 256`
+# The same as: `./example.operator_api_batched_images_paf --model_file ../data/models/TinyVGG-V1-HW=256x384.uff --input_folder ../data/media --input_width 384 --input_height 256`
 ```
 
 The output images will be in the build folder.
 
 ### Using a precise model
 
 ```bash
-./example.operator_api_batched_images_paf --model_file ../data/models/openpose_thin.onnx --input_width 432 --input_height 368 
+./example.operator_api_batched_images_paf --model_file ../data/models/openpose-thin-V2-HW=368x432.onnx --input_width 432 --input_height 368 
 
-./example.operator_api_batched_images_paf --model_file ../data/models/openpose_coco.onnx --input_width 656 --input_height 368 
+./example.operator_api_batched_images_paf --model_file ../data/models/openpose-coco-V2-HW=368x656.onnx --input_width 656 --input_height 368 
+```
+
+### Use PoseProposal model
+
+```bash
+./example.operator_api_batched_images_pose_proposal --model_file ../data/models/ppn-resnet50-V2-HW=384x384.onnx --input_width 368 --input_height 368
 ```
 
 ### Convert models into TensorRT Engine Protobuf format

docs/markdown/tutorial/faq.md

@@ -24,7 +24,7 @@ Refer to [here](https://www.learnopencv.com/tag/install/).
 
 Download them manually:
 
-- All prediction models are available [here](https://github.com/tensorlayer/pretrained-models/tree/master/models/hyperpose).
+- All prediction models are available on [Google Drive](https://drive.google.com/drive/folders/1w9EjMkrjxOmMw3Rf6fXXkiv_ge7M99jR?usp=sharing).
 - The test data are taken from the [OpenPose Project](https://github.com/CMU-Perceptual-Computing-Lab/openpose/tree/master/examples/media).
 
 ## Training 
@@ -34,7 +34,7 @@ Download them manually:
 ### TensorRT Error?
 
 - See the `tensorrt.log`. (it contains more informations about logging and is located in where you execute the binary)
-- You may meet `ERROR: Tensor image cannot be both input and output` when using the `hao28-600000-256x384.uff` model. And just ignore it.
+- You may meet `ERROR: Tensor image cannot be both input and output` when using the `TinyVGG-V1-HW=256x384.uff` model. And just ignore it.
 
 ### Performance?
 

docs/markdown/tutorial/prediction.md

@@ -133,7 +133,7 @@ int main() {
     using namespace hyperpose;
 
     const cv::Size network_resolution{384, 256};
-    const dnn::uff uff_model{ "../data/models/hao28-600000-256x384.uff", "image", {"outputs/conf", "outputs/paf"} };
+    const dnn::uff uff_model{ "../data/models/TinyVGG-V1-HW=256x384.uff", "image", {"outputs/conf", "outputs/paf"} };
 
     // * Input video.
     auto capture = cv::VideoCapture("../data/media/video.avi");