[Backend] add sophgo backend #1015
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| CMAKE_MINIMUM_REQUIRED (VERSION 3.10) | ||
|
|
||
| find_package(libsophon REQUIRED) | ||
| message(${LIBSOPHON_LIB_DIRS}) | ||
| include_directories(${LIBSOPHON_INCLUDE_DIRS}) | ||
| message(${LIBSOPHON_LIB_DIRS}) | ||
| set(SOPHGO_RT_LIB ${LIBSOPHON_LIB_DIRS}/libbmrt.so) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,78 @@ | ||
| # SOPHGO 部署库编译 | ||
|
|
||
| ## SOPHGO 环境准备 | ||
| SOPHGO支持linux下进行编译,系统为Debian/Ubuntu | ||
| 安装包由三个文件构成 | ||
| - [sophon-driver\_0.4.2\_$arch.deb](http://219.142.246.77:65000/sharing/KWqbmEcKp) | ||
| - [sophon-libsophon\_0.4.2\_$arch.deb](http://219.142.246.77:65000/sharing/PlvlBXhWY) | ||
| - [sophon-libsophon-dev\_0.4.2\_$arch.deb](http://219.142.246.77:65000/sharing/zTErLlpS7) | ||
|
|
||
| 其中“$arch”为当前机器的硬件架构，使用以下命令可以获取当前的服务器arch： | ||
| ```shell | ||
| uname -m | ||
| ``` | ||
| 通常x86_64 机器对应的硬件架构为amd64,arm64 机器对应的硬件架构为 arm64: | ||
| ```text | ||
| - sophon-driver_0.4.2_$arch.deb | ||
| - sophon-libsophon_0.4.2_$arch.deb | ||
| - sophon-libsophon-dev_0.4.2_$arch.deb | ||
| ``` | ||
|
|
||
| 其中:sophon-driver 包含了 PCIe 加速卡驱动;sophon-libsophon 包含了运行时环境(库文 | ||
| 件、工具等);sophon-libsophon-dev 包含了开发环境(头文件等)。如果只是在部署环境上安 | ||
| 装,则不需要安装 sophon-libsophon-dev。 | ||
| 可以通过如下步骤安装: | ||
| ```shell | ||
| #安装依赖库,只需要执行一次: | ||
| sudo apt install dkms libncurses5 | ||
| #安装 libsophon: | ||
| sudo dpkg -i sophon-*.deb | ||
| #在终端执行如下命令,或者登出再登入当前用户后即可使用 bm-smi 等命令: | ||
| source /etc/profile | ||
| ``` | ||
| 安装位置为： | ||
| ```text | ||
| /opt/sophon/ | ||
| ├── driver-0.4.2 | ||
| ├── libsophon-0.4.2 | ||
| | ├──bin | ||
| | ├──data | ||
| | ├──include | ||
| | └──lib | ||
| └── libsophon-current->/opt/sophon/libsophon-0.4.2 | ||
| ``` | ||
|
|
||
| ## C++ SDK编译安装 | ||
| 搭建好编译环境之后，编译命令如下： | ||
| ```bash | ||
| # Download the latest source code | ||
| git clone https://github.com/PaddlePaddle/FastDeploy.git | ||
| cd FastDeploy | ||
| mkdir build && cd build | ||
|
|
||
| # CMake configuration with Ascend | ||
| cmake -DENABLE_SOPHGO_BACKEND=ON \ | ||
| -DCMAKE_INSTALL_PREFIX=${PWD}/fastdeploy-sophgo \ | ||
| -DENABLE_VISION=ON \ | ||
| .. | ||
|
|
||
| # Build FastDeploy Ascend C++ SDK | ||
| make -j8 | ||
| make install | ||
| ``` | ||
| 编译完成之后，会在当前的build目录下生成 fastdeploy-sophgo 目录，编译完成。 | ||
|
|
||
| ## Python FastDeploy 库编译 | ||
| 搭建好编译环境之后，编译命令如下： | ||
| ```bash | ||
| # Download the latest source code | ||
| git clone https://github.com/PaddlePaddle/FastDeploy.git | ||
| cd FastDeploy/python | ||
| export ENABLE_SOPHGO_BACKEND=ON | ||
| export ENABLE_VISION=ON | ||
|
|
||
| python setup.py build | ||
| python setup.py bdist_wheel | ||
|
|
||
| #编译完成后,请用户自行安装当前目录的dist文件夹内的whl包. | ||
| ``` | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,77 @@ | ||
|
|
||
| # How to Build SOPHGO Deployment Environment | ||
|
|
||
| ## SOPHGO Environment Preparation | ||
| SOPHGO supports compilation on linux, using Debian/Ubuntu as an example | ||
| The installation package consists of three files | ||
| - [sophon-driver\_0.4.2\_$arch.deb](http://219.142.246.77:65000/sharing/KWqbmEcKp) | ||
| - [sophon-libsophon\_0.4.2\_$arch.deb](http://219.142.246.77:65000/sharing/PlvlBXhWY) | ||
| - [sophon-libsophon-dev\_0.4.2\_$arch.deb](http://219.142.246.77:65000/sharing/zTErLlpS7) | ||
|
|
||
| $arch indicates the hardware architecture of the current machine. Run the following command to obtain the current server arch: | ||
| ```shell | ||
| uname -m | ||
| ``` | ||
| Generally, the hardware architecture of x86_64 machines is amd64, so the hardware architecture is arm64: | ||
| ```text | ||
| - sophon-driver_0.4.2_$arch.deb | ||
| - sophon-libsophon_0.4.2_$arch.deb | ||
| - sophon-libsophon-dev_0.4.2_$arch.deb | ||
| ``` | ||
|
|
||
| sophon-driver contains PCIe acceleration card drivers; sophon-libsophon contains the runtime environment (librarys, tools, etc); sophon-libsophon-dev contains the development environment (header files, etc.). If you install packages only on a deployment environment, you do not need to install sophon-libsophon-dev. | ||
| You can perform the following steps to install: | ||
| ```shell | ||
| #To install a dependency library, you only need to do this once: | ||
| sudo apt install dkms libncurses5 | ||
| #install libsophon: | ||
| sudo dpkg -i sophon-*.deb | ||
| #Run the following command on the terminal, log out and then log in the current user to use commands such as bm-smi: | ||
| source /etc/profile | ||
| ``` | ||
| The position of installation:： | ||
| ```text | ||
| /opt/sophon/ | ||
| ├── driver-0.4.2 | ||
| ├── libsophon-0.4.2 | ||
| | ├──bin | ||
| | ├──data | ||
| | ├──include | ||
| | └──lib | ||
| └── libsophon-current->/opt/sophon/libsophon-0.4.2 | ||
| ``` | ||
|
|
||
| ## How to Build and Install C++ SDK | ||
| After setting up the compilation environment, the compilation command is as follows： | ||
| ```bash | ||
| # Download the latest source code | ||
| git clone https://github.com/PaddlePaddle/FastDeploy.git | ||
| cd FastDeploy | ||
| mkdir build && cd build | ||
|
|
||
| # CMake configuration with Ascend | ||
| cmake -DENABLE_SOPHGO_BACKEND=ON \ | ||
| -DCMAKE_INSTALL_PREFIX=${PWD}/fastdeploy-sophgo \ | ||
| -DENABLE_VISION=ON \ | ||
| .. | ||
|
|
||
| # Build FastDeploy Ascend C++ SDK | ||
| make -j8 | ||
| make install | ||
| ``` | ||
| When the compilation is complete, the fastdeploy-sophgo directory is created in the current build directory, indicating that the FastDeploy library has been compiled. | ||
|
|
||
| ## Compiling Python FastDeploy Libraries | ||
| After setting up the compilation environment, the compilation command is as follows： | ||
| ```bash | ||
| # Download the latest source code | ||
| git clone https://github.com/PaddlePaddle/FastDeploy.git | ||
| cd FastDeploy/python | ||
| export ENABLE_SOPHGO_BACKEND=ON | ||
| export ENABLE_VISION=ON | ||
|
|
||
| python setup.py build | ||
| python setup.py bdist_wheel | ||
|
|
||
| #After the compilation is complete, please install the whl package in the dist folder of the current directory. | ||
| ``` |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,84 @@ | ||
| # PaddleDetection SOPHGO部署示例 | ||
|
|
||
| ## 支持模型列表 | ||
|
|
||
| 目前FastDeploy支持的如下模型的部署[ResNet系列模型](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.4/docs/zh_CN/models/ResNet_and_vd.md) | ||
|
|
||
| ## 准备ResNet部署模型以及转换模型 | ||
|
|
||
| SOPHGO-TPU部署模型前需要将Paddle模型转换成bmodel模型，具体步骤如下: | ||
| - Paddle动态图模型转换为ONNX模型，请参考[Paddle2ONNX模型转换](https://github.com/PaddlePaddle/Paddle2ONNX/tree/develop/model_zoo/classification) | ||
| - ONNX模型转换bmodel模型的过程，请参考[TPU-MLIR](https://github.com/sophgo/tpu-mlir)。 | ||
|
|
||
| ## 模型转换example | ||
|
|
||
| 下面以[ResNet50_vd](https://bj.bcebos.com/paddlehub/fastdeploy/ResNet50_vd_infer.tgz)为例子,教大家如何转换Paddle模型到SOPHGO-TPU模型。 | ||
|
|
||
| ## 导出ONNX模型 | ||
|
|
||
| ### 下载Paddle ResNet50_vd静态图模型并解压 | ||
| ```shell | ||
| wget https://bj.bcebos.com/paddlehub/fastdeploy/ResNet50_vd_infer.tgz | ||
| tar xvf ResNet50_vd_infer.tgz | ||
| ``` | ||
|
|
||
| ### 静态图转ONNX模型，注意，这里的save_file请和压缩包名对齐 | ||
| ```shell | ||
| paddle2onnx --model_dir ResNet50_vd_infer \ | ||
| --model_filename inference.pdmodel \ | ||
| --params_filename inference.pdiparams \ | ||
| --save_file ResNet50_vd_infer.onnx \ | ||
| --enable_dev_version True | ||
| ``` | ||
| ### 导出bmodel模型 | ||
|
|
||
| 以转化BM1684x的bmodel模型为例子，我们需要下载[TPU-MLIR](https://github.com/sophgo/tpu-mlir)工程，安装过程具体参见[TPU-MLIR文档](https://github.com/sophgo/tpu-mlir/blob/master/README.md)。 | ||
| ### 1. 安装 | ||
| ``` shell | ||
| docker pull sophgo/tpuc_dev:latest | ||
|
|
||
| # myname1234是一个示例，也可以设置其他名字 | ||
| docker run --privileged --name myname1234 -v $PWD:/workspace -it sophgo/tpuc_dev:latest | ||
|
|
||
| source ./envsetup.sh | ||
| ./build.sh | ||
| ``` | ||
|
|
||
| ### 2. ONNX模型转换为bmodel模型 | ||
| ``` shell | ||
| mkdir ResNet50_vd_infer && cd ResNet50_vd_infer | ||
|
|
||
| # 在该文件中放入测试图片，同时将上一步转换好的ResNet50_vd_infer.onnx放入该文件夹中 | ||
| cp -rf ${REGRESSION_PATH}/dataset/COCO2017 . | ||
| cp -rf ${REGRESSION_PATH}/image . | ||
| # 放入onnx模型文件ResNet50_vd_infer.onnx | ||
|
|
||
| mkdir workspace && cd workspace | ||
|
|
||
| # 将ONNX模型转换为mlir模型，其中参数--output_names可以通过NETRON查看 | ||
| model_transform.py \ | ||
| --model_name ResNet50_vd_infer \ | ||
| --model_def ../ResNet50_vd_infer.onnx \ | ||
| --input_shapes [[1,3,224,224]] \ | ||
| --mean 0.0,0.0,0.0 \ | ||
| --scale 0.0039216,0.0039216,0.0039216 \ | ||
| --keep_aspect_ratio \ | ||
| --pixel_format rgb \ | ||
| --output_names save_infer_model/scale_0.tmp_1 \ | ||
| --test_input ../image/dog.jpg \ | ||
| --test_result ResNet50_vd_infer_top_outputs.npz \ | ||
| --mlir ResNet50_vd_infer.mlir | ||
|
|
||
| # 将mlir模型转换为BM1684x的F32 bmodel模型 | ||
| model_deploy.py \ | ||
| --mlir ResNet50_vd_infer.mlir \ | ||
| --quantize F32 \ | ||
| --chip bm1684x \ | ||
| --test_input ResNet50_vd_infer_in_f32.npz \ | ||
| --test_reference ResNet50_vd_infer_top_outputs.npz \ | ||
| --model ResNet50_vd_infer_1684x_f32.bmodel | ||
| ``` | ||
| 最终获得可以在BM1684x上能够运行的bmodel模型ResNet50_vd_infer_1684x_f32.bmodel。如果需要进一步对模型进行加速，可以将ONNX模型转换为INT8 bmodel，具体步骤参见[TPU-MLIR文档](https://github.com/sophgo/tpu-mlir/blob/master/README.md)。 | ||
|
|
||
| ## 其他链接 | ||
| - [Cpp部署](./cpp) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,17 @@ | ||
| PROJECT(infer_demo C CXX) | ||
| CMAKE_MINIMUM_REQUIRED (VERSION 3.10) | ||
| # 指定下载解压后的fastdeploy库路径 | ||
| option(FASTDEPLOY_INSTALL_DIR "Path of downloaded fastdeploy sdk.") | ||
|
|
||
| set(ENABLE_LITE_BACKEND OFF) | ||
| #set(FDLIB ${FASTDEPLOY_INSTALL_DIR}) | ||
|
|
||
| include(${FASTDEPLOY_INSTALL_DIR}/FastDeploy.cmake) | ||
|
|
||
| # 添加FastDeploy依赖头文件 | ||
| include_directories(${FASTDEPLOY_INCS}) | ||
| include_directories(${FastDeploy_INCLUDE_DIRS}) | ||
|
|
||
| add_executable(infer_demo ${PROJECT_SOURCE_DIR}/infer.cc) | ||
| # 添加FastDeploy库依赖 | ||
| target_link_libraries(infer_demo ${FASTDEPLOY_LIBS}) | ||
|
|
||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Python不能支持么？