[Model] Support Insightface model inferenceing on RKNPU

docs/cn/faq/rknpu2/rknpu2.md

@@ -13,14 +13,22 @@ ONNX模型不能直接调用RK芯片中的NPU进行运算，需要把ONNX模型
 * ARM CPU使用ONNX框架进行测试
 * NPU均使用单核进行测试
 
-| 任务场景           | 模型                                                                                       | 模型版本(表示已经测试的版本)          | ARM CPU/RKNN速度(ms) |
-|----------------|------------------------------------------------------------------------------------------|--------------------------|--------------------|
-| Detection      | [Picodet](../../../../examples/vision/detection/paddledetection/rknpu2/README.md)        | Picodet-s                | 162/112            |
-| Detection      | [RKYOLOV5](../../../../examples/vision/detection/rkyolo/README.md)                       | YOLOV5-S-Relu(int8)      | -/57               |
-| Detection      | [RKYOLOX](../../../../examples/vision/detection/rkyolo/README.md)                        | -                        | -/-                |
-| Detection      | [RKYOLOV7](../../../../examples/vision/detection/rkyolo/README.md)                       | -                        | -/-                |
-| Segmentation   | [Unet](../../../../examples/vision/segmentation/paddleseg/rknpu2/README.md)              | Unet-cityscapes          | -/-                |
-| Segmentation   | [PP-HumanSegV2Lite](../../../../examples/vision/segmentation/paddleseg/rknpu2/README.md) | portrait(int8)           | 133/43             |
-| Segmentation   | [PP-HumanSegV2Lite](../../../../examples/vision/segmentation/paddleseg/rknpu2/README.md) | human(int8)              | 133/43             |
-| Face Detection | [SCRFD](../../../../examples/vision/facedet/scrfd/rknpu2/README.md)                      | SCRFD-2.5G-kps-640(int8) | 108/42             |
-| Classification | [ResNet](../../../../examples/vision/classification/paddleclas/rknpu2/README.md)         | ResNet50_vd              | -/33               |
+| 任务场景                 | 模型                                                                                       | 模型版本(表示已经测试的版本)          | ARM CPU/RKNN速度(ms) |
+|----------------------|------------------------------------------------------------------------------------------|--------------------------|--------------------|
+| Detection            | [Picodet](../../../../examples/vision/detection/paddledetection/rknpu2/README.md)        | Picodet-s                | 162/112            |
+| Detection            | [RKYOLOV5](../../../../examples/vision/detection/rkyolo/README.md)                       | YOLOV5-S-Relu(int8)      | -/57               |
+| Detection            | [RKYOLOX](../../../../examples/vision/detection/rkyolo/README.md)                        | -                        | -/-                |
+| Detection            | [RKYOLOV7](../../../../examples/vision/detection/rkyolo/README.md)                       | -                        | -/-                |
+| Segmentation         | [Unet](../../../../examples/vision/segmentation/paddleseg/rknpu2/README.md)              | Unet-cityscapes          | -/-                |
+| Segmentation         | [PP-HumanSegV2Lite](../../../../examples/vision/segmentation/paddleseg/rknpu2/README.md) | portrait(int8)           | 133/43             |
+| Segmentation         | [PP-HumanSegV2Lite](../../../../examples/vision/segmentation/paddleseg/rknpu2/README.md) | human(int8)              | 133/43             |
+| Face Detection       | [SCRFD](../../../../examples/vision/facedet/scrfd/rknpu2/README.md)                      | SCRFD-2.5G-kps-640(int8) | 108/42             |
+| Face FaceRecognition | [InsightFace](../../../../examples/vision/faceid/insightface/rknpu2/README_CN.md)        | ms1mv3_arcface_r18(int8) | 81/12              |
+| Classification       | [ResNet](../../../../examples/vision/classification/paddleclas/rknpu2/README.md)         | ResNet50_vd              | -/33               |
+
+## 预编译库下载
+
+为了方便大家进行开发，这里提供1.0.2版本的FastDeploy给大家使用
+
+- [FastDeploy RK356X c++ SDK](https://bj.bcebos.com/fastdeploy/release/cpp/fastdeploy-linux-aarch64-rk356X-1.0.2.tgz)
+- [FastDeploy RK3588 c++ SDK](https://bj.bcebos.com/fastdeploy/release/cpp/fastdeploy-linux-aarch64-rk3588-1.0.2.tgz)

examples/vision/faceid/insightface/cpp/README_CN.md

@@ -101,7 +101,7 @@ VPL模型加载和初始化，其中model_file为导出的ONNX模型格式。
 #### Predict函数
 
 > ```c++
-> ArcFace::Predict(cv::Mat* im, FaceRecognitionResult* result)
+> ArcFace::Predict(const cv::Mat& im, FaceRecognitionResult* result)
 > ```
 >
 > 模型预测接口，输入图像直接输出检测结果。
@@ -121,8 +121,6 @@ VPL模型加载和初始化，其中model_file为导出的ONNX模型格式。
       通过InsightFaceRecognitionPreprocessor::SetAlpha(std::vector<float>& alpha)来进行修改
 > > * **beta**(vector&lt;float&gt;): 预处理归一化的beta值，计算公式为`x'=x*alpha+beta`，beta默认为[-1.f, -1.f, -1.f],
       通过InsightFaceRecognitionPreprocessor::SetBeta(std::vector<float>& beta)来进行修改
-> > * **permute**(bool): 预处理是否将BGR转换成RGB，默认true,
-      通过InsightFaceRecognitionPreprocessor::SetPermute(bool permute)来进行修改
 
 #### InsightFaceRecognitionPostprocessor成员变量(后处理参数)
 > > * **l2_normalize**(bool): 输出人脸向量之前是否执行l2归一化，默认false,

examples/vision/faceid/insightface/python/README_CN.md

@@ -100,7 +100,6 @@ ArcFace模型加载和初始化，其中model_file为导出的ONNX模型格式
 > > * **size**(list[int]): 通过此参数修改预处理过程中resize的大小，包含两个整型元素，表示[width, height], 默认值为[112, 112]
 > > * **alpha**(list[float]): 预处理归一化的alpha值，计算公式为`x'=x*alpha+beta`，alpha默认为[1. / 127.5, 1.f / 127.5, 1. / 127.5]
 > > * **beta**(list[float]): 预处理归一化的beta值，计算公式为`x'=x*alpha+beta`，beta默认为[-1.f, -1.f, -1.f]
-> > * **swap_rb**(bool): 预处理是否将BGR转换成RGB，默认True
 
 #### AdaFacePostprocessor的成员变量
 以下变量为AdaFacePostprocessor的成员变量

examples/vision/faceid/insightface/python/infer_arcface.py

@@ -3,7 +3,6 @@
 import numpy as np
 
 
-# 余弦相似度
 def cosine_similarity(a, b):
     a = np.array(a)
     b = np.array(b)
@@ -56,32 +55,24 @@ def build_option(args):
 
 args = parse_arguments()
 
-# 配置runtime，加载模型
 runtime_option = build_option(args)
 model = fd.vision.faceid.ArcFace(args.model, runtime_option=runtime_option)
 
-# 加载图片
 face0 = cv2.imread(args.face)  # 0,1 同一个人
 face1 = cv2.imread(args.face_positive)
 face2 = cv2.imread(args.face_negative)  # 0,2 不同的人
 
-# 设置 l2 normalize
-model.postprocessor.l2_normalize = True
-
-# 预测图片检测结果
 result0 = model.predict(face0)
 result1 = model.predict(face1)
 result2 = model.predict(face2)
 
-# 计算余弦相似度
 embedding0 = result0.embedding
 embedding1 = result1.embedding
 embedding2 = result2.embedding
 
 cosine01 = cosine_similarity(embedding0, embedding1)
 cosine02 = cosine_similarity(embedding0, embedding2)
 
-# 打印结果
 print(result0, end="")
 print(result1, end="")
 print(result2, end="")

examples/vision/faceid/insightface/python/infer_cosface.py

@@ -3,7 +3,6 @@
 import numpy as np
 
 
-# 余弦相似度
 def cosine_similarity(a, b):
     a = np.array(a)
     b = np.array(b)
@@ -56,32 +55,24 @@ def build_option(args):
 
 args = parse_arguments()
 
-# 配置runtime，加载模型
 runtime_option = build_option(args)
 model = fd.vision.faceid.CosFace(args.model, runtime_option=runtime_option)
 
-# 加载图片
-face0 = cv2.imread(args.face)  # 0,1 同一个人
+face0 = cv2.imread(args.face)
 face1 = cv2.imread(args.face_positive)
-face2 = cv2.imread(args.face_negative)  # 0,2 不同的人
+face2 = cv2.imread(args.face_negative)
 
-# 设置 l2 normalize
-model.postprocessor.l2_normalize = True
-
-# 预测图片检测结果
 result0 = model.predict(face0)
 result1 = model.predict(face1)
 result2 = model.predict(face2)
 
-# 计算余弦相似度
 embedding0 = result0.embedding
 embedding1 = result1.embedding
 embedding2 = result2.embedding
 
 cosine01 = cosine_similarity(embedding0, embedding1)
 cosine02 = cosine_similarity(embedding0, embedding2)
 
-# 打印结果
 print(result0, end="")
 print(result1, end="")
 print(result2, end="")

examples/vision/faceid/insightface/python/infer_partial_fc.py

@@ -3,7 +3,6 @@
 import numpy as np
 
 
-# 余弦相似度
 def cosine_similarity(a, b):
     a = np.array(a)
     b = np.array(b)
@@ -56,32 +55,25 @@ def build_option(args):
 
 args = parse_arguments()
 
-# 配置runtime，加载模型
 runtime_option = build_option(args)
 model = fd.vision.faceid.PartialFC(args.model, runtime_option=runtime_option)
 
 # 加载图片
-face0 = cv2.imread(args.face)  # 0,1 同一个人
+face0 = cv2.imread(args.face)
 face1 = cv2.imread(args.face_positive)
-face2 = cv2.imread(args.face_negative)  # 0,2 不同的人
+face2 = cv2.imread(args.face_negative)
 
-# 设置 l2 normalize
-model.postprocessor.l2_normalize = True
-
-# 预测图片检测结果
 result0 = model.predict(face0)
 result1 = model.predict(face1)
 result2 = model.predict(face2)
 
-# 计算余弦相似度
 embedding0 = result0.embedding
 embedding1 = result1.embedding
 embedding2 = result2.embedding
 
 cosine01 = cosine_similarity(embedding0, embedding1)
 cosine02 = cosine_similarity(embedding0, embedding2)
 
-# 打印结果
 print(result0, end="")
 print(result1, end="")
 print(result2, end="")

examples/vision/faceid/insightface/python/infer_vpl.py

@@ -3,7 +3,6 @@
 import numpy as np
 
 
-# 余弦相似度
 def cosine_similarity(a, b):
     a = np.array(a)
     b = np.array(b)
@@ -56,32 +55,24 @@ def build_option(args):
 
 args = parse_arguments()
 
-# 配置runtime，加载模型
 runtime_option = build_option(args)
 model = fd.vision.faceid.VPL(args.model, runtime_option=runtime_option)
 
-# 加载图片
 face0 = cv2.imread(args.face)  # 0,1 同一个人
 face1 = cv2.imread(args.face_positive)
 face2 = cv2.imread(args.face_negative)  # 0,2 不同的人
 
-# 设置 l2 normalize
-model.postprocessor.l2_normalize = True
-
-# 预测图片检测结果
 result0 = model.predict(face0)
 result1 = model.predict(face1)
 result2 = model.predict(face2)
 
-# 计算余弦相似度
 embedding0 = result0.embedding
 embedding1 = result1.embedding
 embedding2 = result2.embedding
 
 cosine01 = cosine_similarity(embedding0, embedding1)
 cosine02 = cosine_similarity(embedding0, embedding2)
 
-# 打印结果
 print(result0, end="")
 print(result1, end="")
 print(result2, end="")

examples/vision/faceid/insightface/rknpu2/README.md

@@ -0,0 +1,54 @@
+[English](README.md) | 简体中文
+# InsightFace RKNPU准备部署模型
+
+本教程提供InsightFace模型在RKNPU2环境下的部署，模型的详细介绍已经ONNX模型的下载请查看[模型介绍文档](../README.md)。
+
+## 支持模型列表
+目前FastDeploy支持如下模型的部署
+- ArcFace
+- CosFace
+- PartialFC
+- VPL
+
+## 下载预训练ONNX模型
+
+为了方便开发者的测试，下面提供了InsightFace导出的各系列模型，开发者可直接下载使用。（下表中模型的精度来源于源官方库）其中精度指标来源于InsightFace中对各模型的介绍，详情各参考InsightFace中的说明
+
+| 模型                                                                                         | 大小    | 精度 (AgeDB_30) |
+|:-------------------------------------------------------------------------------------------|:------|:--------------|
+| [CosFace-r18](https://bj.bcebos.com/paddlehub/fastdeploy/glint360k_cosface_r18.onnx)       | 92MB  | 97.7          |
+| [CosFace-r34](https://bj.bcebos.com/paddlehub/fastdeploy/glint360k_cosface_r34.onnx)       | 131MB | 98.3          |
+| [CosFace-r50](https://bj.bcebos.com/paddlehub/fastdeploy/glint360k_cosface_r50.onnx)       | 167MB | 98.3          |
+| [CosFace-r100](https://bj.bcebos.com/paddlehub/fastdeploy/glint360k_cosface_r100.onnx)     | 249MB | 98.4          |
+| [ArcFace-r18](https://bj.bcebos.com/paddlehub/fastdeploy/ms1mv3_arcface_r18.onnx)          | 92MB  | 97.7          |
+| [ArcFace-r34](https://bj.bcebos.com/paddlehub/fastdeploy/ms1mv3_arcface_r34.onnx)          | 131MB | 98.1          |
+| [ArcFace-r50](https://bj.bcebos.com/paddlehub/fastdeploy/ms1mv3_arcface_r50.onnx)          | 167MB | -             |
+| [ArcFace-r100](https://bj.bcebos.com/paddlehub/fastdeploy/ms1mv3_arcface_r100.onnx)        | 249MB | 98.4          |
+| [ArcFace-r100_lr0.1](https://bj.bcebos.com/paddlehub/fastdeploy/ms1mv3_r100_lr01.onnx)     | 249MB | 98.4          |
+| [PartialFC-r34](https://bj.bcebos.com/paddlehub/fastdeploy/partial_fc_glint360k_r50.onnx)  | 167MB | -             |
+| [PartialFC-r50](https://bj.bcebos.com/paddlehub/fastdeploy/partial_fc_glint360k_r100.onnx) | 249MB | -             |
+
+
+## 转换为RKNPU模型
+
+```bash
+wget https://bj.bcebos.com/paddlehub/fastdeploy/ms1mv3_arcface_r18.onnx
+
+python -m paddle2onnx.optimize --input_model ./ms1mv3_arcface_r18/ms1mv3_arcface_r18.onnx \
+                               --output_model ./ms1mv3_arcface_r18/ms1mv3_arcface_r18.onnx \
+                               --input_shape_dict "{'data':[1,3,112,112]}"
+
+python  /Path/To/FastDeploy/tools/rknpu2/export.py \
+        --config_path tools/rknpu2/config/arcface_unquantized.yaml \
+        --target_platform rk3588
+```
+
+## 详细部署文档
+
+- [Python部署](python)
+- [C++部署](cpp)
+
+
+## 版本说明
+
+- 本版本文档和代码基于[InsightFace CommitID:babb9a5](https://github.com/deepinsight/insightface/commit/babb9a5) 编写

examples/vision/faceid/insightface/rknpu2/cpp/CMakeLists.txt

@@ -0,0 +1,11 @@
+PROJECT(infer_demo C CXX)
+CMAKE_MINIMUM_REQUIRED (VERSION 3.10)
+
+option(FASTDEPLOY_INSTALL_DIR "Path of downloaded fastdeploy sdk.")
+
+include(${FASTDEPLOY_INSTALL_DIR}/FastDeploy.cmake)
+
+include_directories(${FASTDEPLOY_INCS})
+
+add_executable(infer_arcface_demo ${PROJECT_SOURCE_DIR}/infer_arcface.cc)
+target_link_libraries(infer_arcface_demo ${FASTDEPLOY_LIBS})