[Model] add style transfer model #922

rainyfly · 2022-12-20T07:55:21Z

PR types(PR类型)

Model

Description

Add style transfer models in paddlehub into fastdeploy model repository.

…y/FastDeploy into add_transfer_style_model

fastdeploy/vision/generation/contrib/animegan.cc

…into add_transfer_style_model

heliqi

LGTM

DefTruth · 2022-12-30T03:26:20Z

fastdeploy/vision/generation/contrib/animegan.cc

+           const ModelFormat& model_format) {
+
+  valid_cpu_backends = {Backend::PDINFER};
+  valid_gpu_backends = {Backend::PDINFER, Backend::TRT};


我看到GPU后端支持TRT，那理论上是可以转通过paddle2onnx转onnx的，所以ORT和openvino的后端是否也应该对应测试下？如果支持的话，一般来说性能会不错。

GPU后端暂时不支持TRT，会报错，已经删除，报错原因已经提了icafe卡片，凡是需要转onnx的目前都会报错。

DefTruth · 2022-12-30T03:27:36Z

examples/vision/generation/anemigan/README.md

+|[animegan_v2_paprika_74](https://www.paddlepaddle.org.cn/hubdetail?name=animegan_v2_paprika_74&en_category=GANs)|可将输入的图像转换成今敏红辣椒动漫风格，模型权重转换自AnimeGAN V2官方开源项目|paddle|
+|[animegan_v2_paprika_98](https://www.paddlepaddle.org.cn/hubdetail?name=animegan_v2_paprika_98&en_category=GANs)|可将输入的图像转换成今敏红辣椒动漫风格，模型权重转换自AnimeGAN V2官方开源项目|paddle|
+
+## FastDeploy paddle backend部署和hub速度对比(ips)


这里速度是越高越好吗(ips)？可以加个注释，避免用户看到后产生疑惑

DefTruth · 2022-12-30T03:31:19Z

fastdeploy/vision/generation/contrib/postprocessor.cc

+  results->resize(shape[0]);
+  float* infer_result_data = reinterpret_cast<float*>(output_tensor.Data());
+  for(size_t i = 0; i < results->size(); ++i){
+  float* data = new float[shape[1]*shape[2]*3];


这个new出来的内存没有被释放，会导致内存泄漏。这里建议改用std::vector，初始化时指定数据个数，避免内存泄露

DefTruth · 2022-12-30T03:33:18Z

fastdeploy/vision/generation/contrib/postprocessor.cc

+  Mat fd_image = WrapMat(res);
+  BGR2RGB::Run(&fd_image);
+  res = *(fd_image.GetOpenCVMat());
+  res.copyTo(results->at(i));


另外，这个循环内的代码看起来似乎没有进行缩进，请检查下代码风格

DefTruth · 2022-12-30T03:50:56Z

fastdeploy/vision/generation/contrib/postprocessor.cc

+  Mat result_mat = Mat::Create(shape[1], shape[2], 3, FDDataType::FP32, data);
+  std::vector<float> mean{127.5f, 127.5f, 127.5f};
+  std::vector<float> std{127.5f, 127.5f, 127.5f};
+  Convert::Run(&result_mat, mean, std);


这里可以直接用Mat来创建一个零拷贝引用fp32数据，然后直接转uint8即可。也不需要创建一个中间的data，这样会导致额外的内存创建和拷贝。

Mat result_mat = Mat::Create(shape[1], shape[2], 3, FDDataType::FP32, infer_result_data+i*size);

DefTruth · 2022-12-30T03:55:55Z

fastdeploy/vision/vision_pybind.cc

 void BindKeyPointDetection(pybind11::module& m);
 void BindHeadPose(pybind11::module& m);
 void BindSR(pybind11::module& m);
+void BindGenerationModel(pybind11::module& m);


这里改成BindGenertion吧，保持和其他模块写法的统一

DefTruth · 2022-12-30T03:56:35Z

python/fastdeploy/vision/generation/contrib/anemigan.py

+        :param runtime_option: (fastdeploy.RuntimeOption)RuntimeOption for inference this model, if it's None, will use the default backend on CPU
+        :param model_format: (fastdeploy.ModelForamt)Model format of the loaded model
+        """
+        # 调用基函数进行backend_option的初始化


删除中文注释或修改成英文，其余同理

* [Backend] Update paddle inference version (PaddlePaddle#990) 2.4-dev3 -> 2.4-dev4 * Add uie benchmark * fix trt dy shape * update uie benchmark * Update uie benchmark output * Fix cpu_num_thread->cpu_num_threads * Update backend name * [Model] Update PPSeg Preprocess (PaddlePaddle#1007) * 更新PPSeg pybind and python * 更新PPSeg pybind and python * [Model] Update PPDet Preprocess (PaddlePaddle#1006) * 更新导航文档 * 更新导航文档 * 更新导航文档 * 更新导航文档 * 更新PPDet PreProcess * 更新PPDet PreProcess * 更新PPDet pybind and python * 更新 * 更新ppdet * [Other]Update Paddle Lite for RV1126 (PaddlePaddle#1013) update lite link * [Other] Remove TRT static libs in package (PaddlePaddle#1011) * remove duplicated and useless libs * use os system to run ldd * remove filter libs by ldd * [Serving]update ocr model.py from np.object to np.object_ (PaddlePaddle#1017) * [Serving]update ocr model.py from np.object to np.object_ * Update model.py * [Bug Fix] Fix build with Paddle Inference on Jetson (PaddlePaddle#1019) Fix build with Paddle Inference on Jetson * Update README.md * [Serving]update np.object to np.object_ (PaddlePaddle#1021) np.object to np.object_ * Update README.md * fresh doc version release/1.0.2 (PaddlePaddle#1023) fresh doc version * [Other] Add a interface to get all pretrained models available from hub model server (PaddlePaddle#1022) add get model list * [Doc]Revise one wording (PaddlePaddle#1028) * 第一次提交 * 补充一处漏翻译 * deleted: docs/en/quantize.md * Update one translation * Update en version * Update one translation in code * Standardize one writing * Standardize one writing * Update some en version * Fix a grammer problem * Update en version for api/vision result * Merge branch 'develop' of https://github.com/charl-u/FastDeploy into develop * Checkout the link in README in vision_results/ to the en documents * Modify a title * Add link to serving/docs/ * Finish translation of demo.md * Update english version of serving/docs/ * Update title of readme * Update some links * Modify a title * Update some links * Update en version of java android README * Modify some titles * Modify some titles * Modify some titles * modify article to document * [Model] add style transfer model (PaddlePaddle#922) * add style transfer model * add examples for generation model * add unit test * add speed comparison * add speed comparison * add variable for constant * add preprocessor and postprocessor * add preprocessor and postprocessor * fix * fix according to review Co-authored-by: DefTruth <[email protected]> Co-authored-by: Jack Zhou <[email protected]> Co-authored-by: Zheng-Bicheng <[email protected]> Co-authored-by: yeliang2258 <[email protected]> Co-authored-by: Wang Xinyu <[email protected]> Co-authored-by: heliqi <[email protected]> Co-authored-by: Jason <[email protected]> Co-authored-by: leiqing <[email protected]> Co-authored-by: Zeref996 <[email protected]> Co-authored-by: chenjian <[email protected]> Co-authored-by: charl-u <[email protected]>

rainyfly added 2 commits December 20, 2022 07:52

add style transfer model

935f045

add examples for generation model

6c2f5dc

rainyfly changed the title ~~[Model] add style transfer model (do not merge, examples to be added)~~ [Model] add style transfer model Dec 22, 2022

DefTruth self-requested a review December 22, 2022 11:15

Merge branch 'develop' into add_transfer_style_model

10280dc

jiangjiajun requested a review from heliqi December 26, 2022 07:17

rainyfly added 4 commits December 26, 2022 15:29

add unit test

ceca029

Merge branch 'add_transfer_style_model' of https://github.com/rainyfl…

370b74b

…y/FastDeploy into add_transfer_style_model

add speed comparison

1cc61e3

add speed comparison

82f34a8

heliqi reviewed Dec 27, 2022

View reviewed changes

fastdeploy/vision/generation/contrib/animegan.cc Outdated Show resolved Hide resolved

fastdeploy/vision/generation/contrib/animegan.cc Outdated Show resolved Hide resolved

rainyfly added 5 commits December 28, 2022 02:27

add variable for constant

ae9796a

Merge branch 'develop' of https://github.com/PaddlePaddle/FastDeploy …

56e47ff

…into add_transfer_style_model

add preprocessor and postprocessor

6fff916

add preprocessor and postprocessor

72869e9

fix

a633349

heliqi approved these changes Dec 29, 2022

View reviewed changes

DefTruth requested changes Dec 30, 2022

View reviewed changes

fix according to review

13cf601

DefTruth approved these changes Dec 30, 2022

View reviewed changes

DefTruth requested a review from jiangjiajun December 30, 2022 08:59

Merge branch 'develop' into add_transfer_style_model

a84d718

jiangjiajun approved these changes Jan 3, 2023

View reviewed changes

jiangjiajun merged commit 87bcb5d into PaddlePaddle:develop Jan 3, 2023

[Model] add style transfer model #922

[Model] add style transfer model #922

Uh oh!

Conversation

rainyfly commented Dec 20, 2022

PR types(PR类型)

Description

Uh oh!

Uh oh!

Uh oh!

heliqi left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants