add tensorrt by Superjomn · Pull Request #9891 · PaddlePaddle/Paddle

Superjomn · 2018-04-13T05:11:06Z

This is a naive test for TensorRT library integration with Paddle.

…_tensorrt

dependency has been installed in docker image.

Superjomn · 2018-04-15T11:21:21Z

 option(WITH_AMD_GPU     "Compile PaddlePaddle with AMD GPU"             OFF)
 option(WITH_AVX         "Compile PaddlePaddle with AVX intrinsics"      ${AVX_FOUND})
 option(WITH_MKL         "Compile PaddlePaddle with MKL support."        ${AVX_FOUND})
+option(WITH_TENSORRT    "Compile PaddlePaddle with TensorRT support."   ON)


Will turn off this latter. Changing TeamCity config should more cautious.

luotao1 · 2018-04-16T02:59:16Z

 RUN curl -s -q https://glide.sh/get | sh

+# Install TensorRT
+RUN wget -qO- http://paddlepaddledeps.bj.bcebos.com/TensorRT-4.0.0.3.Ubuntu-16.04.4.x86_64-gnu.cuda-8.0.cudnn7.0.tar.gz | \


这里的tar.gz和官网下载的有所不同，只包含了include和lib包，目的是为了让包减少2/3的大小，从而节省下载时间。需要加comment说明一下。

根据这个包下载的，里面还有targets目录，该目录可以在打包的时候去掉。

TensorRT ├── include ├── lib └── targets

这里使用的NvInfer.h，是对原来的版本做了一点修改的，不然会报错。可以写一个issue说明下报错情况，然后在这里加一个comment。

luotao1 · 2018-04-16T03:00:29Z

 # version(1.7.1 for now), which causes building documentation failed.
-RUN pip install --upgrade pip && \
-    pip install -U wheel && \
+RUN pip install -U wheel && \


#9926 merge后，这里需要更新下。

luotao1 · 2018-04-16T03:27:56Z

 cc_library(dynamic_loader SRCS dynamic_loader.cc DEPS glog gflags enforce)

-list(APPEND CUDA_SRCS cublas.cc cudnn.cc curand.cc nccl.cc)
+list(APPEND CUDA_SRCS cublas.cc cudnn.cc curand.cc nccl.cc tensorrt.cc)


这里需要加编译选项来选择是否添加tensorrt.cc

luotao1 · 2018-04-16T03:33:01Z

+
+// Fix the dynload issue, the following two API are implemented in TensorRT's
+// header file, cannot load from the dynamic library. So create our own
+// implementation and directly trigger the method from the dynamic library.


58-60的注释需要更新下：

fix the dynload issue: 请问issue在哪儿？

API-》APIs

but can not loaded from

luotao1 · 2018-04-16T03:41:42Z


 if(WITH_GPU)
    include(cuda)
+    set(WITH_TENSORRT ON)


这句可以去掉。

luotao1 · 2018-04-16T03:44:57Z

+RUN wget -qO- http://paddlepaddledeps.bj.bcebos.com/TensorRT-4.0.0.3.Ubuntu-16.04.4.x86_64-gnu.cuda-8.0.cudnn7.0.tar.gz | \
+    tar -xz -C /usr/local && \
+    cp -rf /usr/local/TensorRT/include /usr/local && \
+    cp -rf /usr/local/TensorRT/lib /usr/local


缺少类似cudnn.cmake这样的tensorrt.cmake，用户无法用自定义路径的安装形式，可以之后的PR补充。

目前直接安装在/usr/local/include和/usr/local/lib里，应该像cuda和go一样，有一个自己的目录，可以和1一起之后的PR改进。

现在直接复制到 /usr下了，可以后续pr改下

luotao1 · 2018-04-16T04:18:46Z

编译成功，运行单测存在：

75: unknown file: Failure
75: C++ exception with description "Failed to find dynamic library: libnvinfer.so ( libnvinfer.so: cannot open shared object file: No such file or directory ) 
75:  Please specify its path correctly using following ways: 
75:  Method. set environment variable LD_LIBRARY_PATH on Linux or DYLD_LIBRARY_PATH on Mac OS. 
75:  For instance, issue command: export LD_LIBRARY_PATH=... 
75:  Note: After Mac OS 10.11, using the DYLD_LIBRARY_PATH is impossible unless System Integrity Protection (SIP) is disabled. at [/Paddle/paddle/fluid/platform/dynload/dynamic_loader.cc:133]
75: PaddlePaddle Call Stacks: 
75: 0             0x432bf9p paddle::platform::EnforceNotMet::EnforceNotMet(std::__exception_ptr::exception_ptr, char const*, int) + 761
75: 1             0x4b2367p
75: 2             0x4b3032p paddle::platform::dynload::GetTensorRtDsoHandle() + 98
75: 3             0x435371p void std::__once_call_impl<std::_Bind_simple<decltype (createInferBuilder_INTERNAL({parm#1}...)) paddle::platform::dynload::DynLoad__createInferBuilder_INTERNAL::operator()<nvinfer1::ILogger*, int>(nvinfer1::ILogger*, int)::{lambda()#1} ()> >() + 33
75: 4       0x7f0d873a1a99p
75: 5             0x42ce5fp createInferBuilder(nvinfer1::ILogger&) + 111
75: 6             0x42d028p CreateNetwork() + 72
75: 7             0x42f138p TensorrtTest_BasicFunction_Test::TestBody() + 40
75: 8             0x4d7da3p void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) + 67
75: 9             0x4cdb8ap testing::Test::Run() + 186
75: 10            0x4cdcd8p testing::TestInfo::Run() + 280
75: 11            0x4cdde5p testing::TestCase::Run() + 229
75: 12            0x4d0287p testing::internal::UnitTestImpl::RunAllTests() + 583
75: 13            0x4d05b9p testing::UnitTest::Run() + 89
75: 14            0x42c349p main + 329
75: 15      0x7f0d86645830p __libc_start_main + 240
75: 16            0x42c989p _start + 41
75: " thrown in the test body.
75: [  FAILED  ] TensorrtTest.BasicFunction (1 ms)
75: [----------] 1 test from TensorrtTest (1 ms total)
75: 
75: [----------] Global test environment tear-down
75: [==========] 1 test from 1 test case ran. (1 ms total)
75: [  PASSED  ] 0 tests.
75: [  FAILED  ] 1 test, listed below:
75: [  FAILED  ] TensorrtTest.BasicFunction
75: 
75:  1 FAILED TEST
1/1 Test #75: test_tensorrt ....................***Failed    5.52 sec

0% tests passed, 1 tests failed out of 1

将/usr/local/lib下的相关so文件拷贝到/usr/lib下即可。

luotao1 · 2018-04-16T05:54:27Z

@@ -58,3 +58,11 @@ void GetWarpCTCDsoHandle(void** dso_handle);
 *
 */
 void GetLapackDsoHandle(void** dso_handle);


DynamicLoader.h 可以不用修改，是老paddle用的。

luotao1

lgtm

Superjomn added 10 commits April 13, 2018 13:05

add tensorrt

a3140d3

set tensorrt on as default

a60189f

add cudnn dependency

b95d819

Merge branch 'develop' of github.com:PaddlePaddle/Paddle into fea/add…

87fc090

…_tensorrt

nvtest

8dda580

add tensorrt dynamic loader

92480b5

add tensorrt as dyload

5891896

Merge branch 'develop' of github.com:PaddlePaddle/Paddle into fea/add…

1b475b3

…_tensorrt

finish test

9d617b8

remove tensorrt.cmake

0e8e85f

dependency has been installed in docker image.

Superjomn force-pushed the fea/add_tensorrt branch from b7cd337 to 0e8e85f Compare April 15, 2018 11:19

Superjomn commented Apr 15, 2018

View reviewed changes

Superjomn requested review from Xreki and luotao1 April 15, 2018 11:23

Superjomn added 2 commits April 15, 2018 19:24

fix pip upgrade pip error

63b6a74

add flag definition for tensorrt_dir

d492547

Xreki added the 预测原名Inference，包含Capi预测问题等 label Apr 16, 2018

luotao1 reviewed Apr 16, 2018

View reviewed changes

update

5132a2b

Superjomn force-pushed the fea/add_tensorrt branch from 0b5ce6c to 5132a2b Compare April 16, 2018 04:44

change cmake config

1fe9f63

Superjomn force-pushed the fea/add_tensorrt branch from d6d030b to 1fe9f63 Compare April 16, 2018 05:49

luotao1 reviewed Apr 16, 2018

View reviewed changes

turn WITH_TENSORRT OFF

d3a0c23

luotao1 approved these changes Apr 16, 2018

View reviewed changes

Superjomn merged commit 1866597 into PaddlePaddle:develop Apr 16, 2018

Superjomn deleted the fea/add_tensorrt branch April 16, 2018 12:33

luotao1 mentioned this pull request Apr 16, 2018

auto find tensorrt library and install in user root #9958

Merged

Conversation

Superjomn commented Apr 13, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Superjomn Apr 16, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

luotao1 commented Apr 16, 2018

Uh oh!

Choose a reason for hiding this comment

Uh oh!

luotao1 left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Superjomn commented Apr 13, 2018 •

edited

Loading

Superjomn Apr 16, 2018 •

edited

Loading