test_PyDataProvider2 failed due to segfault

[yu239-zz]

[idl57-Alienware-Area-51-R2]haonanyu:~/paddle/build(ret:8)$ PYTHONPATH= ctest -R DataProvider2 -V
UpdateCTestConfiguration from :/home/haonanyu/paddle/build/DartConfiguration.tcl
UpdateCTestConfiguration from :/home/haonanyu/paddle/build/DartConfiguration.tcl
Test project /home/haonanyu/paddle/build
Constructing a list of tests
Done constructing a list of tests
Checking test dependency graph...
Checking test dependency graph end
test 28
Start 28: test_PyDataProvider2

28: Test command: /home/haonanyu/paddle/paddle/.set_python_path.sh "-d" "/home/haonanyu/paddle/paddle/gserver/tests:/home/haonanyu/paddle/python" "/home/haonanyu/paddle/build/paddle/gserver/tests/test_PyDataProvider2"
28: Test timeout computed to be: 9.99988e+06
28: + getopts d: opt
28: + case $opt in
28: + PYPATH=/home/haonanyu/paddle/paddle/gserver/tests:/home/haonanyu/paddle/python
28: + getopts d: opt
28: + shift 2
28: + export PYTHONPATH=/home/haonanyu/paddle/paddle/gserver/tests:/home/haonanyu/paddle/python:
28: + PYTHONPATH=/home/haonanyu/paddle/paddle/gserver/tests:/home/haonanyu/paddle/python:
28: + /home/haonanyu/paddle/build/paddle/gserver/tests/test_PyDataProvider2
28: WARNING: Logging before InitGoogleLogging() is written to STDERR
28: I1110 17:27:47.987854 27631 hl_dso_loader.cc:51] Try to find cuda library: libcudart.so from default system path.
28: I1110 17:27:47.988507 27631 Util.cpp:155] commandline: /home/haonanyu/paddle/build/paddle/gserver/tests/test_PyDataProvider2
28: I1110 17:27:48.128258 27631 hl_dso_loader.cc:51] Try to find cuda library: libcublas.so from default system path.
28: I1110 17:27:48.208549 27631 hl_dso_loader.cc:51] Try to find cuda library: libcurand.so from default system path.
28: I1110 17:27:48.217546 27631 hl_dso_loader.cc:51] Try to find cuda library: libcudnn.so from default system path.
28: I1110 17:27:50.104863 27631 Util.cpp:130] Calling runInitFunctions
28: I1110 17:27:50.105031 27631 Util.cpp:143] Call runInitFunctions done.
28: File "", line 11
28: paddle version:
28: ^
28: SyntaxError: invalid syntax
28: [==========] Running 11 tests from 1 test case.
28: [----------] Global test environment set-up.
28: [----------] 11 tests from PyDataProvider2
28: [ RUN ] PyDataProvider2.dense_no_seq
28: I1110 17:27:50.160810 27631 PyDataProvider2.cpp:257] loading dataprovider test_PyDataProvider2::test_dense_no_seq
28: [ OK ] PyDataProvider2.dense_no_seq (104 ms)
28: [ RUN ] PyDataProvider2.index_no_seq
28: I1110 17:27:50.216799 27631 PyDataProvider2.cpp:257] loading dataprovider test_PyDataProvider2::test_index_no_seq
28: [ OK ] PyDataProvider2.index_no_seq (1 ms)
28: [ RUN ] PyDataProvider2.init_hook
28: I1110 17:27:50.218468 27631 PyDataProvider2.cpp:257] loading dataprovider test_PyDataProvider2::test_init_hook
28: [ OK ] PyDataProvider2.init_hook (2 ms)
28: [ RUN ] PyDataProvider2.sparse_no_value_no_seq
28: I1110 17:27:50.219805 27631 PyDataProvider2.cpp:257] loading dataprovider test_PyDataProvider2::test_sparse_non_value_no_seq
28: [ OK ] PyDataProvider2.sparse_no_value_no_seq (1 ms)
28: [ RUN ] PyDataProvider2.sparse_value_no_seq
28: I1110 17:27:50.220360 27631 PyDataProvider2.cpp:257] loading dataprovider test_PyDataProvider2::test_sparse_value_no_seq
28: *** Aborted at 1478827670 (unix time) try "date -d @1478827670" if you are using GNU date ***
28: PC: @ 0x7f22cf507bf8 (unknown)
28: *** SIGSEGV (@0xa9) received by PID 27631 (TID 0x7f2266550700) from PID 169; stack trace: ***
28: @ 0x7f22cff34340 (unknown)
28: @ 0x7f22cf507bf8 (unknown)
28: @ 0x7f22cf5174be (unknown)
28: @ 0x7f22cf4a18b7 (unknown)
28: @ 0x7f22cf4a24eb (unknown)
28: @ 0x7f22cf4a2556 (unknown)
28: @ 0x7f22cf4e943f (unknown)
28: @ 0x7f22cf4caf7e (unknown)
28: @ 0x7f22cf504153 (unknown)
28: @ 0x7f22cf4cbe20 (unknown)
28: @ 0x7f22cf504153 (unknown)
28: @ 0x7f22cf41347b (unknown)
28: @ 0x65c6cb paddle::PyDataProvider2::loadThread()
28: @ 0x7f22cec96a60 (unknown)
28: @ 0x7f22cff2c182 start_thread
28: @ 0x7f22ce3fe47d (unknown)
28: @ 0x0 (unknown)
28: /home/haonanyu/paddle/paddle/.set_python_path.sh: line 42: 27631 Segmentation fault (core dumped) $@
1/1 Test #28: test_PyDataProvider2 .............***Failed 2.58 sec

0% tests passed, 1 tests failed out of 1

Total Test time (real) = 2.58 sec

The following tests FAILED:
28 - test_PyDataProvider2 (Failed)
Errors while running CTest

[reyoung]

Could you use debug mode to rerun this test for a better callstack?

To build a debug version, you need set CMAKE_BUILD_TYPE=Debug when build paddle by cmake

[yu239-zz]

30: I1110 19:30:14.924366 17254 Util.cpp:155] commandline: /home/haonanyu/paddle/build/paddle/gserver/tests/test_PyDataProvider2
30: I1110 19:30:17.291312 17254 Util.cpp:130] Calling runInitFunctions
30: I1110 19:30:17.291770 17254 Util.cpp:143] Call runInitFunctions done.
30: [==========] Running 12 tests from 1 test case.
30: [----------] Global test environment set-up.
30: [----------] 12 tests from PyDataProvider2
30: [ RUN ] PyDataProvider2.dense_no_seq
30: I1110 19:30:17.353341 17254 PyDataProvider2.cpp:257] loading dataprovider test_PyDataProvider2::test_dense_no_seq
30: [ OK ] PyDataProvider2.dense_no_seq (117 ms)
30: [ RUN ] PyDataProvider2.index_no_seq
30: I1110 19:30:17.419425 17254 PyDataProvider2.cpp:257] loading dataprovider test_PyDataProvider2::test_index_no_seq
30: [ OK ] PyDataProvider2.index_no_seq (1 ms)
30: [ RUN ] PyDataProvider2.init_hook
30: I1110 19:30:17.422075 17254 PyDataProvider2.cpp:257] loading dataprovider test_PyDataProvider2::test_init_hook
30: [ OK ] PyDataProvider2.init_hook (4 ms)
30: [ RUN ] PyDataProvider2.sparse_no_value_no_seq
30: I1110 19:30:17.424038 17254 PyDataProvider2.cpp:257] loading dataprovider test_PyDataProvider2::test_sparse_non_value_no_seq
30: [ OK ] PyDataProvider2.sparse_no_value_no_seq (0 ms)
30: [ RUN ] PyDataProvider2.sparse_value_no_seq
30: I1110 19:30:17.424867 17254 PyDataProvider2.cpp:257] loading dataprovider test_PyDataProvider2::test_sparse_value_no_seq
30: *** Aborted at 1478835017 (unix time) try "date -d @1478835017" if you are using GNU date ***
30: PC: @ 0x7f1d85046bf8 (unknown)
30: *** SIGSEGV (@0xa9) received by PID 17254 (TID 0x7f1d1c7a9700) from PID 169; stack trace: ***
30: @ 0x7f1d85a73340 (unknown)
30: @ 0x7f1d85046bf8 (unknown)
30: @ 0x7f1d850564be (unknown)
30: @ 0x7f1d84fe08b7 (unknown)
30: @ 0x7f1d84fe14eb (unknown)
30: @ 0x7f1d84fe1556 (unknown)
30: @ 0x7f1d8502843f (unknown)
30: @ 0x7f1d85009f7e (unknown)
30: @ 0x7f1d85043153 (unknown)
30: @ 0x7f1d8500ae20 (unknown)
30: @ 0x7f1d85043153 (unknown)
30: @ 0x7f1d84f5247b (unknown)
30: @ 0x9cf62c paddle::py::iterNext()
30: @ 0x9d4143 paddle::PyDataProvider2::loadThread()
30: @ 0x9d49a1 _ZZN6paddle15PyDataProvider29resetImplEbENKUlvE_clEv
30: @ 0x9e257c _ZNSt12_Bind_simpleIFZN6paddle15PyDataProvider29resetImplEbEUlvE_vEE9_M_invokeIJEEEvSt12_Index_tupleIJXspT_EEE
30: @ 0x9e24d3 _ZNSt12_Bind_simpleIFZN6paddle15PyDataProvider29resetImplEbEUlvE_vEEclEv
30: @ 0x9e246c _ZNSt6thread5_ImplISt12_Bind_simpleIFZN6paddle15PyDataProvider29resetImplEbEUlvE_vEEE6_M_runEv
30: @ 0x7f1d847d5a60 (unknown)
30: @ 0x7f1d85a6b182 start_thread
30: @ 0x7f1d83f3d47d (unknown)
30: @ 0x0 (unknown)
30: /home/haonanyu/paddle/paddle/.set_python_path.sh: line 42: 17254 Segmentation fault (core dumped) $@
1/1 Test #30: test_PyDataProvider2 .............***Failed 2.88 sec

This is what I got after set Debug flag.