Paddle docker build readme need to illustrate how to test paddle. Current implementation of test is not correct.

[helinwang]

Currently readme only mentioned "test" in this command

docker run -v $PWD:/paddle -e "GPU=OFF" -e "AVX=ON" -e "TEST=ON" paddle:dev

But there is no explanation about how will the test be performed. E.g., will the test run during building docker image paddle:dev, or later when paddle is building the final output image.

Current implementation is paddle will run test inside paddle:dev image, which is incorrect. Since the environment maybe different from the final image. Actually I encountered this problem: the test fails in paddle:dev since the numpy version for paddle:dev is too old, but the numpy version is good in the final produced image.

[typhoonzero]

Well currently it's the only way to run tests in paddle:dev. Or else we need to package all the test binaries into production image, and write a test script instead of running make test or make coverall. I'll fix this by updating the paddle:dev environment.

[typhoonzero]

Encounterd a strange issue when running build and test in paddle:dev: /bin/sh: 1: eval: /usr/local/bin/cmake: Argument list too long

58/60 Test #18: test_Allocator ......................   Passed    0.20 sec
59/60 Test #19: test_Tensor .........................   Passed   82.51 sec
60/60 Test #47: test_CompareTwoNets .................   Passed   98.26 sec

100% tests passed, 0 tests failed out of 60

Total Test time (real) =  98.27 sec
/bin/sh: 1: eval: /usr/local/bin/cmake: Argument list too long
make[3]: *** [CMakeFiles/coveralls_generate] Error 2
make[2]: *** [CMakeFiles/coveralls_generate.dir/all] Error 2
make[1]: *** [CMakeFiles/coveralls.dir/rule] Error 2
make: *** [coveralls] Error 2

[helinwang]

@gangliao ^^ could you please check above comment?

[helinwang]

@typhoonzero I think @gangliao is familiar with cmake as well, maybe he can give some suggestions.

[typhoonzero]

@helinwang I've talked with @gangliao . we'll try to figure out a solution later.

[helinwang]

I tried to use production image for test, after making ctest binary available in the production image (the test programs are generated to paddle/build, which gets mounted into the production image), I can run unit test. However, some tests failed due to binary not found:

      Start  1: test_Thread
Could not find executable test_Thread
Looked in the following places:
test_Thread
test_Thread
Release/test_Thread
Release/test_Thread
Debug/test_Thread
Debug/test_Thread
MinSizeRel/test_Thread
MinSizeRel/test_Thread
RelWithDebInfo/test_Thread
RelWithDebInfo/test_Thread
Deployment/test_Thread
Deployment/test_Thread
Development/test_Thread
Development/test_Thread
Unable to find executable: test_Thread
...
50% tests passed, 32 tests failed out of 64

Total Test time (real) =  63.93 sec

The following tests FAILED:
	  1 - test_Thread (Not Run)
	  2 - test_StringUtils (Not Run)
	  3 - test_CustomStackTrace (Not Run)
	  4 - test_ThreadBarrier (Not Run)
	  5 - test_SpinLock (Not Run)
	  6 - test_SIMDFlags (Not Run)
	  7 - test_Error (Not Run)
	  8 - test_CustomStackTracePrint (Failed)
	 37 - test_PyDataProvider (Failed)
	 39 - test_WarpCTCLayer (OTHER_FAULT)
	 40 - test_RecurrentGradientMachine (Failed)
	 41 - test_NetworkCompare (Failed)
	 42 - test_PyDataProvider2 (Failed)
	 46 - test_Compare (Failed)
	 47 - test_Trainer (Failed)
	 48 - test_TrainerOnePass (Failed)
	 49 - test_CompareTwoNets (Failed)
	 50 - test_CompareTwoOpts (Failed)
	 51 - test_CompareSparse (Failed)
	 52 - test_recurrent_machine_generation (Failed)
	 53 - test_PyDataProviderWrapper (Failed)
	 54 - test_config_parser (Failed)
	 55 - test_swig_api (Failed)
	 56 - layers_test (Failed)
	 57 - test_reset_hook (Failed)
	 58 - test_layerHelpers (Failed)
	 59 - test_v2_api (Failed)
	 60 - test_v2_layer (Failed)
	 61 - test_v2_rnn_layer (Failed)
	 62 - test_topology (Failed)
	 63 - reader_tests (Failed)
	 64 - test_ploter (Failed)

Maybe this can be a direction to solve the problem: copy everything unit test need into paddle/build and use production image to test it.

[gongweibao]

Feel free to reopen it.