Skip to content

Commit e19861c

Browse files
author
Helin Wang
committed
refine doc for run paddle on docker
1 parent 06b2e4d commit e19861c

1 file changed

Lines changed: 159 additions & 122 deletions

File tree

‎doc/getstarted/build_and_install/docker_install_en.rst‎

Lines changed: 159 additions & 122 deletions
Original file line numberDiff line numberDiff line change
@@ -8,199 +8,236 @@ Please be aware that you will need to change `Dockers settings
88
<https://github.com/PaddlePaddle/Paddle/issues/627>`_ to make full use
99
of your hardware resource on Mac OS X and Windows.
1010

11+
Working With Docker
12+
-------------------
13+
14+
Here we will describe the basic docker concepts that we will be using
15+
in this tutorial.
16+
17+
- *container* is an environment for running applications
18+
19+
- *image* is an immutable snapshot of a docker container. One can run
20+
a container based on a docker image by using command :code:`docker
21+
run docker_image_name`.
22+
23+
- By default docker container have an isolated file system namespace,
24+
we can not see the files in the host file system. By using *volume*,
25+
mounted files in host will be visible inside docker container.
26+
Following command will mount current dirctory into /data inside
27+
docker container, run docker container from debian image with
28+
command :code:`ls /data`.
29+
30+
.. code-block:: bash
31+
32+
docker run --rm -v $(pwd):/data debian ls /data
1133
1234
Usage of CPU-only and GPU Images
1335
----------------------------------
1436

15-
For each version of PaddlePaddle, we release 2 types of Docker images: development
16-
image and production image. Production image includes CPU-only version and a CUDA
17-
GPU version and their no-AVX versions. We put the docker images on
18-
`dockerhub.com <https://hub.docker.com/r/paddledev/paddle/>`_. You can find the
19-
latest versions under "tags" tab at dockerhub.com.
20-
1. development image :code:`paddlepaddle/paddle:<version>-dev`
37+
For each version of PaddlePaddle, we release 2 types of Docker images:
38+
development image and production image. Production image includes
39+
CPU-only version and a CUDA GPU version and their no-AVX versions. We
40+
put the docker images on `dockerhub.com
41+
<https://hub.docker.com/r/paddledev/paddle/>`_. You can find the
42+
latest versions under "tags" tab at dockerhub.com
2143

22-
This image has packed related develop tools and runtime environment. Users and
23-
developers can use this image instead of their own local computer to accomplish
24-
development, build, releasing, document writing etc. While different version of
25-
paddle may depends on different version of libraries and tools, if you want to
26-
setup a local environment, you must pay attention to the versions.
27-
The development image contains:
28-
- gcc/clang
29-
- nvcc
30-
- Python
31-
- sphinx
32-
- woboq
33-
- sshd
34-
Many developers use servers with GPUs, they can use ssh to login to the server
35-
and run :code:`docker exec` to enter the docker container and start their work.
36-
Also they can start a development docker image with SSHD service, so they can login to
37-
the container and start work.
44+
1. Production images, this image might have multiple variants:
3845

39-
To run the CPU-only image as an interactive container:
46+
- GPU/AVX：:code:`paddlepaddle/paddle:<version>-gpu`
47+
- GPU/no-AVX：:code:`paddlepaddle/paddle:<version>-gpu-noavx`
48+
- CPU/AVX：:code:`paddlepaddle/paddle:<version>`
49+
- CPU/no-AVX：:code:`paddlepaddle/paddle:<version>-noavx`
4050

41-
.. code-block:: bash
51+
Please be aware that the CPU-only and the GPU images both use the
52+
AVX instruction set, but old computers produced before 2008 do not
53+
support AVX. The following command checks if your Linux computer
54+
supports AVX:
4255

43-
docker run -it --rm paddledev/paddle:<version> /bin/bash
56+
.. code-block:: bash
4457
45-
or, we can run it as a daemon container
58+
if cat /proc/cpuinfo | grep -i avx; then echo Yes; else echo No; fi
4659
47-
.. code-block:: bash
60+
61+
To run the CPU-only image as an interactive container:
4862

49-
docker run -d -p 2202:22 -p 8888:8888 paddledev/paddle:<version>
63+
.. code-block:: bash
5064
51-
and SSH to this container using password :code:`root`:
65+
docker run -it --rm paddlepaddle/paddle:0.10.0rc2 /bin/bash
5266
53-
.. code-block:: bash
67+
Above method work with the GPU image too -- the recommended way is
68+
using `nvidia-docker <https://github.com/NVIDIA/nvidia-docker>`_.
5469

55-
ssh -p 2202 root@localhost
70+
Please install nvidia-docker first following this `tutorial
71+
<https://github.com/NVIDIA/nvidia-docker#quick-start>`_.
5672

57-
An advantage of using SSH is that we can connect to PaddlePaddle from
58-
more than one terminals. For example, one terminal running vi and
59-
another one running Python interpreter. Another advantage is that we
60-
can run the PaddlePaddle container on a remote server and SSH to it
61-
from a laptop.
73+
Now you can run a GPU image:
6274

75+
.. code-block:: bash
6376
64-
2. Production images, this image might have multiple variants:
65-
- GPU/AVX：:code:`paddlepaddle/paddle:<version>-gpu`
66-
- GPU/no-AVX：:code:`paddlepaddle/paddle:<version>-gpu-noavx`
67-
- CPU/AVX：:code:`paddlepaddle/paddle:<version>`
68-
- CPU/no-AVX：:code:`paddlepaddle/paddle:<version>-noavx`
77+
nvidia-docker run -it --rm paddlepaddle/paddle:0.10.0rc2-gpu /bin/bash
6978
70-
Please be aware that the CPU-only and the GPU images both use the AVX
71-
instruction set, but old computers produced before 2008 do not support
72-
AVX. The following command checks if your Linux computer supports
73-
AVX:
79+
2. development image :code:`paddlepaddle/paddle:<version>-dev`
7480

75-
.. code-block:: bash
81+
This image has packed related develop tools and runtime
82+
environment. Users and developers can use this image instead of
83+
their own local computer to accomplish development, build,
84+
releasing, document writing etc. While different version of paddle
85+
may depends on different version of libraries and tools, if you
86+
want to setup a local environment, you must pay attention to the
87+
versions. The development image contains:
88+
89+
- gcc/clang
90+
- nvcc
91+
- Python
92+
- sphinx
93+
- woboq
94+
- sshd
95+
96+
Many developers use servers with GPUs, they can use ssh to login to
97+
the server and run :code:`docker exec` to enter the docker
98+
container and start their work. Also they can start a development
99+
docker image with SSHD service, so they can login to the container
100+
and start work.
76101

77-
if cat /proc/cpuinfo | grep -i avx; then echo Yes; else echo No; fi
78102

103+
Train Model Using Python API
104+
----------------------------
79105

80-
If it doesn't, we will use the non-AVX images.
106+
Our official docker image provides a runtime for PaddlePaddle
107+
programs. The typical workflow will be as follows:
81108

82-
Above methods work with the GPU image too -- just please don't forget
83-
to install GPU driver. To support GPU driver, we recommend to use
84-
[nvidia-docker](https://github.com/NVIDIA/nvidia-docker). Run using
109+
Create a directory as workspace:
85110

86-
.. code-block:: bash
111+
.. code-block:: bash
87112
88-
nvidia-docker run -it --rm paddledev/paddle:0.10.0rc1-gpu /bin/bash
113+
mkdir ~/workspace
89114
90-
Note: If you would have a problem running nvidia-docker, you may try the old method we have used (not recommended).
115+
Edit a PaddlePaddle python program using your favourite editor
91116

92-
.. code-block:: bash
117+
.. code-block:: bash
93118
94-
export CUDA_SO="$(\ls /usr/lib64/libcuda* | xargs -I{} echo '-v {}:{}') $(\ls /usr/lib64/libnvidia* | xargs -I{} echo '-v {}:{}')"
95-
export DEVICES=$(\ls /dev/nvidia* | xargs -I{} echo '--device {}:{}')
96-
docker run ${CUDA_SO} ${DEVICES} -it paddledev/paddle:<version>-gpu
119+
emacs ~/workspace/example.py
97120
121+
Run the program using docker:
98122

99-
3. Use production image to release you AI application
100-
Suppose that we have a simple application program in :code:`a.py`, we can test and run it using the production image:
123+
.. code-block:: bash
101124
102-
```bash
103-
docker run -it -v $PWD:/work paddle /work/a.py
104-
```
125+
docker run -it --rm -v ~/workspace:/workspace paddlepaddle/paddle:0.10.0rc2 python /workspace/example.py
105126
106-
But this works only if all dependencies of :code:`a.py` are in the production image. If this is not the case, we need to build a new Docker image from the production image and with more dependencies installs.
127+
Or if you are using GPU for training:
107128

129+
.. code-block:: bash
108130
109-
PaddlePaddle Book
110-
------------------
131+
nvidia-docker run -it --rm -v ~/workspace:/workspace paddlepaddle/paddle:0.10.0rc2-gpu python /workspace/example.py
111132
112-
The Jupyter Notebook is an open-source web application that allows
113-
you to create and share documents that contain live code, equations,
114-
visualizations and explanatory text in a single browser.
133+
Above commands will start a docker container by running :code:`python
134+
/workspace/example.py`. It will stop once :code:`python
135+
/workspace/example.py` finishes.
115136

116-
PaddlePaddle Book is an interactive Jupyter Notebook for users and developers.
117-
We already exposed port 8888 for this book. If you want to
118-
dig deeper into deep learning, PaddlePaddle Book definitely is your best choice.
137+
Another way is to tell docker to start a :code:`/bin/bash` session and
138+
run PaddlePaddle program interactively:
119139

120-
We provide a packaged book image, simply issue the command:
140+
.. code-block:: bash
141+
142+
docker run -it -v ~/workspace:/workspace paddlepaddle/paddle:0.10.0rc2 /bin/bash
143+
# now we are inside docker container
144+
cd /workspace
145+
python example.py
146+
147+
Running with GPU is identical:
121148

122149
.. code-block:: bash
123150
124-
docker run -p 8888:8888 paddlepaddle/book
151+
nvidia-docker run -it -v ~/workspace:/workspace paddlepaddle/paddle:0.10.0rc2-gpu /bin/bash
152+
# now we are inside docker container
153+
cd /workspace
154+
python example.py
125155
126-
Then, you would back and paste the address into the local browser:
127156
128-
.. code-block:: text
157+
Develop PaddlePaddle or Train Model Using C++ API
158+
---------------------------------------------------
129159

130-
http://localhost:8888/
160+
We will be using PaddlePaddle development image since it contains all
161+
compiling tools and dependencies.
131162

132-
That's all. Enjoy your journey!
163+
Let's clone PaddlePaddle repo first:
133164

134-
Development Using Docker
135-
------------------------
165+
.. code-block:: bash
136166
137-
Developers can work on PaddlePaddle using Docker. This allows
138-
developers to work on different platforms -- Linux, Mac OS X, and
139-
Windows -- in a consistent way.
167+
git clone https://github.com/PaddlePaddle/Paddle.git && cd Paddle
140168
141-
1. Build the Development Docker Image
169+
Mount both workspace folder and paddle code folder into docker
170+
container, so we can access them inside docker container. There are
171+
two ways of using PaddlePaddle development docker image:
142172

143-
.. code-block:: bash
173+
- run interactive bash directly
144174

145-
git clone --recursive https://github.com/PaddlePaddle/Paddle
146-
cd Paddle
147-
docker build -t paddle:dev .
175+
.. code-block:: bash
148176
149-
Note that by default :code:`docker build` wouldn't import source
150-
tree into the image and build it. If we want to do that, we need docker the
151-
development docker image and then run the following command:
177+
# use nvidia-docker instead of docker if you need to use GPU
178+
docker run -it -v ~/workspace:/workspace -v $(pwd):/paddle paddlepaddle/paddle:0.10.0rc2-dev /bin/bash
179+
# now we are inside docker container
152180
153-
.. code-block:: bash
181+
- or, we can run it as a daemon container
154182

155-
docker run -v $PWD:/paddle -e "WITH_GPU=OFF" -e "WITH_AVX=ON" -e "TEST=OFF" paddle:dev
183+
.. code-block:: bash
156184
185+
# use nvidia-docker instead of docker if you need to use GPU
186+
docker run -d -p 2202:22 -p 8888:8888 -v ~/workspace:/workspace -v $(pwd):/paddle paddlepaddle/paddle:0.10.0rc2-dev /usr/sbin/sshd -D
157187
158-
2. Run the Development Environment
188+
and SSH to this container using password :code:`root`:
159189

160-
Once we got the image :code:`paddle:dev`, we can use it to develop
161-
Paddle by mounting the local source code tree into a container that
162-
runs the image:
190+
.. code-block:: bash
163191
164-
.. code-block:: bash
192+
ssh -p 2202 root@localhost
165193
166-
docker run -d -p 2202:22 -p 8888:8888 -v $PWD:/paddle paddle:dev sshd
194+
An advantage is that we can run the PaddlePaddle container on a
195+
remote server and SSH to it from a laptop.
167196

168-
This runs a container of the development environment Docker image
169-
with the local source tree mounted to :code:`/paddle` of the
170-
container.
197+
When developing PaddlePaddle, you can edit PaddlePaddle source code
198+
from outside of docker container using your favoriate editor. To
199+
compile PaddlePaddle, run inside container:
171200

172-
The above :code:`docker run` commands actually starts
173-
an SSHD server listening on port 2202. This allows us to log into
174-
this container with:
201+
.. code-block:: bash
175202
176-
.. code-block:: bash
203+
WITH_GPU=OFF WITH_AVX=ON WITH_TEST=ON bash /paddle/paddle/scripts/docker/build.sh
177204
178-
ssh root@localhost -p 2202
205+
This builds everything about Paddle in :code:`/paddle/build`. And we
206+
can run unit tests there:
179207

180-
Usually, I run above commands on my Mac. I can also run them on a
181-
GPU server :code:`xxx.yyy.zzz.www` and ssh from my Mac to it:
208+
.. code-block:: bash
182209
183-
.. code-block:: bash
210+
cd /paddle/build
211+
ctest
184212
185-
my-mac$ ssh root@xxx.yyy.zzz.www -p 2202
213+
When training model using C++ API, we can edit paddle program in
214+
~/workspace outside of docker. And build from /workspace inside of
215+
docker.
186216

187-
3. Build and Install Using the Development Environment
217+
PaddlePaddle Book
218+
------------------
188219

189-
Once I am in the container, I can use
190-
:code:`paddle/scripts/docker/build.sh` to build, install, and test
191-
Paddle:
220+
The Jupyter Notebook is an open-source web application that allows
221+
you to create and share documents that contain live code, equations,
222+
visualizations and explanatory text in a single browser.
192223

193-
.. code-block:: bash
224+
PaddlePaddle Book is an interactive Jupyter Notebook for users and developers.
225+
We already exposed port 8888 for this book. If you want to
226+
dig deeper into deep learning, PaddlePaddle Book definitely is your best choice.
194227

195-
/paddle/paddle/scripts/docker/build.sh
228+
We provide a packaged book image, simply issue the command:
196229

197-
This builds everything about Paddle in :code:`/paddle/build`. And
198-
we can run unit tests there:
230+
.. code-block:: bash
199231
200-
.. code-block:: bash
232+
docker run -p 8888:8888 paddlepaddle/book
201233
202-
cd /paddle/build
203-
ctest
234+
Then, you would back and paste the address into the local browser:
235+
236+
.. code-block:: text
237+
238+
http://localhost:8888/
239+
240+
That's all. Enjoy your journey!
204241

205242

206243
Documentation

0 commit comments

Comments
 (0)