Skip to content

Commit 864758b

Browse files
author
Suzanne Scala
authored
Merge pull request apache#81 from mesosphere/document-custom-build
Document building a custom image
2 parents 11c1e16 + d95a0fb commit 864758b

File tree

1 file changed

+27
-0
lines changed

1 file changed

+27
-0
lines changed

docs/user-docs.md

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -563,6 +563,33 @@ To set Spark properties with a configuration file, create a
563563
`spark-defaults.conf` file and set the environment variable
564564
`SPARK_CONF_DIR` to the containing directory. [Learn more][15].
565565
566+
## Custom Docker Image
567+
568+
Note: custom docker images are not supported by Mesosphere.
569+
570+
You can customize the Docker image in which Spark runs by extending
571+
the standard Spark Docker image. In this way, you can install your own
572+
libraries, such as a custom Python library.
573+
574+
1. In your Dockerfile, extend from the standard Spark image and add your
575+
customizations:
576+
577+
```
578+
FROM mesosphere/spark:1.0.4-2.0.1
579+
RUN apt-get install -y python-pip
580+
RUN pip install requests
581+
```
582+
583+
1. Then build an image from your Dockerfile.
584+
585+
$ docker build -t username/image:tag .
586+
$ docker push username/image:tag
587+
588+
1. Reference your custom Docker image with the `--docker-image` option
589+
when running a Spark job.
590+
591+
$ dcos spark run --docker-image=myusername/myimage:v1 --submit-args="http://external.website/mysparkapp.py 30"
592+
566593
<a name="pysparkshell"></a>
567594
# Interactive Spark Shell
568595

0 commit comments

Comments
 (0)