File tree Expand file tree Collapse file tree 1 file changed +27
-0
lines changed
Expand file tree Collapse file tree 1 file changed +27
-0
lines changed Original file line number Diff line number Diff line change @@ -563,6 +563,33 @@ To set Spark properties with a configuration file, create a
563563`spark-defaults.conf` file and set the environment variable
564564`SPARK_CONF_DIR` to the containing directory. [Learn more][15].
565565
566+ ## Custom Docker Image
567+
568+ Note: custom docker images are not supported by Mesosphere.
569+
570+ You can customize the Docker image in which Spark runs by extending
571+ the standard Spark Docker image. In this way, you can install your own
572+ libraries, such as a custom Python library.
573+
574+ 1. In your Dockerfile, extend from the standard Spark image and add your
575+ customizations:
576+
577+ ```
578+ FROM mesosphere/spark:1.0.4-2.0.1
579+ RUN apt-get install -y python-pip
580+ RUN pip install requests
581+ ```
582+
583+ 1. Then build an image from your Dockerfile.
584+
585+ $ docker build -t username/image:tag .
586+ $ docker push username/image:tag
587+
588+ 1. Reference your custom Docker image with the `--docker-image` option
589+ when running a Spark job.
590+
591+ $ dcos spark run --docker-image=myusername/myimage:v1 --submit-args="http://external.website/mysparkapp.py 30"
592+
566593<a name="pysparkshell"></a>
567594# Interactive Spark Shell
568595
You can’t perform that action at this time.
0 commit comments