Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions docs/source/developer_guide/pipelines.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ For more details on the **pipeline json definition** see it's [json schema](http
"label": "generate-contributions",
"component_parameters": {
"filename": "demo-pipelines/generate-contributions.ipynb",
"runtime_image": "tensorflow/tensorflow:2.0.0-py3",
"runtime_image": "tensorflow/tensorflow@sha256:7c01f75d58fadc2cd1109d5baac1925ed131e05925d840b1b49363c794d1c4db",
"outputs": ["community_contributions.csv"],
"env_vars": ["GITHUB_TOKEN=xxxx"],
"dependencies": ["contributors.csv"],
Expand Down Expand Up @@ -89,7 +89,7 @@ For more details on the **pipeline json definition** see it's [json schema](http
"label": "generate-stats",
"component_parameters": {
"filename": "demo-pipelines/generate-stats.ipynb",
"runtime_image": "tensorflow/tensorflow:2.0.0-py3",
"runtime_image": "tensorflow/tensorflow@sha256:7c01f75d58fadc2cd1109d5baac1925ed131e05925d840b1b49363c794d1c4db",
"outputs": ["community_stats.csv"],
"env_vars": ["GITHUB_TOKEN=xxxx"],
"dependencies": ["contributors.csv"],
Expand Down Expand Up @@ -125,7 +125,7 @@ For more details on the **pipeline json definition** see it's [json schema](http
"label": "overview",
"component_parameters": {
"filename": "demo-pipelines/overview.ipynb",
"runtime_image": "elyra/tensorflow:1.15.2-py3",
"runtime_image": "tensorflow/tensorflow@sha256:7c01f75d58fadc2cd1109d5baac1925ed131e05925d840b1b49363c794d1c4db",
"include_subdirectories": false
},
"ui_data": {
Expand Down
8 changes: 4 additions & 4 deletions docs/source/user_guide/command-line-interface.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,11 +57,11 @@ Specify the `--json` parameter to return the results in JSON to allow for progra
```
$ elyra-metadata list runtime-images --json | jq ".[].display_name"
"R Script"
"Anaconda (2020.07) with Python 3.x"
"Tensorflow 2.3.0"
"Pandas 1.1.1"
"Anaconda (2021.11) with Python 3.x"
"Tensorflow 2.8.0"
"Pandas 1.4.1"
"Pytorch 1.4 with CUDA-devel"
"Tensorflow 2.3.0 with GPU"
"Tensorflow 2.8.0 with GPU"
"Pytorch 1.4 with CUDA-runtime"
"Pandas on quay.io"
```
Expand Down
17 changes: 14 additions & 3 deletions docs/source/user_guide/runtime-image-conf.md
Original file line number Diff line number Diff line change
Expand Up @@ -185,20 +185,31 @@ Example: `My custom runtime container image`

##### Image Name (image_name)

The name and tag of an existing container image in a container registry that meets the stated prerequisites. This property is required.
The name and tag or name and SHA256-digest of an existing container image in a container registry that meets the stated prerequisites. This property is required.

Example:

- `jdoe/my-image:1.0.0`
- `jdoe/my-image:1.0.0`
OR
- `jdoe/my-image@sha256:ee783a4c0fccc7317c150450e84579544e171dd01a3f76cf2711262aced85767`

Providing only `owner/image:tag` uses default registry: Docker Hub registry
Providing only `owner/image[@SHA256]:tag/hash` uses default registry: Docker Hub registry

In general for other public container registries, the URL shall contain also `registry`, therefore the complete URL to be used in this case is: `registry/owner/image:tag`

Example:

- `quay.io/jdoe/my-image:1.0.0`

NOTE: Use SHA256-digest format in airgapped Openshift environments to support ImageContentSourcePolicy and automatic referral of images to a target repository server or to
just support unique non-modifiable image digests instead of potentially mutable tags. SHA256-digest format is supported on all platforms (Kubernetes, Openshift, Docker)
for image references.

NOTE: If you want to verify the hash signatures of our factory images you can compare the image hash signature to its official published image in DockerHub.
For example, our tensorflow image is defined as `tensorflow/tensorflow@sha256:7c01f75d58fadc2cd1109d5baac1925ed131e05925d840b1b49363c794d1c4db`
You can navigate to the tensorflow's [official Dockerhub page](https://hub.docker.com/layers/tensorflow/tensorflow/2.8.0/images/sha256-7c01f75d58fadc2cd1109d5baac1925ed131e05925d840b1b49363c794d1c4db?context=explore)
find the image version (in this example its 2.8.0) and verify the hash matches the one listed under `DIGEST: sha256: XXXXX.....`

##### Image Pull Policy (pull_policy)

This field will be the pull policy of the image when the image is selected to be part of the pipeline. This field
Expand Down
2 changes: 1 addition & 1 deletion etc/config/metadata/runtime-images/anaconda.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
"display_name": "Anaconda (2021.11) with Python 3.x",
"metadata": {
"description": "Anaconda 2021.11",
"image_name": "continuumio/anaconda3:2021.11",
"image_name": "continuumio/anaconda3@sha256:a2816acd3acda208d92e0bf6c11eb41fda9009ea20f24e123dbf84bb4bd4c4b8",
"tags": ["anaconda"]
},
"schema_name": "runtime-image"
Expand Down
2 changes: 1 addition & 1 deletion etc/config/metadata/runtime-images/pandas.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
"display_name": "Pandas 1.4.1",
"metadata": {
"description": "Pandas 1.4.1",
"image_name": "amancevice/pandas:1.4.1",
"image_name": "amancevice/pandas@sha256:f74bef70689b19d3cd610ef67227fce1c9a6ed8fa950ac2aff39ce72310d5520",
"tags": ["pandas"]
},
"schema_name": "runtime-image"
Expand Down
2 changes: 1 addition & 1 deletion etc/config/metadata/runtime-images/pytorch-devel.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
"display_name": "Pytorch 1.4 with CUDA-devel",
"metadata": {
"description": "PyTorch 1.4 (with GPU support)",
"image_name": "pytorch/pytorch:1.4-cuda10.1-cudnn7-devel",
"image_name": "pytorch/pytorch@sha256:c612782acc39256aac0637d58d297644066c62f6f84f0b88cfdc335bb25d0d22",
"tags": ["gpu", "pytorch"]
},
"schema_name": "runtime-image"
Expand Down
2 changes: 1 addition & 1 deletion etc/config/metadata/runtime-images/pytorch-runtime.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
"display_name": "Pytorch 1.4 with CUDA-runtime",
"metadata": {
"description": "PyTorch 1.4 (with GPU support)",
"image_name": "pytorch/pytorch:1.4-cuda10.1-cudnn7-runtime",
"image_name": "pytorch/pytorch@sha256:ee783a4c0fccc7317c150450e84579544e171dd01a3f76cf2711262aced85bf7",
"tags": ["gpu", "pytorch"]
},
"schema_name": "runtime-image"
Expand Down
2 changes: 1 addition & 1 deletion etc/config/metadata/runtime-images/r.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
"display_name": "R Script",
"metadata": {
"description": "R Script",
"image_name": "jupyter/r-notebook:r-4.1.2",
"image_name": "jupyter/r-notebook@sha256:08b7c3a1be31deac36cd9d65db6f6a4d58dc0cba32087101f3166d093c1a9c82",
"tags": ["R"]
},
"schema_name": "runtime-image"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
"display_name": "Tensorflow 2.8.0 with GPU",
"metadata": {
"description": "TensorFlow 2.8 (with GPU support)",
"image_name": "tensorflow/tensorflow:2.8.0-gpu",
"image_name": "tensorflow/tensorflow@sha256:1e03623e335aac1610b1a3cfa6a96cf10156acb095287f9d6031df3980148663",
"tags": ["gpu", "tensorflow"]
},
"schema_name": "runtime-image"
Expand Down
2 changes: 1 addition & 1 deletion etc/config/metadata/runtime-images/tensorflow_2x_py3.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
"display_name": "Tensorflow 2.8.0",
"metadata": {
"description": "TensorFlow 2.8",
"image_name": "tensorflow/tensorflow:2.8.0",
"image_name": "tensorflow/tensorflow@sha256:7c01f75d58fadc2cd1109d5baac1925ed131e05925d840b1b49363c794d1c4db",
"tags": ["tensorflow"]
},
"schema_name": "runtime-image"
Expand Down
10 changes: 5 additions & 5 deletions tests/integration/pipeline.ts
Original file line number Diff line number Diff line change
Expand Up @@ -189,23 +189,23 @@ describe('Pipeline Editor tests', () => {
});
cy.get('#root_component_parameters_runtime_image').within(() => {
cy.get('select[id="root_component_parameters_runtime_image"]').select(
'continuumio/anaconda3:2021.11'
'continuumio/anaconda3@sha256:a2816acd3acda208d92e0bf6c11eb41fda9009ea20f24e123dbf84bb4bd4c4b8'
);
});

// consumer props
cy.findByText('consumer.ipynb').click();
cy.get('#root_component_parameters_runtime_image').within(() => {
cy.get('select[id="root_component_parameters_runtime_image"]').select(
'continuumio/anaconda3:2021.11'
'continuumio/anaconda3@sha256:a2816acd3acda208d92e0bf6c11eb41fda9009ea20f24e123dbf84bb4bd4c4b8'
);
});

// setup props
cy.findByText('setup.py').click();
cy.get('#root_component_parameters_runtime_image').within(() => {
cy.get('select[id="root_component_parameters_runtime_image"]').select(
'continuumio/anaconda3:2021.11'
'continuumio/anaconda3@sha256:a2816acd3acda208d92e0bf6c11eb41fda9009ea20f24e123dbf84bb4bd4c4b8'
);
});
cy.get('#root_component_parameters_dependencies').within(() => {
Expand All @@ -225,7 +225,7 @@ describe('Pipeline Editor tests', () => {
cy.findByText('create-source-files.py').click();
cy.get('#root_component_parameters_runtime_image').within(() => {
cy.get('select[id="root_component_parameters_runtime_image"]').select(
'continuumio/anaconda3:2021.11'
'continuumio/anaconda3@sha256:a2816acd3acda208d92e0bf6c11eb41fda9009ea20f24e123dbf84bb4bd4c4b8'
);
});
cy.get('#root_component_parameters_outputs').within(() => {
Expand All @@ -244,7 +244,7 @@ describe('Pipeline Editor tests', () => {
cy.findByText('producer-script.py').click();
cy.get('#root_component_parameters_runtime_image').within(() => {
cy.get('select[id="root_component_parameters_runtime_image"]').select(
'continuumio/anaconda3:2021.11'
'continuumio/anaconda3@sha256:a2816acd3acda208d92e0bf6c11eb41fda9009ea20f24e123dbf84bb4bd4c4b8'
);
});
cy.get('#root_component_parameters_outputs').within(() => {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ Object {
"output-1.csv",
"output-2.csv",
],
"runtime_image": "continuumio/anaconda3:2021.11",
"runtime_image": "continuumio/anaconda3@sha256:a2816acd3acda208d92e0bf6c11eb41fda9009ea20f24e123dbf84bb4bd4c4b8",
},
"label": "",
"ui_data": Object {
Expand Down Expand Up @@ -94,7 +94,7 @@ Object {
"kubernetes_tolerations": Array [],
"mounted_volumes": Array [],
"outputs": Array [],
"runtime_image": "continuumio/anaconda3:2021.11",
"runtime_image": "continuumio/anaconda3@sha256:a2816acd3acda208d92e0bf6c11eb41fda9009ea20f24e123dbf84bb4bd4c4b8",
},
"label": "",
"ui_data": Object {
Expand Down Expand Up @@ -153,7 +153,7 @@ Object {
"kubernetes_tolerations": Array [],
"mounted_volumes": Array [],
"outputs": Array [],
"runtime_image": "continuumio/anaconda3:2021.11",
"runtime_image": "continuumio/anaconda3@sha256:a2816acd3acda208d92e0bf6c11eb41fda9009ea20f24e123dbf84bb4bd4c4b8",
},
"label": "",
"ui_data": Object {
Expand Down Expand Up @@ -213,7 +213,7 @@ Object {
"input-1.csv",
"input-2.csv",
],
"runtime_image": "continuumio/anaconda3:2021.11",
"runtime_image": "continuumio/anaconda3@sha256:a2816acd3acda208d92e0bf6c11eb41fda9009ea20f24e123dbf84bb4bd4c4b8",
},
"label": "",
"ui_data": Object {
Expand Down Expand Up @@ -273,7 +273,7 @@ Object {
"output-3.csv",
"output-4.csv",
],
"runtime_image": "continuumio/anaconda3:2021.11",
"runtime_image": "continuumio/anaconda3@sha256:a2816acd3acda208d92e0bf6c11eb41fda9009ea20f24e123dbf84bb4bd4c4b8",
},
"label": "",
"ui_data": Object {
Expand Down