Skip to content

Commit 00645ad

Browse files
authored
Merge pull request #360 from keboola/jv-PAT-129-purge-docs-non-ecr
Clarify AWS ECR as the only supported Docker registry for Keboola components
2 parents 7bfd517 + b2ec400 commit 00645ad

File tree

8 files changed

+73
-82
lines changed

8 files changed

+73
-82
lines changed

extend/component/deployment/index.md

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -299,8 +299,7 @@ allows you to obtain the repository for a component and push credentials to that
299299
[running components](/extend/component/running/#running-a-component), for example, how to obtain the AWS registry credentials.
300300
If you want to get even more low level, you can use the [Developer Portal API](https://kebooladeveloperportal.docs.apiary.io/#) directly.
301301
It also allows you to [generate credentials for a service account](https://kebooladeveloperportal.docs.apiary.io/#reference/0/vendor/create-service-account)
302-
programmatically. Apart from our AWS ECR registry, we also support running images stored in [Quay.io](https://quay.io/repository/)
303-
and [Docker Hub](https://hub.docker.com/) registries.
302+
programmatically. We use our AWS ECR registry for hosting all component images.
304303

305304
## Test Live Configurations
306305
Testing your component can be simply added as part of the script in `.travis.yml` file. See an example in

extend/component/docker-tutorial/index.md

Lines changed: 3 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -34,10 +34,9 @@ Docker Images are created by executing the instructions written in a **Dockerfil
3434
file consisting mostly of shell commands which must be executed to prepare the application for running.
3535
Docker images can be based on other images. So if
3636
you need minor modification to a system, you do not have to build the whole thing from scratch. If you want Images to be
37-
reused, *push* your Dockerfile to a **Docker registry**. The registries ([Dockerhub](https://hub.docker.com/),
38-
[Quay](https://quay.io/)) will build the image; anyone interested in using it can download it.
39-
[AWS ECR](https://aws.amazon.com/ecr/) is a private repository and has no build triggers. You need to push the images manually or
40-
using a [deploy script](/extend/component/deployment/) in your CI pipeline.
37+
reused, *push* your Dockerfile to a **Docker registry**. For Keboola components, we use
38+
[AWS ECR](https://aws.amazon.com/ecr/). You need to push the images using a
39+
[deploy script](/extend/component/deployment/) in your CI pipeline.
4140

4241
Docker image names are based on the following scheme: `registry-name/account-name/image-name:tag` where _registry-name_
4342
and _account-name_ can sometimes be omitted. For example, you can refer to a Docker _hello-world_ image as: `hello-world`

extend/component/docker-tutorial/registry.md

Lines changed: 3 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -11,11 +11,8 @@ redirect_from:
1111

1212
An important part of the Docker ecosystem is a **Docker registry**. It acts as a folder of images, taking
1313
care of their storing and building.
14-
[Docker Hub](https://hub.docker.com/) is the official Docker registry.
15-
16-
For reliability reasons, we strongly recommend to use the [Amazon AWS ECR](https://aws.amazon.com/ecr/)
17-
[provisioned by the **Keboola Developer Portal**](/extend/component/deployment/).
18-
We also support Docker Hub and [Quay](https://quay.io/), both public and private repositories.
14+
For reliability reasons, we use the [Amazon AWS ECR](https://aws.amazon.com/ecr/)
15+
[provisioned by the **Keboola Developer Portal**](/extend/component/deployment/). This is the only supported Docker registry for Keboola components.
1916

2017
## Working with Registry
2118
In order to run an image, **pull** (`docker pull`) the image to your machine. The `docker run`
@@ -88,4 +85,4 @@ trigger a new build of the Docker image. Also note that the image automatically
8885
or branch name. So, when you push a commit to the `master` branch, you will get an image with a tag `master` (which
8986
will move away from any older image builds). When creating a `1.0.0` tag, you will get an image with a `1.0.0` tag.
9087

91-
When using images in Keboola, we **highly recommend to use our [ECR repository](/extend/component/deployment/)**.
88+
When using images in Keboola, you must use our [ECR repository](/extend/component/deployment/).
Lines changed: 61 additions & 61 deletions
Original file line numberDiff line numberDiff line change
@@ -1,61 +1,61 @@
1-
---
2-
title: Installation and Running
3-
permalink: /extend/component/docker-tutorial/setup/
4-
redirect_from:
5-
- /extend/docker/tutorial/setup/
6-
---
7-
8-
* TOC
9-
{:toc}
10-
11-
To work with Docker, you need to have it installed. If you do not have a computer with a Docker machine, you can obtain a
12-
cheap [hosted server](https://marketplace.digitalocean.com/apps/docker). You can also
13-
run everything locally. To install a Docker machine on Win/Mac,
14-
use [Docker Community Edition](https://hub.docker.com/search?type=edition&offering=community). For
15-
other systems, see the [documentation](https://docs.docker.com/install/).
16-
17-
## Getting Started
18-
To test that everything is running correctly, start with an example
19-
from the Docker [documentation](https://docs.docker.com/get-started/).
20-
Run the following commands on the command line:
21-
22-
docker run hello-world
23-
24-
25-
or
26-
27-
docker run docker.io/library/hello-world:latest
28-
29-
If this works, use any image published in any Docker
30-
registry, e.g. [Docker Hub](https://hub.docker.com/), [Quay](https://quay.io/) or [AWS ECR](https://aws.amazon.com/ecr/).
31-
Note that in some configurations, you may need to use `sudo` to run `docker`. If you run into problems, consult the
32-
official troubleshooting ([Windows](https://docs.docker.com/docker-for-windows/troubleshoot/), [Mac](https://docs.docker.com/docker-for-mac/troubleshoot/)).
33-
34-
## Generally Useful Commands
35-
- [`docker run`](https://docs.docker.com/engine/reference/run/): Run an
36-
image (create a container and run the command in `ENTRYPOINT` or `CMD` section of **Dockerfile**).
37-
- [`docker build`](https://docs.docker.com/engine/reference/commandline/build/): Build
38-
an image (execute instructions in **Dockerfile** and create a runnable image).
39-
- [`docker pull`](https://docs.docker.com/engine/reference/commandline/pull/): Pull
40-
a newer version of an image (force update of the cached copy).
41-
42-
## Sharing Files
43-
Sharing files between the host OS and the container is done using the `--volume` parameter of the `docker run` command:
44-
45-
docker run --volume=/hostPath/:/containerPath/ imageName
46-
47-
Do not use any spaces around `:`. The details of file sharing are somewhat
48-
[dependent on the host OS](https://docs.docker.com/storage/volumes/) you are using.
49-
There is a very
50-
useful [guide for Rocker image](https://github.com/rocker-org/rocker/wiki/Sharing-files-with-host-machine), which
51-
describes all the sharing options in great detail.
52-
53-
On Linux systems, where Docker runs natively, there is really not much to think about. The only things that can bite
54-
you are file and directory permissions. Keep in mind that files created within the running Docker container on
55-
a mounted host volume will be created with the permissions of users of that container (not the host OS).
56-
57-
On Mac OS, follow the [official example](https://docs.docker.com/docker-for-mac/osxfs/).
58-
The limitation is that your host path must be in a shared directory (e.g., `/Users/`).
59-
60-
On Windows, follow the [official example](https://docs.docker.com/docker-for-windows/#shared-drives).
61-
The limitation is that you must use absolute paths with `docker run`.
1+
---
2+
title: Installation and Running
3+
permalink: /extend/component/docker-tutorial/setup/
4+
redirect_from:
5+
- /extend/docker/tutorial/setup/
6+
---
7+
8+
* TOC
9+
{:toc}
10+
11+
To work with Docker, you need to have it installed. If you do not have a computer with a Docker machine, you can obtain a
12+
cheap [hosted server](https://marketplace.digitalocean.com/apps/docker). You can also
13+
run everything locally. To install a Docker machine on Win/Mac,
14+
use [Docker Community Edition](https://hub.docker.com/search?type=edition&offering=community). For
15+
other systems, see the [documentation](https://docs.docker.com/install/).
16+
17+
## Getting Started
18+
To test that everything is running correctly, start with an example
19+
from the Docker [documentation](https://docs.docker.com/get-started/).
20+
Run the following commands on the command line:
21+
22+
docker run hello-world
23+
24+
25+
or
26+
27+
docker run docker.io/library/hello-world:latest
28+
29+
If this works, use any image published in any Docker
30+
registry, e.g. [Docker Hub](https://hub.docker.com/), [Quay](https://quay.io/) or [AWS ECR](https://aws.amazon.com/ecr/).
31+
Note that in some configurations, you may need to use `sudo` to run `docker`. If you run into problems, consult the
32+
official troubleshooting ([Windows](https://docs.docker.com/docker-for-windows/troubleshoot/), [Mac](https://docs.docker.com/docker-for-mac/troubleshoot/)).
33+
34+
## Generally Useful Commands
35+
- [`docker run`](https://docs.docker.com/engine/reference/run/): Run an
36+
image (create a container and run the command in `ENTRYPOINT` or `CMD` section of **Dockerfile**).
37+
- [`docker build`](https://docs.docker.com/engine/reference/commandline/build/): Build
38+
an image (execute instructions in **Dockerfile** and create a runnable image).
39+
- [`docker pull`](https://docs.docker.com/engine/reference/commandline/pull/): Pull
40+
a newer version of an image (force update of the cached copy).
41+
42+
## Sharing Files
43+
Sharing files between the host OS and the container is done using the `--volume` parameter of the `docker run` command:
44+
45+
docker run --volume=/hostPath/:/containerPath/ imageName
46+
47+
Do not use any spaces around `:`. The details of file sharing are somewhat
48+
[dependent on the host OS](https://docs.docker.com/storage/volumes/) you are using.
49+
There is a very
50+
useful [guide for Rocker image](https://github.com/rocker-org/rocker/wiki/Sharing-files-with-host-machine), which
51+
describes all the sharing options in great detail.
52+
53+
On Linux systems, where Docker runs natively, there is really not much to think about. The only things that can bite
54+
you are file and directory permissions. Keep in mind that files created within the running Docker container on
55+
a mounted host volume will be created with the permissions of users of that container (not the host OS).
56+
57+
On Mac OS, follow the [official example](https://docs.docker.com/docker-for-mac/osxfs/).
58+
The limitation is that your host path must be in a shared directory (e.g., `/Users/`).
59+
60+
On Windows, follow the [official example](https://docs.docker.com/docker-for-windows/#shared-drives).
61+
The limitation is that you must use absolute paths with `docker run`.

extend/component/tutorial/index.md

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -236,9 +236,7 @@ We offer free hosting of your Docker images in the **[Amazon Container Registry
236236
All repositories in AWS ECR are private. When you create your component using the method shown above, we
237237
have just provisioned you with the Docker image hosting and you do not need to worry about it any more.
238238

239-
We also support the DockerHub and Quay.io registries, both public and private. However, as they are more prone to
240-
outages and beyond our control, we recommend using our reliable AWS ECR. Use DockerHub or Quay.io only if you,
241-
for instance, want the image to be public.
239+
AWS ECR is the **only supported registry** for Keboola components, providing reliable hosting for all Docker images.
242240

243241
## Summary
244242
You have just created your own Keboola component. Although it does not do much, it shows the easiest path

extend/generic-extractor/publish.md

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -36,9 +36,7 @@ have created the component, edit it, and fill in the following details:
3636
- **Region** -- leave empty
3737
- **UI options** --- set to `genericTemplatesUI`
3838

39-
For a list of available tags, see the [Generic Extractor GitHub repository](https://github.com/keboola/generic-extractor/) or
40-
[Generic Extractor Quay repository](https://quay.io/repository/keboola/generic-extractor/), both of which contain the same tags
41-
as the above AWS ECR repository. It is also possible to use the `latest` tag, which points to the highest available tag. However,
39+
For a list of available tags, see the [Generic Extractor GitHub repository](https://github.com/keboola/generic-extractor/). It is also possible to use the `latest` tag, which points to the highest available tag. However,
4240
we recommend that you configure your component with a specific tag and update it manually to avoid problems with breaking changes
4341
in future Generic Extractor releases.
4442

extend/generic-extractor/running.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -75,11 +75,11 @@ configuration you want to execute. For example:
7575

7676
Then run Generic Extractor in the current directory by executing the following command on *nix systems:
7777

78-
docker run -v ($pwd):/data quay.io/keboola/generic-extractor:latest
78+
docker run -v ($pwd):/data 147946154733.dkr.ecr.us-east-1.amazonaws.com/developer-portal-v2/ex-generic-v2:latest
7979

8080
or on Windows:
8181

82-
docker run -v %cd%:/data quay.io/keboola/generic-extractor:latest
82+
docker run -v %cd%:/data 147946154733.dkr.ecr.us-east-1.amazonaws.com/developer-portal-v2/ex-generic-v2:latest
8383

8484
You should see:
8585

overview/index.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,7 @@ The following chart shows how Keboola is structured. All Keboola parts are brief
2121
Everything you can do in the Keboola UI can be done programatically using the API of the corresponding component.
2222
All of our components have API documentation on [Apiary](https://keboola.docs.apiary.io/#) and
2323
most of them have a public [Github repository](https://github.com/keboola/).
24-
Our Docker components are built either on [DockerHub](https://github.com/keboola/), [Quay](https://quay.io/organization/keboola) or privately on [AWS ECR](https://aws.amazon.com/ecr/).
24+
Our Docker components are built on [AWS ECR](https://aws.amazon.com/ecr/).
2525

2626
This means that there are virtually **endless possibilities of what can be done with Keboola programmatically**.
2727

0 commit comments

Comments
 (0)