Chectl function to fetch the list of needed images and download them (performance tuning, OCP 3.11 airgap)

[MichalMaler]

Is your enhancement related to a problem? Please describe.

Right now we have a table that should list the recommend list of images with digest for image puller to pre-pull:
https://www.eclipse.org/che/docs/che-7/administration-guide/caching-images-for-faster-workspace-start/#deploying-image-puller-using-openshift-templates_caching-images-for-faster-workspace-start

This table is almost identical to the table of images needed for an installation in restricted environments.
So, the problem is to, from the doc perspective, maintain two almost identical tables in two files that are also used in downstream, but with different images.

1. These tables are frequently updated and their keeping up is problematic.

2. The whole section is complex and instead of having multiple procedures about how to get those images pulled (where some are pulled by imagepuller, some have to be downloaded by skopeo), it would be better to have this "in code".
   https://www.eclipse.org/che/docs/che-7/administration-guide/caching-images-for-faster-workspace-start/

Describe the solution you'd like

I would like to request a script that would fetch the list of needed images, download them, provide the list of remaining images from which a user can pick and download, based on a user's use-case.

This script would be a new feature in chectl. It would be great if it could have two options:
a) download images for faster caching
b) download images for use in restricted environments

Describe alternatives you've considered

Doc alternative:

1. Reduce Table 3. List of recommended images to pre-pull to list ONLY the 2 brokers and Che-Theia.

• add attributes for versions to keep this up to date. Latest brokers can be found from either https://github.com/eclipse/che-plugin-broker/tags

• or by checking tags on https://quay.io/repository/eclipse/che-plugin-metadata-broker?tab=tags

• tags for theia can be found here: https://github.com/eclipse/che-theia/tags or https://quay.io/repository/eclipse/che-theia?tab=tags

2. add a section about curling the whole list of images (with a note that some might not be pullable this way, like machine-exec)

3. Add a note that if the image puller can't pull images and keep them alive (no sleep command, etc.), then the backup process is to simply use skopeo copy --all to mirror images into your cluster's image registry; this is ALSO how to do airgap setup for OCP 3.11, so might be able to reuse content/link to other section

Additional context

https://issues.redhat.com/browse/RHDEVDOCS-2289

[MichalMaler]

@themr0c @nickboldt @rhopp

[nickboldt]

Not sure how these are different:

a) download images for faster caching
b) download images for use in restricted environments

Is the intent that the new chectl server:images command (I'm guessing that this new command would go under server since it's related to deployments and server administration):

• would read the CSV data embedded in chectl (or pull from an external URL provided via --starting-csv=starting-csv, the same flag used with server:start), then
• skopeo copy --all those images into the connected OCP cluster's image registry, using the SAME digest names (and maybe optionally could use flags instead?)

If so, then the cached images would be effective for both online/offline & airgapped/non-airgapped environments. So (a) and (b) are the same solution for different needs.

Note too that for OCP 4.3+, this functionality is already provided by OCP itself via OLM, so this is really only a requirement for OCP 3.11 and maybe for k8s deployments (?).

Or if there's a way to make triggering the OLM mirroring easier via chectl, then perhaps we would want two paths inside chectl server:images - one that calls the OLM API and one that does skopeo copy --all for OCP 3.11.

WDYT @tolusha ?

[MichalMaler]

Yeah, that's right, Nick. I am not a PRO, so that's why I tagged you :) Fabrice is the author of this idea, btw, so all the glory and fame -> @themr0c :)

[themr0c]

@nickboldt I believe that would be really a good point to have one single command able to fetch images for both user stories.

[themr0c]

• New features in the Operator will solve the problem for image puller.