Cannot (easily) build huge images.

[bryan-lunt]

My Docker images are huge, and I keep getting out of space errors when I try to build with the basic docker2singularity commandline.

I know I can probably (on my own machine anyway) allocate more space for my Docker VM so it would have a larger /tmp slice, but that's not always possible.

This has lead me to write a bash script to setup various mountpoints for docker2singularity.

It would be nice if some of these things could be better documented, it would also be nice if the docker2singularity.sh script had more control over where its temporary directories go.

Here is the most important part of what I have so far, but it still doesn't work.

docker run -v /var/run/docker.sock:/var/run/docker.sock \
-v ${OUTPUT_DIRECTORY}:/output \
-v ${TMP_FOR_BUILD}:/tmp -e SINGULARITY_TMPDIR=/tmp/buildtmp \
-v ${TMP_FOR_BUILD}/singcache:/tmp/singcache -e SINGULARITY_CACHEDIR=/tmp/singcache \
-v ${TMP_FOR_BUILD}/tmpother:/tmp/tmpother -e TMP=/tmp/tmpother -e TEMP=/tmp/tmpother -e TEMPDIR=/tmp/tmpother \
--privileged -t --rm \
quay.io/singularity/docker2singularity \
${BUILD_IMAGE}

[vsoch]

It’s generated via /tmp in the image you have total control of binding that somewhere on your host, as you do with the output folder. I’m not sure what you are asking or suggesting but if you test a new variable in the script that you think would be useful happy to discuss.

And generally I’d recommend these days pulling Docker images directly with Singularity - this script was made before that was supported.

[bryan-lunt]

I'll see if I can build the image on the machine I actually want to run on.

[vsoch]

Sure thing! And please try singularity pull docker://<docker-uri> it really is intended for this use case.

[bryan-lunt]

Looks like my sysadmins have not included mksquashfs for us.

While making image from oci registry: error fetching image to cache: while building SIF from layers: unable to create new build: while searching for mksquashfs: exec: "mksquashfs": executable file not found in $PATH

I know I can run singularity images on the cluster, but I have trouble building them there. A reason I had wanted to build them from within a Docker container is to make it easier for future TAs (Teaching Assistants) to deploy newer containers without forcing them to do much installation on their local machines. (Nor to trust campus IT to give us what we need.)

[bryan-lunt]

Could I build a Singularity image from inside a Singularity image?

[vsoch]

Ug, I'm sorry about that! I would reach out to them and report the error and have a conversation about Singularity. This is a conversation people were having years ago (2016 / 2017) and I'm sorry that your cluster doesn't seem to have real support for it. If they say that it's installed, then perhaps ask them how you can pull an image from Docker Hub without that? Perhaps they expect you to pull locally, and then scp onto the cluster to use? It's annoying but it could possibly work.

Could I build a Singularity image from inside a Singularity image?

You could try it, but at least when I tried a few years back, it won't work because of the permissions required.

If the sysadmins aren't supportive of Singularity then you can ask them about Podman, which is another rootless container technology (more like docker) that is being better embraced at national labs. If they are dismissive to both then you probably need to escalate to someone in management there or a higher up in a department because not being able to run HPC friendly containers on a cluster in 2022 is really not acceptable.

[bryan-lunt]

Well, it looks like one of the older singularity installations on the cluster can build, so I've got my issues sorted.
Thank you.