Add tiny and small build images #259
Conversation
maribu
force-pushed
the
tinybuild
branch
3 times, most recently
from
43fa08c to
90b407a
Compare
maribu
force-pushed
the
tinybuild
branch
3 times, most recently
from
ef51e5e to
157cc19
Compare
|
Nice! Any thoughts already on how we could integrate this into our CI? Having separate docker images is certainly less straightforward. But even summing all of them up into a single container would end up with way less than what we currently have (maybe also just because some tools for, e.g., static tests are not included?). In any case I'd prefer not having both the old debian-based container and the new alpine-based container(s) around in the future to avoid confusion. What would be currently still missing to do the switch? Or are those mostly meant to be used for local development instead of CI? |
|
A single container that would contain all listed toolchains is very much possible and ends up being about 2 GiB. I can also add the tools needed for static tests. I think we would end up with about 2.5 GiB then. However, this will not be able to replace the current image:
So from the RIOT's PoV, those images will probably remain a local option for users to use with That said, using them in a CI is for downstream projects highly interesting. The typical downstream project consists of one (or a few) boards, one (or a few) modules, and a RIOT git submodule. E.g. my business card project could very much make use of the I have some interest in maintaining small containers capable for powering the CI of a downstream project that only needs to build for a single arch. |
| @@ -0,0 +1,28 @@ | |||
| #include <stdio.h> | |||
There was a problem hiding this comment.
Do we also "enforce" Copyright headers for this kind of files?
There was a problem hiding this comment.
I could add one. I don't think that I would actually be able to claim authorship for that file anyway, as it is probably too trivial for that. But adding it just in case won't hurt.
|
By the way... how do you test these locally? They aren't pushed to I played around with it yesterday but didn't get very far. I'm not super experienced with Docker... |
|
You can label them to e.g. like this: and then use the |
|
I got it working with the aforementioned changes and my PR applied with the following modification: diff --git a/makefiles/docker.inc.mk b/makefiles/docker.inc.mk
index fee33ac013..8ec546c8a0 100644
--- a/makefiles/docker.inc.mk
+++ b/makefiles/docker.inc.mk
@@ -8,7 +8,7 @@
DOCKER_TESTED_IMAGE_REPO_DIGEST := 08fa7da2c702ac4db7cf57c23fc46c1971f3bffc4a6eff129793f853ec808736
# "DOCKER_IMAGE_VARIANT" is a placeholder that is substituted by `$(DOCKER_IMAGE_VARIANT)`
-DOCKER_PULL_IDENTIFIER := docker.io/riot/DOCKER_IMAGE_VARIANT@sha256:$(DOCKER_TESTED_IMAGE_REPO_DIGEST)
+DOCKER_PULL_IDENTIFIER := local/DOCKER_IMAGE_VARIANT
export DOCKER_BUILD_ROOT ?= /data/riotbuild
DOCKER_RIOTBASE ?= $(DOCKER_BUILD_ROOT)/riotbaseVery nice :) |
|
For reference these are the commands I used: |
|
https://github.com/RIOT-OS/riotdocker/actions/runs/19107811806 This is why the Build test is not executed, but I'm not sure why? Perhaps this needs a rebase, the only difference I could find was this line: riotdocker/.github/workflows/build.yml Line 33 in 6662ca1 and the versions: riotdocker/.github/workflows/build.yml Lines 40 to 41 in 6662ca1 |
maribu
force-pushed
the
tinybuild
branch
2 times, most recently
from
7794c27 to
0f08a73
Compare
This adds Alpine based images intended for use with
make BUILD_IN_DOCKER=1 DOCKER_IMAGE=docker.io/riot/tinybuild-<ARCH>
This adds the following images:
- tinybuild-native64: C toolchain needed to build RIOT apps for `native64`
- smallbuild-native64: C, C++, and rust toolchain needed to build RIOT apps for `native64`
- tinybuild-arm: C toolchain needed to build RIOT apps for ARM7 & ARM Cortex M boards
- smallbuild-arm: C, C++ and rust toolchain libc needed to build RIOT apps for ARM7 & ARM Cortex M boards
- tinybuild-avr: C and C++ toolchain and AVR libc needed to build RIOT apps for AVR boards
- tinybuild-msp430: C toolchain needed to build RIOT apps for MSP430 boards
- smallbuild-msp430: C and C++ toolchain needed to build RIOT apps for MSP430 boards
- tinybuild-risc-v: C toolchain needed to build RIOT apps for RISC-V boards
- smallbuild-risc-v: C, C++, and rust toolchain needed to build RIOT apps for RISC-V boards
Co-authored-by: crasbe <[email protected]>
|
I just realized that However, I had to disable C++ on MSP430 a while ago, see https://gcc.gnu.org/bugzilla/show_bug.cgi?id=119953 I'm retesting now to see if the bug has been fixed as side-effect of something. But I don't have high hopes. In the worst case, |
|
Nope, still no C++ for MSP430: |
crasbe
left a comment
crasbe
left a comment
There was a problem hiding this comment.
This needs an additional case in the image handling, C++ for MSP430 needs to fall back to riotbuild then.
Or is there no C++ on MSP430 at all?
Due to the bug in https://gcc.gnu.org/bugzilla/show_bug.cgi?id=119953 we cannot build g++ for MSP430 right now.
Older GCC releases still build g++ for MSP430. I think the issue might get solved in upstream at some point in time. Until then, |
That's ironic. No space left for tinybuild 😅 On which machine is that running? 🤔 |
|
On a Github runner. It does make sense that those runners don't have unlimited storage. We mighty be able to split the tasks somehow, so that tinybuild related build tasks and functionality tests are done on independent workers? |
Sounds like a good idea 👍 |
On Debian bookworm (oldstable) without diff --git a/tinybuild-arm/Dockerfile b/tinybuild-arm/Dockerfile
index b09d2d6..0a729f9 100644
--- a/tinybuild-arm/Dockerfile
+++ b/tinybuild-arm/Dockerfile
@@ -5,7 +5,7 @@ LABEL maintainer="Marian Buschsieweke <[email protected]>"
RUN \
--mount=type=cache,id=apk-cache,sharing=locked,target=/var/cache/apk \
- --mount=type=bind,target=/tinybuild-apks,source=/output,from=tinybuild-apks \
+ --mount=type=bind,target=/tinybuild-apks,source=/output,from=local/tinybuild-apks \
apk add \
newlib-arm-none-eabi \
picolibc-arm-none-eabi
diff --git a/tinybuild-base/Dockerfile b/tinybuild-base/Dockerfile
index e59f1ae..ef58927 100644
--- a/tinybuild-base/Dockerfile
+++ b/tinybuild-base/Dockerfile
@@ -4,7 +4,7 @@ LABEL maintainer="Marian Buschsieweke <[email protected]>"
RUN \
--mount=type=cache,id=apk-cache,sharing=locked,target=/var/cache/apk \
- --mount=type=bind,target=/tinybuild-apks,source=/output,from=tinybuild-apks \
+ --mount=type=bind,target=tinybuild-apks,source=/output,from=local/tinybuild-apks \
echo "@riotapks /tinybuild-apks/riotapks" >> /etc/apk/repositories && \
cp /tinybuild-apks/*.rsa.pub /etc/apk/keys/ && \
apk update && \before building with |
|
To run tests with diff --git a/tinybuild-native64/Dockerfile b/tinybuild-native64/Dockerfile
index 26447d0..fdc5b02 100644
--- a/tinybuild-native64/Dockerfile
+++ b/tinybuild-native64/Dockerfile
@@ -5,9 +5,10 @@ LABEL maintainer="Marian Buschsieweke <[email protected]>"
RUN \
--mount=type=cache,id=apk-cache,sharing=locked,target=/var/cache/apk \
- --mount=type=bind,target=/tinybuild-apks,source=/output,from=tinybuild-apks \
+ --mount=type=bind,target=/tinybuild-apks,source=/output,from=local/tinybuild-apks \
apk add \
libucontext-dev@riotapks \
gcc \
musl-dev \
- linux-headers
+ linux-headers \
+ py3-pexpect |
|
I've just went ahead and ran all tests for See the full list of build and test failures below. The test failures are mostly due to missing python packages, namely Regarding build failures, we have:
So, in short:
Failures during compilation:
Failures during test:
|
Turns out |
3 participants
Add tiny build images
This adds Alpine based images intended for use with
This adds the following images:
native64native64For comparison, this is the size:
I tested to build
examples/basic/defaultfor one board with each docker image successfully.