Skip to content

libcnb-test: Consider using --network bridge when running pack build #943

@edmorley

Description

@edmorley

When running the Python CNB's integration tests locally, if I increase the number of concurrent test threads by a lot, I get an error about network resource exhaustion:

$ cargo test -- --ignored --test-threads 40
...
ERROR: failed to build: executing lifecycle: failed to create ephemeral bridge network: Error response from daemon: all predefined address pools have been fully subnetted

This appears to be due this networking change in Pack CLI v0.35.0:
https://github.com/buildpacks/pack/releases/tag/v0.35.0

Run build containers in a separate ephemeral Docker bridge network

...which was done in response to the security review:

And means that for every pack build, pack creates a new temporary bridge network for the build container, which is then deleted after the build.

However, this additional network isolation isn't needed when we run our integration tests, and only serves to:

  1. Cause the resource exhaustion error shown above
  2. Add additional Pack CLI setup/teardown steps for every build (timed locally on my high end M4 Max it adds 200-300ms to each build - which isn't a lot for successful builds, but is a lot for the long tail of tests that test invalid build config type scenarios - which otherwise run in 1-2 seconds)
  3. Add to the list of things that don't get cleaned up properly if I Ctrl-C a pack build (check docker network ls after doing so...)

This behaviour can be changed using pack build's --network option - which supports any of the driver names specified here:
https://docs.docker.com/engine/network/drivers/

Specifically, to return to pre Pack v0.35 behaviour, we could pass --network bridge, which uses the permanent Docker bridge network (as opposed to the ephemeral ones that Pack creates/destroys every time).

This bridge mode is the Docker default when no network is specified, and still isolates the container from the host - just not from other contains on that machine. See also:
https://docs.docker.com/engine/network/

As such, I think libcnb-test should always pass --network bridge - similar to how we also pass other options that help with performance in a testing scenario (such as --trust-extra-buildpacks and --pull-policy if-not-present).

(For bonus points, we could also support optionally overriding the network to none, which would allow for integration testing of connection timeout/retry behaviours for eg our language runtime downloads should we wish.)

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions