Skip to content

Use dev images for independent cluster testing #518

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
charlesbluca merged 4 commits into from
May 16, 2022
Merged

Use dev images for independent cluster testing #518

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
2a1e95d
Switch to dask dev images
charlesbluca May 11, 2022
baa39b2
Use mamba for conda installs in images
charlesbluca May 11, 2022
3dfe756
Remove sleep call for installation
charlesbluca May 11, 2022
07e5542
Use timeout / until to wait for cluster to be initialized
charlesbluca May 16, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions .github/docker-compose.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,15 +3,16 @@ version: '3'
services:
dask-scheduler:
container_name: dask-scheduler
image: daskdev/dask:latest
image: daskdev/dask:dev
command: dask-scheduler
ports:
- "8786:8786"
dask-worker:
container_name: dask-worker
image: daskdev/dask:latest
image: daskdev/dask:dev
command: dask-worker dask-scheduler:8786
environment:
USE_MAMBA: "true"
EXTRA_CONDA_PACKAGES: "pyarrow>1.0.0" # required for parquet IO
volumes:
- /tmp:/tmp
4 changes: 2 additions & 2 deletions .github/workflows/test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -173,8 +173,8 @@ jobs:
run: |
docker-compose -f .github/docker-compose.yaml up -d

# Wait for installation
sleep 40
# periodically ping logs until a connection has been established; assume failure after 2 minutes
timeout 2m bash -c 'until docker logs dask-worker 2>&1 | grep -q "Starting established connection"; do sleep 1; done'

docker logs dask-scheduler
docker logs dask-worker
Expand Down