Skip to content

ZeroDevisionError when using 0<dask_distributed_local_core_fraction<1 on machine with one CPU #32

@observingClouds

Description

@observingClouds

While writing a CI test, I realize that the following fails when using a dask_distributed_local_core_fraction other than 0 or 1 if the machine has only one core (like the standard GitHub CI compute).

n_system_cores = os.cpu_count()
# compute the number of cores to use
n_local_cores = int(args.dask_distributed_local_core_fraction * n_system_cores)
# get the total system memory
total_memory = psutil.virtual_memory().total
# compute the memory per worker
memory_per_worker = (
total_memory / n_local_cores * args.dask_distributed_local_memory_fraction
)

Even though this case will hardly ever be an issue for our computations, I suggest to replace int() with numpy.ceil() or raise a specific warning. Currently one just gets a ZeroDevisionError.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions