Skip to content

fix: use math.ceil() instead of int() for n_local_cores to prevent ZeroDivisionError on single-core machines#91

Open
Mani212005 wants to merge 1 commit intomllam:mainfrom
Mani212005:fix/math.ceil
Open

fix: use math.ceil() instead of int() for n_local_cores to prevent ZeroDivisionError on single-core machines#91
Mani212005 wants to merge 1 commit intomllam:mainfrom
Mani212005:fix/math.ceil

Conversation

@Mani212005
Copy link

@Mani212005 Mani212005 commented Feb 20, 2026

Describe your changes

When using --dask-distributed-local-core-fraction with a value between 0 and 1 on a
single-core machine (e.g. standard GitHub CI), the following line:

n_local_cores = int(args.dask_distributed_local_core_fraction * n_system_cores)

truncates towards zero. For example:

int(0.5 * 1) = 0

This causes a ZeroDivisionError when computing memory_per_worker:

memory_per_worker = total_memory / n_local_cores * args.dask_distributed_local_memory_fraction
#                                  ^^^^^^^^^^^^^ ZeroDivisionError when n_local_cores == 0

The fix replaces int() with math.ceil() to always round up to at least 1 worker:

n_local_cores = math.ceil(args.dask_distributed_local_core_fraction * n_system_cores)

This ensures n_local_cores >= 1 regardless of the fraction or number of system cores.

Changes:

  • cli.py — Added import math and replaced int() with math.ceil() for the n_local_cores calculation.
  • test_distributed.py — Added "0.5" as a parametrized test case to cover the fractional value scenario that triggered the bug.

Testing — all 4 distributed tests pass:

tests/test_distributed.py::test_run_distributed[args0]  PASSED  (core-fraction=1.0)
tests/test_distributed.py::test_run_distributed[args1]  PASSED  (core-fraction=0.5) ← new
tests/test_distributed.py::test_run_distributed[args2]  PASSED  (core-fraction=0.0)
tests/test_distributed.py::test_run_distributed[args3]  PASSED  (no dask args)

4 passed in 150.24s

Issue Link

Closes #32

Type of change

  • 🐛 Bug fix (non-breaking change that fixes an issue)
  • ✨ New feature (non-breaking change that adds functionality)
  • 💥 Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • 📖 Documentation (Addition or improvements to documentation)

Checklist before requesting a review

  • My branch is up-to-date with the target branch - if not update your fork with the changes from the target branch (use pull with --rebase option if possible).
  • I have performed a self-review of my code
  • For any new/modified functions/classes I have added docstrings that clearly describe its purpose, expected inputs and returned values
  • I have placed in-line comments to clarify the intent of any hard-to-understand passages of my code
  • I have updated the documentation to cover introduced code changes
  • I have added tests that prove my fix is effective or that my feature works
  • I have given the PR a name that clearly describes the change, written in imperative form (context).
  • I have requested a reviewer and an assignee (assignee is responsible for merging)

Checklist for reviewers

Each PR comes with its own improvements and flaws. The reviewer should check the following:

  • the code is readable
  • the code is well tested
  • the code is documented (including return types and parameters)
  • the code is easy to maintain

Author checklist after completed review

  • I have added a line to the CHANGELOG describing this change, in a section reflecting type of change (add section where missing):
    • added — when you have added new functionality
    • changed — when default behaviour of the code has been changed
    • fixes — when your contribution fixes a bug

Checklist for assignee

  • PR is up to date with the base branch
  • the tests pass
  • author has added an entry to the changelog (and designated the change as added, changed or fixed)
  • Once the PR is ready to be merged, squash commits and merge the PR.

@joeloskarsson
Copy link
Contributor

Hi! Could you reformat this PR using our existing PR template: https://github.com/mllam/mllam-data-prep/blob/main/.github/pull_request_template.md ? This guarantees that we follow the development workflows that we have in place. If you have questions about some parts of it feel free to ask!

@Mani212005
Copy link
Author

@joeloskarsson
Sorry for that! I have corrected the format kindly review the code.
Thank you !

[
["example.danra.yaml", "--dask-distributed-local-core-fraction", "1.0"],
["example.danra.yaml", "--dask-distributed-local-core-fraction", "0.5"],
["example.danra.yaml", "--dask-distributed-local-core-fraction", "0.0"],
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How did this test pass if --dask-distributed-local-core-fraction 0.0 results in division by zero? Seems to potentially point to an issue with the test that should then be fixed here as well.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hello @joeloskarsson, I believe the reason the test passes is due to

args.dask_distributed_local_core_fraction > 0.0:

in the cli.py at 54th line file as when 0.0 is provided the above condition becomes False so the entire
dask block is skipped thus the test passes. But you are right I should have been more clear I will add comments for better clarity.
Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ZeroDevisionError when using 0<dask_distributed_local_core_fraction<1 on machine with one CPU

2 participants