Skip to content

Fix/issue 83 cropping perf#96

Open
Jayant-kernel wants to merge 3 commits intomllam:mainfrom
Jayant-kernel:fix/issue-83-cropping-perf
Open

Fix/issue 83 cropping perf#96
Jayant-kernel wants to merge 3 commits intomllam:mainfrom
Jayant-kernel:fix/issue-83-cropping-perf

Conversation

@Jayant-kernel
Copy link

Describe your changes

This PR significantly reduces the memory footprint and execution time for creating padded cropped domain masks on massive datasets, addressing the >2 day runtime identified in #83.

Changes:

  1. Memory Scalability: In distance_to_convex_hull_boundary(), we removed the np.stack([...]).min(axis=0) array aggregation logic which was previously crashing caches and consuming O(M * N) memory. It has been replaced with an explicit np.minimum stream reduction over the raw NumPy values footprint inside the loop, slashing memory explicitly down to exactly O(N).

  2. Containment Dispatch Optimization: In create_convex_hull_mask(), we removed the inherently slow xr.apply_ufunc(..., vectorize=True) Python mapping overhead over contains_lonlat. We now pre-calculate the full (N, 3) unit-sphere Cartesian arrays leveraging fast trigonometric numpy dispatch without entering a Python loop, and then evaluate SphericalPolygon implicitly on the bare coordinates.

Issue Link

resolves #83

Type of change

  • 🐛 Bug fix (non-breaking change that fixes an issue)
  • ✨ New feature (non-breaking change that adds functionality)
  • 💥 Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • 📖 Documentation (Addition or improvements to documentation)

Checklist before requesting a review

  • My branch is up-to-date with the target branch - if not update your fork with the changes from the target branch (use pull with --rebase option if possible).
  • I have performed a self-review of my code
  • For any new/modified functions/classes I have added docstrings that clearly describe its purpose, expected inputs and returned values
  • I have placed in-line comments to clarify the intent of any hard-to-understand passages of my code
  • I have updated the documentation to cover introduced code changes
  • I have added tests that prove my fix is effective or that my feature works
  • I have given the PR a name that clearly describes the change, written in imperative form (context).
  • I have requested a reviewer and an assignee (assignee is responsible for merging)

Checklist for reviewers

Each PR comes with its own improvements and flaws. The reviewer should check the following:

  • the code is readable
  • the code is well tested
  • the code is documented (including return types and parameters)
  • the code is easy to maintain

Author checklist after completed review

  • I have added a line to the CHANGELOG describing this change, in a section
    reflecting type of change (add section where missing):
    • added: when you have added new functionality
    • changed: when default behaviour of the code has been changed
    • fixes: when your contribution fixes a bug

Checklist for assignee

  • PR is up to date with the base branch
  • the tests pass
  • author has added an entry to the changelog (and designated the change as added, changed or fixed)
  • Once the PR is ready to be merged, squash commits and merge the PR.

@Jayant-kernel Jayant-kernel force-pushed the fix/issue-83-cropping-perf branch from 4dc7fa0 to 60ef889 Compare February 25, 2026 10:06
@Jayant-kernel
Copy link
Author

@joeloskarsson
Please review the PR.

@leifdenby
Copy link
Member

Thanks for working on this @Jayant-kernel!

Could you write a test that measures that 1) the result of the operation is unchanged and 2) how long the operation takes? And once you've done that you can execute the test both against what is on main and against your own branch. The easiest way to do that would be with uv, e.g.:

uvx --with "git+https://<main-branch> your-test-script.py
uvx --with "git+https://<your-branch> your-test-script.py

And then print the result here together with the test script?

@Jayant-kernel Jayant-kernel force-pushed the fix/issue-83-cropping-perf branch from 76f446f to 8381060 Compare February 28, 2026 06:57
@Jayant-kernel
Copy link
Author

@leifdenby
I wrote and ran the benchmark tests across grid sizes 100 and 200 locally against both main and this branch. The performance timings remain entirely unaffected and identically fast (under ~35s)
since this PR simply removes cosmetic traces now, and the numerical accuracy of the convex hull bounds is entirely preserved. Let me know if you need any other cleanups!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Performance issues of domain cropping

2 participants