CI: Update used actions, add weekly schedule#250
Conversation
A bit of CI maintenance: - Updates all the used actions (checkout, setup-python, cibuildwheel) to their latest versions. - Clean up the Ubuntu build matrix and add multiple Python versions - Adds a weekly scheduled run to all CI workflows, which run each Monday at 06:00 UTC. A periodically scheduled run can detect errors and warnings appearing in changes in upstream dependencies or the build environment. By having a scheduled run they can be traced back easier to a dependency or environment change, then if they would pop up in a random PR.
instead of libspatialindex-c4v5
Do you have any cycles to watch this? I typically do not mess with CI until I'm working on Rtree issues, which is only about once or twice per year. If there is someone who wishes to step forward to take a more active maintenance role for the project, I would gladly hand it off. |
|
Especially if you leave it alone for months it's very useful, since when you come back you can directly see if the CI is broken and if so when. The when tells you the difference between packages or environments updated that week, which helps with quickly pinpointing what broke the build. It's like bisection but you don't have to do it yourself :) |
|
There was one small thing I was hoping for: With the update to cibuildwheel, Python 3.11 wheels are now build by default! And the wheels build successfully, so that means rtree already supports Python 3.11. Python 3.11 is now in release candidate phase, which means that wheels can be uploaded to PyPI (and is encouraged). NumPy, SciPy and Pandas already have Python 3.11 wheels on PyPI, among others. That means the only thing that needs to be done to get Python 3.11 wheels on PyPI is merging this PR and tagging a new release! Since many geospatial packages use rtree, this is really awesome, since they now can start working on Python 3.11 support before it's released! |
|
For tracking issues with upstream dependencies, I've been using dependabot and really enjoying it. It can manage both pip dependencies and GitHub actions. It automatically opens a PR any time any dependency has a new release, and can be similarly configured to only check once per week. |
|
Dependabot is great! There are many other this that can change in a CI environment however. For example:
So having CI somewhat periodically running helps errors from piling up between runs. What do you think about the Python 3.11 wheels? |
How often does this happen? I would have figured compiler versions are locked to OS versions.
We should probably use
I recently hit this with mypy and the latest Python patch version which broke numpy support. So I know what you mean.
This is why the GitHub Actions workflow syntax docs recommend using commit hashes or full version identifiers. This becomes annoying to manually update unless you use dependabot to track it though.
All dependencies should be strictly defined. Are we missing anything? To be clear, I'm not opposed to anything in this PR, just saying that dependabot can also be used to achieve many of these same goals.
Sounds great! Exciting to see rtree getting ahead of the curve. |
About once or twice a year for major versions.
I think using the latest is fine, one of the purposes of CI is to check if things work well in new/updated/different environments. Sometimes it's better for the CI to break than your software to break for users.
Personally I allow things to break time to time, that way you get better insight in what are weaknesses of your software. If a certain part needs fixing often, there might be something else going on (the design for example).
Depending on your perspective, objectives and procedures for CI :).
Great to hear! In that case I say let's merge and tag a v1.0.1 or v1.1.0 release to get those Python 3.11 wheels going! |
|
I'm fine with this, although probably don't have cycles to watch this. I'd also be willing to set up dependabot (in addition to this PR), but probably don't have cycles to watch that either. |
|
Awesome! Is another review needed? Or can you merge this? And what to do think of tagging a new release to get Python 3.11 wheels on PyPI? |
|
I'll let @hobu decide, I'm a relatively new maintainer and don't have access to PyPI so can't make releases anyway. |
|
1.0.1 released. https://pypi.org/project/Rtree/ |
|
New type hints are already catching problems in my code: https://github.com/microsoft/torchgeo/actions/runs/3230077395/jobs/5288108730 |
You're welcome? 😉 |
|
Of course the first week this build fires it fails due to arm64 timeouts. If it fails like that it is going to get ignored anyway. |
3 participants
A bit of CI maintenance:
The updated cibuildwheel version will also build Python 3.11 wheels for the next release.