Test against NumPy 2.0

[jpivarski]

Ruff has tools for this, and there are NumPy 2.0 prereleases (or there will be soon).

[jpivarski]

https://numpy.org/devdocs/release/2.0.0-notes.html

https://pythonspeed.com/articles/numpy-2/

[jpivarski]

NumPy 2.0's release notes serve as a migration guide, and their umbrella issue, numpy/numpy/issues/24300, is a good source of news.

[jpivarski]

The development wheels are here: https://anaconda.org/scientific-python-nightly-wheels/numpy!

[rgommers]

Some potential concern was raised at pybind/pybind11#5050 (comment) about Awkward Array using structured dtypes, which are a potential hiccup in pybind11's support for numpy 2.0. It's unclear if that risk will materialize. Have you done any testing with numpy 2.0 nightlies already @jpivarski?

[agoose77]

Hmm, I'm not sure what @henryiii is exactly saying in pybind/pybind11#5050. AFAICR, we don't inherently use any C structs via NumPy structured dtypes. But, perhaps I'm forgetting. @jpivarski might recall.

I believe I've tested locally against a NumPy 2.0 pre/dev-release, and didn't spot anything.

[henryiii]

The change only landed in NumPy quite recently from what I understand. If you didn’t have to use a pybind11 branch, it shouldn’t have worked, I believe (regardless of if any custom dtypes are used). It might be that all structured types are handled by the forth layer?

Anyway, I’ll try to test with boost-histogram, which uses structured dtyoes for the weighed and mean storages.

[rgommers]

The change only landed in NumPy quite recently from what I understand.

Indeed, last Thursday or so.

If you didn’t have to use a pybind11 branch, it shouldn’t have worked, I believe (regardless of if any custom dtypes are used)

pybind11 2.11.1 works just fine for SciPy and Matplotlib right now - it just emits verbose warnings about pybind11 digging into private NumPy internals that have moved, but that doesn't break the build (at least not yet - it will at some point).

[jpivarski]

We have not tested with the nightly NumPy releases, and clearly we need to.

On the point about structured dtypes, we don't use them in any pybind11 calls. The closest that Awkward Array gets to structured dtypes is that this is an output type of ak.to_numpy if an array is a RecordArray, but they're not used internally at all, least of all in compiled code.