Upgrade to NumPy 2.1

[MaximilianHess]

Problem description

We're planning to use NumPy 2.1 in our project due to its performance improvements. However, Gensim currently depends on NumPy 1.26.4. Are there any plans to update Gensim's dependencies to support NumPy 2.1? Currently, attempting to use Gensim with NumPy 2.1 results in a failure.

[mpenkov]

Closing as duplicate of #3535

[gojomo]

#3535 was a quickie investigational PR into the effort required for Numpy 2.x support – & it might not get much attention. It's probably not the right place to discuss project plans, if any, for Numpy 2.x support - an issue which might spawn many other PRs, bug reports, &
frequent-questions.

[mpenkov]

OK, let's make this the discussion place instead of #3535.

[hechth]

@mpenkov @YoungMind1 and @piskvorky I've had a quick look at the numpy update and building the package seems fine and only few tests are broken. I'll give it a go tomorrow and see if I can fix the tests.

[knevan]

I'm also having trouble with my project because it uses spacy which blis and thinc use numpy>2.0 while gensim is still with numpy 1.26.4.

[melanopsis]

Just a quick note that numpy v2.2 is now also in the wild.

https://numpy.org/news/#releases

[celestinoxp]

and add support to scipy 1.15

[Amato789]

We're using gensim in our project and need compatibility with NumPy 2.X.

Is there any update or plan regarding NumPy 2.x support in gensim?

[julianpollmann]

Hey @hechth @mpenkov @piskvorky,
I fixed most of the things related to numpy 2 based on PR 3562 and will open a new PR.
However test for Ubuntu / Python 3.9 (only 3.9!) / nmslib will fail in most_similiar(). For Python >3.9 tests with nmslib are skipped. nmslib itself is on numpy 1.

How should we tackle this? Is removing NMSLibIndexer an option?

[piskvorky]

Yes, removing NMSLibIndexer is an option.

@mpenkov WDYT? Is it time for a new release, assuming @julianpollmann fixes the numpy 2 build?

[gojomo]

I see that while Python 3.9 is still receiving official security fixes, it's fallen out of bugfix support (and reaches eol In just a few months: October 2025). While Numpy-2.0.0 supported Python 3.9, Numpy-2.1.0 in August 2024 dropped Python 3.9 support.

So I would suggest for simplicity that Gensim's first Numpy-2.x release assume at least Numpy-2.1, and at least Python 3.10.

Even though I don't know of any significant new Gensim features of the sort that would normally justify a 'major' version increment, it still might be worth revving the major digit to '5' as a clear demarcation line: "To use Numpy-2.1+ you must use Gensim-5.x+, and if you must use Numpy-1.x you must use a Gensim 4.x (or earlier)."