Newer `scipy` version triggering `UserWarning`

[AVHopp]

The newest version of scipy contains a new optimization shortcut for the case of lower and upper bounds being equal. This procedure passes a non-writable array without copying it. As botorch calls torch.from_numpy on this array, a UserWarning is triggered. This causes our current tests to fail, see #843

Proposed solution:

• Potentially silence the error temporarily?
• Just accept that issue for now as it only concerns a single test
• Create PRs at scipy/botorch for fixes: Turns out scipy is sufficient
• Wait for fix on scipy being accepted and merged: BUG: Copy non-writable array when all bounds are equal to prevent UserWarning scipy/scipy#25541

Opinions on whether or not to temporarily fix @AdrianSosic @Scienfitz @kalama-ai ?

[AVHopp]

See here for an MWE creating this issue:

import numpy as np
import torch
from scipy.optimize import minimize
from scipy.optimize._optimize import MemoizeJac


def objective(x):
    t = torch.from_numpy(x)  # warns: x is non-writable
    return float((t**2).sum()), 2 * x


fun = MemoizeJac(objective)

# All bounds equal => scipy 1.18 short-circuits via
# _optimize_result_for_equal_bounds and passes a non-writable x0
x0 = np.array([1.0, 2.0])
bounds = [(1.0, 1.0), (2.0, 2.0)]

minimize(fun, x0, jac=fun.derivative, method="L-BFGS-B", bounds=bounds)

which produces

UserWarning: The given NumPy array is not writable, and PyTorch does not support non-writable tensors. This means writing to this tensor will result in undefined behavior. You may want to copy the array to protect its data or make it writable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at /Users/runner/work/pytorch/pytorch/torch/csrc/utils/tensor_numpy.cpp:219.)
  t = torch.from_numpy(x)  # warns: x is non-writable

[Scienfitz]

• which baybe line/concept triggers this?
• why is it triggered in coretest and not fulltest?

[AVHopp]

This only happens for scipy>=1.18. Full tests installed a lower version 1.17.1 while core tests installed 1.18.0 (please don't ask me why), but this is the reason for that.

This is currently trigger by one of the cardinality tests - probably because we create an edge case situation here by setting bounds to the same values (or botorch doing that internally), would need to dig deeper into that. This is the exact error:

FAILED tests/constraints/test_subset_constraints_hybrid.py::test_subset_constraints_hybrid[continuous_cardinality] - UserWarning: The given NumPy array is not writable, and PyTorch does not support non-writable tensors. This means writing to this tensor will result in undefined behavior. You may want to copy the array to protect its data or make it writable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at /pytorch/torch/csrc/utils/tensor_numpy.cpp:213.)

[Scienfitz]

but that then is an indication that even fixing it upstream might not help us here in practice because the newest versions are not necessarily installed so even after going through the (often lengthy) process of PR and release of another package it might not be enough. Enforcing a version bump due to this minor warning is also not acceptable

(Im not saying this shouldnt be fixed upstream, it just will not guarantee us a solution to this issue of failing CI)

[AVHopp]

Well, it will be a fix at some point because we will probably (at least implicitly) only support sufficiently high versions.

What is the alternative then, and isn't this then a general and larger issue with any issues that requires fixing in the packages we consume? Easiest way to solve that would probably be an xfail if the installed scipy version is below 1.18.0, or do you see another alternative?

[Scienfitz]

we first need to think whether its even reasonable to create such situation, hence I was asking for the code block that creates this issue on the baybe side

[AVHopp]

Will then deep dive into this again later and share more specific information here, I do not know this from the top of my head right now but need to re-investigate anyway.

[Scienfitz]

Since the upstream issue seems related to upper and lower bounds: I'm mainly wondering: why are we doing/ allowing upper and lower bounds to be equal and use that in a test? Maybe its reasonable due to something but at first thought it seems thats a situation we dont even want in the first place?

[AVHopp]

The test is one of the cardinality tests in a hybrid space. I think the issue is that in this tes, we probably fix all of the continuous variables to have value 0 in the continuous subspace and then optimize over the remaining part. This explains why is only happens in that one hybrid case.

[AVHopp]

Note that the PR is scipy has been merged: scipy/scipy#25541