MAINT: Ensure nightly jobs use newer packages as required

[david-cortes-intel]

Description

With scikit-learn 1.8, some new tests for array API functionalities were added which require newer versions of package array-api-compat that are not available for older python versions.

This PR updates the python version used for the nightly jobs that run against the main branch of sklearn.

Note that this doesn't actually solve the test failures, just shifts them to a different error, but the change is nevertheless required.

An attempt to deselect these tests on older python versions is handled in a different PR: #2803

---

Checklist:

Completeness and readability

• I have commented my code, particularly in hard-to-understand areas.
• Git commit message contains an appropriate signed-off-by string (see CONTRIBUTING.md for details).
• I have resolved any merge conflicts that might occur with the base branch.

Testing

• All CI jobs are green or I have provided justification why they aren't.

[david-cortes-intel]

/azp run Nightly

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

Flag	Coverage Δ
azure	80.43% <ø> (-0.06%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.
see 1 file with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[david-cortes-intel]

/azp run Nightly

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[david-cortes-intel]

The failing tests that now have a different error message are due to incorrect enabling of array API support in scipy, should be fixed in this PR:
#2806