Skip to content

Conversation

@Gankra
Copy link
Contributor

@Gankra Gankra commented May 30, 2025

This requires some more tests and almost certainly some error messages, but, I think this is the implementation?

The first commit is a large driveby move of the RequiresPython type to uv-distribution-types to allow us to generate the appropriate markers at this point in the code. It also migrates RequiresPython from pubgrub::Range to version_ranges::Ranges, and makes several pub(crate) items pub, as it's no longer defined in uv_resolver.

Fixes #11606

Comment on lines 148 to 149
// Add the `dev` group, if `dev-dependencies` is defined.
if let Some(dev_dependencies) = dev_dependencies {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: the legacy special tool.uv.dev-dependencies group gets injected post-flattening. Currently this means we don't support tool.uv.dependency-groups.dev.requires-python affecting it. I think this is fine (you can just migrate to dependency-groups.dev if you want to apply requires-python to it).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we error eagerly if you use both?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm... like "you're using Modern dependency-groups features, please migrate off Legacy dependency-groups?" even if there's no actual interaction with how they're using them?

Gankra added a commit that referenced this pull request May 30, 2025
(or legacy tool.uv.workspace).

This cleaves out a dedicated SourcedDependencyGroups type based on RequiresDist
but with only the DependencyGroup handling implemented. This allows `uv pip`
to read `dependency-groups` from pyproject.tomls that only have that table
defined, per PEP 735, and as implemented by `pip`.

However we want our implementation to respect various uv features when they're
available:

* `tool.uv.sources`
* `tool.uv.index`
* `tool.uv.dependency-groups.mygroup.requires-python` (#13735)

As such we want to opportunistically detect "as much as possible" while doing
as little as possible when things are missing. The issue with the old RequiresDist
path was that it fundamentally wanted to build the package, and if `[project]`
was missing it would try to desperately run setuptools on the pyproject.toml
to try to find metadata and make a hash of things.

At the same time, the old code also put in a lot of effort to try to pretend that
`uv pip` dependency-groups worked like `uv` dependency-groups with defaults
and non-only semantics, only to separate them back out again. By explicitly
separating them out, we confidently get the expected behaviour.

Note that dependency-group support is still included in RequiresDist, as some
`uv` paths still use it. It's unclear to me if those paths want this same
treatment -- for now I conclude no.

Fixes #13138
@Gankra Gankra added the enhancement New feature or improvement to existing functionality label May 30, 2025
@konstin
Copy link
Member

konstin commented Jun 2, 2025

I find the current behavior around adding markers to the group members unintuitive:

[project]
name = "debug"
version = "0.1.0"
requires-python = ">=3.9"
dependencies = ["typing-extensions"]

[dependency-groups]
docs = ["sphinx"]

[tool.uv.dependency-groups]
docs = { requires-python = ">=3.12" }

When I now run

uv run -p 3.11 --group docs sphinx-build

it fails to run sphinx, but it doesn't tell me that the docs groups requires a newer Python version. I would have expected this call to fail and report that I need at least Python 3.12. Similar to how the project requires-python works:

$ uv run -p 3.8 --group docs sphinx-build
Using CPython 3.8.18 interpreter at: /home/konsti/.local/bin/python3.8
error: The requested interpreter resolved to Python 3.8.18, which is incompatible with the project's Python requirement: `>=3.9`

@Gankra
Copy link
Contributor Author

Gankra commented Jun 2, 2025

I agree I'd prefer a better diagnostic, zanie mentioned last week that I might need to do some work on... i think they said "resolver hints" or something? Or are you saying lowering to markers is entirely wrong, and I need to be resolving this metadata in a different part of uv?

@konstin
Copy link
Member

konstin commented Jun 2, 2025

My preference is that the requires-python is a mandatory bound, so uv run -p 3.8 --group docs sphinx-build would error. This would match the name requires-python and how project.requires-python works (both for the current project and transitively). If we're implementing the current mechanism of adding this as markers to each requirement, I would choose a different name for it to make the semantic differences with requires-python clearer.

Separately, do want to land baf9895 as a split out PR? r+ me it's a pure refactoring anyway.

@Gankra
Copy link
Contributor Author

Gankra commented Jun 2, 2025

Separately, do want to land baf9895 as a split out PR? r+ me it's a pure refactoring anyway.

The ranges migration is good but if moving it out of the crate is based on me misunderstanding this problem I'd rather not land that part.

@Gankra
Copy link
Contributor Author

Gankra commented Jun 2, 2025

I'll take a closer look at how the top-level requires-python is implemented to better understand what that impl would look like.

@zanieb
Copy link
Member

zanieb commented Jun 2, 2025

Konsti is talking about confirming the selected Python interpreter meets the resolved requires-python range for the group. You'll need to ensure we're taking the fully resolved requires-python request into account when selecting a Python interpreter.

Separately, what I was talking about, is if there are conflicts between the requires-python in multiple groups or with the project.requires-python.

@konstin
Copy link
Member

konstin commented Jun 2, 2025

My preference is pretty much #11606 (comment): I think the current changes are required to get the right resolver behavior (split at the reuires-python version and resolve the dependencies only in the upper split, since e.g. sphinx has a higher requires-python itself and otherwise fails to resolve; Adding the markers should facilitate that), but I think we should also error when the Python version is too low for the group. CC @zanieb wdyt?

@zanieb
Copy link
Member

zanieb commented Jun 2, 2025

Yes, that's correct.

Gankra added a commit that referenced this pull request Jun 12, 2025
(or legacy tool.uv.workspace).

This cleaves out a dedicated SourcedDependencyGroups type based on RequiresDist
but with only the DependencyGroup handling implemented. This allows `uv pip`
to read `dependency-groups` from pyproject.tomls that only have that table
defined, per PEP 735, and as implemented by `pip`.

However we want our implementation to respect various uv features when they're
available:

* `tool.uv.sources`
* `tool.uv.index`
* `tool.uv.dependency-groups.mygroup.requires-python` (#13735)

As such we want to opportunistically detect "as much as possible" while doing
as little as possible when things are missing. The issue with the old RequiresDist
path was that it fundamentally wanted to build the package, and if `[project]`
was missing it would try to desperately run setuptools on the pyproject.toml
to try to find metadata and make a hash of things.

At the same time, the old code also put in a lot of effort to try to pretend that
`uv pip` dependency-groups worked like `uv` dependency-groups with defaults
and non-only semantics, only to separate them back out again. By explicitly
separating them out, we confidently get the expected behaviour.

Note that dependency-group support is still included in RequiresDist, as some
`uv` paths still use it. It's unclear to me if those paths want this same
treatment -- for now I conclude no.

Fixes #13138
Comment on lines 381 to 400
// Running `uv sync --no-dev` should ideally succeed, locking for Python 3.8.
// ...but once we pick the 3.8 interpreter the lock freaks out because it sees
// that the dependency-group containing sphinx will never succesfully install,
// even though it's not enabled!
uv_snapshot!(context.filters(), context.sync()
.arg("--no-dev"), @r"
success: false
exit_code: 1
----- stdout -----
----- stderr -----
Using CPython 3.8.[X] interpreter at: [PYTHON-3.8]
Creating virtual environment at: .venv
× No solution found when resolving dependencies for split (python_full_version == '3.8.*'):
╰─▶ Because the requested Python version (>=3.8) does not satisfy Python>=3.9 and sphinx==7.2.6 depends on Python>=3.9, we can conclude that sphinx==7.2.6 cannot be used.
And because only sphinx<=7.2.6 is available, we can conclude that sphinx>=7.2.6 cannot be used.
And because pharaohs-tomp:dev depends on sphinx>=7.2.6 and your project requires pharaohs-tomp:dev, we can conclude that your project's requirements are unsatisfiable.
hint: The `requires-python` value (>=3.8) includes Python versions that are not supported by your dependencies (e.g., sphinx==7.2.6 only supports >=3.9). Consider using a more restrictive `requires-python` value (like >=3.9).
");
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is now an opportunity for us to emit a Better diagnostic suggesting group-requires-python (let's make this a followup...).

@Gankra Gankra enabled auto-merge (squash) June 13, 2025 21:58
@Gankra Gankra merged commit 5021840 into main Jun 13, 2025
107 checks passed
@Gankra Gankra deleted the gankra/group-python-real branch June 13, 2025 22:04
Gankra added a commit that referenced this pull request Jun 13, 2025
(or legacy tool.uv.workspace).

This cleaves out a dedicated SourcedDependencyGroups type based on RequiresDist
but with only the DependencyGroup handling implemented. This allows `uv pip`
to read `dependency-groups` from pyproject.tomls that only have that table
defined, per PEP 735, and as implemented by `pip`.

However we want our implementation to respect various uv features when they're
available:

* `tool.uv.sources`
* `tool.uv.index`
* `tool.uv.dependency-groups.mygroup.requires-python` (#13735)

As such we want to opportunistically detect "as much as possible" while doing
as little as possible when things are missing. The issue with the old RequiresDist
path was that it fundamentally wanted to build the package, and if `[project]`
was missing it would try to desperately run setuptools on the pyproject.toml
to try to find metadata and make a hash of things.

At the same time, the old code also put in a lot of effort to try to pretend that
`uv pip` dependency-groups worked like `uv` dependency-groups with defaults
and non-only semantics, only to separate them back out again. By explicitly
separating them out, we confidently get the expected behaviour.

Note that dependency-group support is still included in RequiresDist, as some
`uv` paths still use it. It's unclear to me if those paths want this same
treatment -- for now I conclude no.

Fixes #13138
Gankra added a commit that referenced this pull request Jun 13, 2025
#13742)

(or legacy tool.uv.workspace).

This cleaves out a dedicated SourcedDependencyGroups type based on
RequiresDist but with only the DependencyGroup handling implemented.
This allows `uv pip` to read `dependency-groups` from pyproject.tomls
that only have that table defined, per PEP 735, and as implemented by
`pip`.

However we want our implementation to respect various uv features when
they're available:

* `tool.uv.sources`
* `tool.uv.index`
* `tool.uv.dependency-groups.mygroup.requires-python` (#13735)

As such we want to opportunistically detect "as much as possible" while
doing as little as possible when things are missing. The issue with the
old RequiresDist path was that it fundamentally wanted to build the
package, and if `[project]` was missing it would try to desperately run
setuptools on the pyproject.toml to try to find metadata and make a hash
of things.

At the same time, the old code also put in a lot of effort to try to
pretend that `uv pip` dependency-groups worked like `uv`
dependency-groups with defaults and non-only semantics, only to separate
them back out again. By explicitly separating them out, we confidently
get the expected behaviour.

Note that dependency-group support is still included in RequiresDist, as
some `uv` paths still use it. It's unclear to me if those paths want
this same treatment -- for now I conclude no.

Fixes #13138
tmeijn pushed a commit to tmeijn/dotfiles that referenced this pull request Jun 24, 2025
This MR contains the following updates:

| Package | Update | Change |
|---|---|---|
| [astral-sh/uv](https://github.com/astral-sh/uv) | patch | `0.7.13` -> `0.7.14` |

MR created with the help of [el-capitano/tools/renovate-bot](https://gitlab.com/el-capitano/tools/renovate-bot).

**Proposed changes to behavior should be submitted there as MRs.**

---

### Release Notes

<details>
<summary>astral-sh/uv (astral-sh/uv)</summary>

### [`v0.7.14`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#0714)

[Compare Source](astral-sh/uv@0.7.13...0.7.14)

##### Enhancements

- Add XPU to `--torch-backend` ([#&#8203;14172](astral-sh/uv#14172))
- Add ROCm backends to `--torch-backend` ([#&#8203;14120](astral-sh/uv#14120))
- Remove preview label from `--torch-backend` ([#&#8203;14119](astral-sh/uv#14119))
- Add `[tool.uv.dependency-groups].mygroup.requires-python` ([#&#8203;13735](astral-sh/uv#13735))
- Add auto-detection for AMD GPUs ([#&#8203;14176](astral-sh/uv#14176))
- Show retries for HTTP status code errors ([#&#8203;13897](astral-sh/uv#13897))
- Support transparent Python patch version upgrades ([#&#8203;13954](astral-sh/uv#13954))
- Warn on empty index directory ([#&#8203;13940](astral-sh/uv#13940))
- Publish to DockerHub ([#&#8203;14088](astral-sh/uv#14088))

##### Performance

- Make cold resolves about 10% faster ([#&#8203;14035](astral-sh/uv#14035))

##### Bug fixes

- Don't use walrus operator in interpreter query script ([#&#8203;14108](astral-sh/uv#14108))
- Fix handling of changes to `requires-python` ([#&#8203;14076](astral-sh/uv#14076))
- Fix implied `platform_machine` marker for `win_amd64` platform tag ([#&#8203;14041](astral-sh/uv#14041))
- Only update existing symlink directories on preview uninstall ([#&#8203;14179](astral-sh/uv#14179))
- Serialize Python requests for tools as canonicalized strings ([#&#8203;14109](astral-sh/uv#14109))
- Support netrc and same-origin credential propagation on index redirects ([#&#8203;14126](astral-sh/uv#14126))
- Support reading `dependency-groups` from pyproject.tomls with no `[project]` ([#&#8203;13742](astral-sh/uv#13742))
- Handle an existing shebang in `uv init --script` ([#&#8203;14141](astral-sh/uv#14141))
- Prevent concurrent updates of the environment in `uv run` ([#&#8203;14153](astral-sh/uv#14153))
- Filter managed Python distributions by platform before querying when included in request ([#&#8203;13936](astral-sh/uv#13936))

##### Documentation

- Replace cuda124 with cuda128 ([#&#8203;14168](astral-sh/uv#14168))
- Document the way member sources shadow workspace sources ([#&#8203;14136](astral-sh/uv#14136))
- Sync documented PyTorch integration index for CUDA and ROCm versions from PyTorch website ([#&#8203;14100](astral-sh/uv#14100))

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever MR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this MR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this MR, check this box

---

This MR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0MC42Mi4xIiwidXBkYXRlZEluVmVyIjoiNDAuNjIuMSIsInRhcmdldEJyYW5jaCI6Im1haW4iLCJsYWJlbHMiOlsiUmVub3ZhdGUgQm90Il19-->
sai-rekhawar pushed a commit to sai-rekhawar/cloe-nessy-py that referenced this pull request Jul 1, 2025
This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
| [ghcr.io/astral-sh/uv](https://github.com/astral-sh/uv) | final | patch | `0.7.13` -> `0.7.15` |

---

### Release Notes

<details>
<summary>astral-sh/uv (ghcr.io/astral-sh/uv)</summary>

### [`v0.7.15`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#0715)

[Compare Source](astral-sh/uv@0.7.14...0.7.15)

##### Enhancements

-   Consistently use `Ordering::Relaxed` for standalone atomic use cases ([#&#8203;14190](astral-sh/uv#14190))
-   Warn on ambiguous relative paths for `--index` ([#&#8203;14152](astral-sh/uv#14152))
-   Skip GitHub fast path when rate-limited ([#&#8203;13033](astral-sh/uv#13033))
-   Preserve newlines in `schema.json` descriptions ([#&#8203;13693](astral-sh/uv#13693))

##### Bug fixes

-   Add check for using minor version link when creating a venv on Windows ([#&#8203;14252](astral-sh/uv#14252))
-   Strip query parameters when parsing source URL ([#&#8203;14224](astral-sh/uv#14224))

##### Documentation

-   Add a link to PyPI FAQ to clarify what per-project token is ([#&#8203;14242](astral-sh/uv#14242))

##### Preview features

-   Allow symlinks in the build backend ([#&#8203;14212](astral-sh/uv#14212))

### [`v0.7.14`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#0714)

[Compare Source](astral-sh/uv@0.7.13...0.7.14)

##### Enhancements

-   Add XPU to `--torch-backend` ([#&#8203;14172](astral-sh/uv#14172))
-   Add ROCm backends to `--torch-backend` ([#&#8203;14120](astral-sh/uv#14120))
-   Remove preview label from `--torch-backend` ([#&#8203;14119](astral-sh/uv#14119))
-   Add `[tool.uv.dependency-groups].mygroup.requires-python` ([#&#8203;13735](astral-sh/uv#13735))
-   Add auto-detection for AMD GPUs ([#&#8203;14176](astral-sh/uv#14176))
-   Show retries for HTTP status code errors ([#&#8203;13897](astral-sh/uv#13897))
-   Support transparent Python patch version upgrades ([#&#8203;13954](astral-sh/uv#13954))
-   Warn on empty index directory ([#&#8203;13940](astral-sh/uv#13940))
-   Publish to DockerHub ([#&#8203;14088](astral-sh/uv#14088))

##### Performance

-   Make cold resolves about 10% faster ([#&#8203;14035](astral-sh/uv#14035))

##### Bug fixes

-   Don't use walrus operator in interpreter query script ([#&#8203;14108](astral-sh/uv#14108))
-   Fix handling of changes to `requires-python` ([#&#8203;14076](astral-sh/uv#14076))
-   Fix implied `platform_machine` marker for `...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or improvement to existing functionality

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Allow dependency groups to define a separate requires-python

4 participants