Support reading dependency-groups from pyproject.tomls with no project #13742

Gankra · 2025-05-30T20:02:34Z

(or legacy tool.uv.workspace).

This cleaves out a dedicated SourcedDependencyGroups type based on RequiresDist but with only the DependencyGroup handling implemented. This allows uv pip to read dependency-groups from pyproject.tomls that only have that table defined, per PEP 735, and as implemented by pip.

However we want our implementation to respect various uv features when they're available:

tool.uv.sources
tool.uv.index
tool.uv.dependency-groups.mygroup.requires-python (Add [tool.uv.dependency-groups].mygroup.requires-python #13735)

As such we want to opportunistically detect "as much as possible" while doing as little as possible when things are missing. The issue with the old RequiresDist path was that it fundamentally wanted to build the package, and if [project] was missing it would try to desperately run setuptools on the pyproject.toml to try to find metadata and make a hash of things.

At the same time, the old code also put in a lot of effort to try to pretend that uv pip dependency-groups worked like uv dependency-groups with defaults and non-only semantics, only to separate them back out again. By explicitly separating them out, we confidently get the expected behaviour.

Note that dependency-group support is still included in RequiresDist, as some uv paths still use it. It's unclear to me if those paths want this same treatment -- for now I conclude no.

Fixes #13138

crates/uv-distribution/src/metadata/dependency_groups.rs

Gankra · 2025-05-30T20:08:41Z

crates/uv-workspace/src/workspace.rs

+            Ok(Self::NonProject(workspace))
+        } else if default_missing_workspace {
+            // Otherwise it's a pyproject.toml that maybe contains dependency-groups
+            // that we want to treat like a project/workspace to handle those uniformly
+            let project_path = std::path::absolute(project_root)
+                .map_err(WorkspaceError::Normalize)?
+                .clone();
+
+            let workspace = Workspace::collect_members(
+                project_path,
+                ToolUvWorkspace::default(),
+                pyproject_toml,
+                None,
+                options,
+                cache,
+            )
+            .await?;
+
            Ok(Self::NonProject(workspace))


This part is why the existing virtual workspace support is insufficient (at least one test fails if I try to do this universally -- although it's a test that specifically checks that we error out if [project] and [tool.uv.workspace] are missing).

Gankra · 2025-05-30T20:09:11Z

crates/uv/src/commands/pip/operations.rs

+        for (pyproject_path, groups) in groups {
+            let empty = PathBuf::new();
+            let pyproject_parent = pyproject_path.parent().unwrap_or(&empty);
+            // TODO(Gankra): this surely isn't right but VirtualProject::discover asserts on relative paths... hrm.
+            let project_dir = current_dir().unwrap().join(pyproject_parent);
+            let metadata = SourcedDependencyGroups::from_virtual_project(


I think the assert is telling me I'm still using the wrong API though:

This method requires an absolute path and panics otherwise, i.e. this method only supports discovering the main workspace.

I think this is saying it won't properly detect a package under a workspace?

Ok I now properly understand this code and the assert and my solution are both essentially correct, but the scary "this can only discover the root" warning is wrong or misleading. Cleaned it up.

Gankra · 2025-05-30T20:10:13Z

As always: this requires Way More Tests...

codspeed-hq · 2025-05-30T20:20:00Z

CodSpeed Walltime Performance Report

Merging #13742 will not alter performance

_{Comparing gankra/group-refactor (fb63a50) with main (5021840)}

Summary

✅ 12 untouched benchmarks

zanieb · 2025-06-02T22:27:30Z

@konstin I think it's good for Charlie to review, but it'd be good for you to own other support that's needed here.

Will be fixed in: astral-sh/uv#13742

crates/uv/src/commands/pip/install.rs

crates/uv/src/commands/pip/operations.rs

crates/uv-distribution/src/metadata/dependency_groups.rs

crates/uv-workspace/src/workspace.rs

Gankra · 2025-06-11T13:29:45Z

(nb this is temporarily based on top of #13735 because the two merge conflict)

konstin · 2025-06-13T09:53:49Z

Could you add like a table for the different cases of the current interactions of workspace and sources when we read dependency groups? E.g., a no-project file that is {member,root} of a workspace where the {root, a member} defines sources. I know these didn't change with this PR, but it would be helpful to have them documented.

zanieb · 2025-06-13T13:33:10Z

If they're not changing in this pull request we probably shouldn't block this pull request on it? Can we open an issue for that and follow-up there?

konstin · 2025-06-13T13:39:36Z

We don't change the behavior for workspace discovery and sources here, but we apply them to a new class of inputs, and it helps for reviewing to have a clear understanding which semantics we're applying, especially with non-project pyproject.toml in workspace being somewhat of an odd case.

Gankra · 2025-06-13T14:12:34Z

E.g., a no-project file that is {member,root} of a workspace where the {root, a member} defines sources.

So I think the crux of "nothing interesting has changed here" is that a no-project file cannot be a member of a workspace. It can only be the root of a workspace. The mode I added just allows you to omit the [tool.uv.workspace] file, in which case it's the only member of its workspace.

Gankra · 2025-06-13T14:13:27Z

Furthermore, this ultra-special case I've added is only permitted by uv pip --group

konstin · 2025-06-13T14:16:57Z

So I think the crux of "nothing interesting has changed here" is that a no-project file cannot be a member of a workspace. It can only be the root of a workspace.

I see, that explains it!

konstin · 2025-06-12T14:58:03Z

crates/uv/src/commands/pip/operations.rs

+                                metadata
+                                    .name
+                                    .clone()
+                                    .unwrap_or(PackageName::from_str("none").unwrap()),


Can we make this an option instead?

Huh ok this is used way less than I thought, literally only in this one place. Cool fixed.

(or legacy tool.uv.workspace). This cleaves out a dedicated SourcedDependencyGroups type based on RequiresDist but with only the DependencyGroup handling implemented. This allows `uv pip` to read `dependency-groups` from pyproject.tomls that only have that table defined, per PEP 735, and as implemented by `pip`. However we want our implementation to respect various uv features when they're available: * `tool.uv.sources` * `tool.uv.index` * `tool.uv.dependency-groups.mygroup.requires-python` (#13735) As such we want to opportunistically detect "as much as possible" while doing as little as possible when things are missing. The issue with the old RequiresDist path was that it fundamentally wanted to build the package, and if `[project]` was missing it would try to desperately run setuptools on the pyproject.toml to try to find metadata and make a hash of things. At the same time, the old code also put in a lot of effort to try to pretend that `uv pip` dependency-groups worked like `uv` dependency-groups with defaults and non-only semantics, only to separate them back out again. By explicitly separating them out, we confidently get the expected behaviour. Note that dependency-group support is still included in RequiresDist, as some `uv` paths still use it. It's unclear to me if those paths want this same treatment -- for now I conclude no. Fixes #13138

This MR contains the following updates: | Package | Update | Change | |---|---|---| | [astral-sh/uv](https://github.com/astral-sh/uv) | patch | `0.7.13` -> `0.7.14` | MR created with the help of [el-capitano/tools/renovate-bot](https://gitlab.com/el-capitano/tools/renovate-bot). **Proposed changes to behavior should be submitted there as MRs.** --- ### Release Notes <details> <summary>astral-sh/uv (astral-sh/uv)</summary> ### [`v0.7.14`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#0714) [Compare Source](astral-sh/uv@0.7.13...0.7.14) ##### Enhancements - Add XPU to `--torch-backend` ([#14172](astral-sh/uv#14172)) - Add ROCm backends to `--torch-backend` ([#14120](astral-sh/uv#14120)) - Remove preview label from `--torch-backend` ([#14119](astral-sh/uv#14119)) - Add `[tool.uv.dependency-groups].mygroup.requires-python` ([#13735](astral-sh/uv#13735)) - Add auto-detection for AMD GPUs ([#14176](astral-sh/uv#14176)) - Show retries for HTTP status code errors ([#13897](astral-sh/uv#13897)) - Support transparent Python patch version upgrades ([#13954](astral-sh/uv#13954)) - Warn on empty index directory ([#13940](astral-sh/uv#13940)) - Publish to DockerHub ([#14088](astral-sh/uv#14088)) ##### Performance - Make cold resolves about 10% faster ([#14035](astral-sh/uv#14035)) ##### Bug fixes - Don't use walrus operator in interpreter query script ([#14108](astral-sh/uv#14108)) - Fix handling of changes to `requires-python` ([#14076](astral-sh/uv#14076)) - Fix implied `platform_machine` marker for `win_amd64` platform tag ([#14041](astral-sh/uv#14041)) - Only update existing symlink directories on preview uninstall ([#14179](astral-sh/uv#14179)) - Serialize Python requests for tools as canonicalized strings ([#14109](astral-sh/uv#14109)) - Support netrc and same-origin credential propagation on index redirects ([#14126](astral-sh/uv#14126)) - Support reading `dependency-groups` from pyproject.tomls with no `[project]` ([#13742](astral-sh/uv#13742)) - Handle an existing shebang in `uv init --script` ([#14141](astral-sh/uv#14141)) - Prevent concurrent updates of the environment in `uv run` ([#14153](astral-sh/uv#14153)) - Filter managed Python distributions by platform before querying when included in request ([#13936](astral-sh/uv#13936)) ##### Documentation - Replace cuda124 with cuda128 ([#14168](astral-sh/uv#14168)) - Document the way member sources shadow workspace sources ([#14136](astral-sh/uv#14136)) - Sync documented PyTorch integration index for CUDA and ROCm versions from PyTorch website ([#14100](astral-sh/uv#14100)) </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever MR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this MR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this MR, check this box --- This MR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).

Gankra commented May 30, 2025

View reviewed changes

crates/uv-distribution/src/metadata/dependency_groups.rs Outdated Show resolved Hide resolved

Gankra commented May 30, 2025

View reviewed changes

Gankra added the bug Something isn't working label May 30, 2025

konstin requested a review from charliermarsh June 2, 2025 16:32

zanieb assigned konstin Jun 2, 2025

autophagy added a commit to autophagy/flink that referenced this pull request Jun 5, 2025

Remove uv pip install from groups

a27017d

Will be fixed in: astral-sh/uv#13742

konstin reviewed Jun 6, 2025

View reviewed changes

Gankra force-pushed the gankra/group-refactor branch from 0d0d59a to 690e750 Compare June 10, 2025 19:05

Gankra temporarily deployed to release June 10, 2025 19:05 — with GitHub Actions Inactive

Gankra changed the base branch from main to gankra/group-python-real June 12, 2025 13:49

Gankra force-pushed the gankra/group-refactor branch from 690e750 to 3605cf3 Compare June 12, 2025 13:50

konstin mentioned this pull request Jun 12, 2025

Use fast path in noop with pip install --group #13999

Open

konstin approved these changes Jun 13, 2025

View reviewed changes

Base automatically changed from gankra/group-python-real to main June 13, 2025 22:04

Gankra added 4 commits June 13, 2025 18:06

cleanups from review

e765c51

more review feedback

7a8a461

use an Option

fb63a50

Gankra force-pushed the gankra/group-refactor branch from 488e0c7 to fb63a50 Compare June 13, 2025 22:07

Gankra enabled auto-merge (squash) June 13, 2025 22:07

Gankra merged commit ff9c2c3 into main Jun 13, 2025
86 checks passed

Gankra deleted the gankra/group-refactor branch June 13, 2025 22:16

This was referenced Jun 17, 2025

Add a non-package-mode #10204

Open

allow "uv run" to work without a project #8666

Closed

BrewTestBot mentioned this pull request Jun 23, 2025

uv 0.7.14 Homebrew/homebrew-core#227918

Merged

autophagy mentioned this pull request Jul 14, 2025

[FLINK-37775][python] Manage python development/test dependencies in pyproject.yaml apache/flink#26642

Closed

Support reading dependency-groups from pyproject.tomls with no project #13742

Support reading dependency-groups from pyproject.tomls with no project #13742

Uh oh!

Conversation

Gankra commented May 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Gankra May 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Gankra May 30, 2025

Choose a reason for hiding this comment

Uh oh!

Gankra May 30, 2025

Choose a reason for hiding this comment

Uh oh!

Gankra Jun 12, 2025

Choose a reason for hiding this comment

Uh oh!

Gankra commented May 30, 2025

Uh oh!

codspeed-hq bot commented May 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

CodSpeed Walltime Performance Report

Merging #13742 will not alter performance

Summary

Uh oh!

zanieb commented Jun 2, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Gankra commented Jun 11, 2025

Uh oh!

konstin commented Jun 13, 2025

Uh oh!

zanieb commented Jun 13, 2025

Uh oh!

konstin commented Jun 13, 2025

Uh oh!

Gankra commented Jun 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Gankra commented Jun 13, 2025

Uh oh!

konstin commented Jun 13, 2025

Uh oh!

konstin Jun 12, 2025

Choose a reason for hiding this comment

Uh oh!

Gankra Jun 13, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Gankra commented May 30, 2025 •

edited

Loading

Gankra May 30, 2025 •

edited

Loading

codspeed-hq bot commented May 30, 2025 •

edited

Loading

Gankra commented Jun 13, 2025 •

edited

Loading