fix issue with optional fields in dependency validator

[tdyas]

Summary

Fix FieldSet annotation handling so optional field annotations like SomeField | None are recognized as FieldSet fields.

This addresses a bug in Python dependency validation where DependencyValidationFieldSet.resolve (annotated as PythonResolveField | None) was being skipped by FieldSet.fields(), causing dependency validation to lose the depender’s resolve and potentially fall back to global interpreter constraints.

The issue was observed in the shoalsoft-pants-opentelemetry-plugin when trying to upgrade it to support Pants v2.32 & Python 3.14. With [python].default_to_resolve_interpreter_constraints set to true, unspecified interpreter_constraints on parametrized targets were defaulting to the global ICs and not the resolve-specific ICs (set via [python].resolves_to_interpreter_constraints) as expected, leading to errors like the following:

InvalidFieldException: The target src/python/shoalsoft/pants_opentelemetry_plugin/exception_logging_processor.py@parametrize=pants-2.28 has the `interpreter_constraints` ('==3.11.*', '==3.14.*'), which are not a subset of the `interpreter_constraints` of some of its dependencies:

  * ('==3.11.*',): src/python/shoalsoft/pants_opentelemetry_plugin/processor.py@parametrize=pants-2.28

Root cause and solution

FieldSet.fields() only collected annotations that were direct subclasses of Field:

• resolve: PythonResolveField -> recognized
• resolve: PythonResolveField | None -> skipped

As a result, DependencyValidationFieldSet.create(tgt) populated resolve=None even when the target had a resolve.

Solution

Update FieldSet.fields() to unwrap union annotations and extract the underlying Field type when the annotation is a union containing exactly one Field subclass (e.g. Field | None / Optional[Field]).

Testing

This PR includes a unit test demonstrating the issue and which passes with the fix.

AI Usage

Codex was used to diagnose the issue, write the reproduction test, and then write the solution.

[tdyas]

My main question here is whether Field | None is even valid for FieldSet. The issue is caused by DependencyValidationFieldSet.

If the syntax is not valid, then we probably need an alternate solution than the one in this PR which is to extract a Field from Field | None.

[tdyas]

Fixing this bug resulting in these uses of python_distribution defaulting to python-default resolve even though only a and b resolves are defined. python_distribution does not appear to have resolve field, is that intentional?

[tdyas]

cc @benjyw re the above question

[benjyw]

Presumably it is intentional, since the python_distribution is just some metadata on how to build a wheel, and the resolve doesn't enter into that?

[tdyas]

Then it seems incorrect for the PR's solution to even apply the notion of a resolve to python_distribution then. Do you concur?

[tdyas]

These tests began failing with this PR's solution in place and I added default resolve options to "fix" the test failures. Maybe this PR needs to account for the optionality of PythonResolveField | None in DependencyValidationFieldSet or that optionality is incorrect and FieldSets should never have optional types?

[benjyw]

I think that is the only Pythonic target type with no resolve field, so maybe even just for uniformity... The trouble is that "empty resolve field" defaults to the repo default, IIRC, and if that is not compatible with the resolve of the dependencies then things would blow up, right?

[tdyas]

I dug in more on the issue today. The issue here is that when an optional field in a FieldSet is missing from a target, then the field is instantiated with its default value instead of being set to None as a result of Target.get.

So DependencyValidationFieldSet is trying to use a semantic (i.e., return None for missing fields) which FieldSet does not support at all. And that is why the resolve-specific ICs are not applied at

pants/src/python/pants/backend/python/target_types_rules.py

Lines 649 to 651 in 73d7a82

	target_ics = request.field_set.interpreter_constraints.value_or_configured_default(
	python_setup, request.field_set.resolve
	)

since resolve on DependencyValidationFieldSet is always None.

So potential solutions:

1. python_distribution appears to be the only Python target without a resolve field. Yet, the source code for the distribution would need to be specified as dependencies and would need to be from a single resolve, right? So one view is that python_distribution should have a resolve field. Could a python_distribution ever be compatible with dependency targets from multiple resolves?

2. We keep python_distribution without a resolve field and just fix the FieldSet system to understand Field | None. I have that support already in place locally. It will be pushed after this comment. It also validates that all FieldSets have fields annotated with types either Field or Field | None.

[benjyw]

Given what you've discovered, seems like 1. should work? IIUC the behavior is effectively this already (since the target doesn't have this field, we assume the default resolve, not no resolve)?

[tdyas]

With the original fix, the code assumes the default resolve. With the modified fix in today's commits, the FieldSet machinery is able to return None for PythonResolveField | None as a reader might have assumed to be the case. So with the modified fix (properly supporting optionality), there is no need for python_distribution to gain a resolve field.

[tdyas]

Also, the original fix could have broken existing users by requiring them to set a valid default resolve even though their code base wasn't using the default resolve if they had a python_distribution target.