Skip to content

[ci] Govern dnceng internal pipeline - #12530

Merged
jonathanpeppers merged 9 commits into
mainfrom
jonathanpeppers-govern-internal-pipeline
Aug 28, 2026
Merged

jonathanpeppers merged 9 commits into
mainfrom
jonathanpeppers-govern-internal-pipeline

Conversation

@jonathanpeppers

@jonathanpeppers jonathanpeppers commented Aug 26, 2026 •

Copy link
Copy Markdown
Member

Summary

  • Migrate dotnet-android-internal to the official 1ES governed pipeline template while preserving its triggers, stage graph, pool images, signing flow, artifact names, and package-test JDK selection.
  • Resolve repository-local templates explicitly from @self and classify auxiliary repositories for SDL scanning.
  • Use the lightweight dotnet/macios-devtools repository wherever a second checkout exists only to keep the primary repository at s/android. This covers the internal, public, official/shared, and nightly pipelines; comments document why the checkout is required. Real MAUI integration jobs continue to check out dotnet/maui.
  • Remove preinstalled .NET SDKs and runtimes from hosted macOS test agents before setup to prevent low-disk warnings without weakening the existing warning gate.

Validation

  • Azure DevOps template previews succeeded for the internal, public, official, PR, and nightly pipeline definitions.
  • Internal validation build 3060061 ran commit c33b84759: 38 jobs passed and all hosted macOS jobs completed with zero low-disk warnings.
  • The five failed jobs contain the same existing CheckLint test failures reproduced by the non-governed main build; no migration, checkout-shim, or disk-cleanup regression was found.

Migrate dotnet-android-internal to the official 1ES Pipeline Template while preserving its stage graph, internal pool images, signing flow, and artifacts.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI lite review requested due to automatic review settings August 26, 2026 20:31

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR migrates the internal dotnet-android-internal Azure DevOps pipeline (build-tools/automation/azure-pipelines-internal.yaml) to the governed 1ES Official pipeline template, while keeping the existing stage graph and artifact outputs compatible with the current production definition.

Changes:

  • Switch the internal pipeline from a raw stages: graph to extends: v1/1ES.Official.PipelineTemplate.yml@1ESPipelineTemplates and move artifact publishing to templateContext / 1ES.PublishPipelineArtifact@1.
  • Add an opt-in switch in the package test stage template to use a “public” setup template (avoids internal-only checkout template dependencies).
  • Update setup-test-environment-public.yaml to pass use1ESTemplate through to its underlying steps template.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File Description
build-tools/automation/yaml-templates/stage-package-tests.yaml Adds a toggle to select the public vs. internal setup template for package test stages.
build-tools/automation/yaml-templates/setup-test-environment-public.yaml Makes use1ESTemplate configurable instead of hard-coded false.
build-tools/automation/azure-pipelines-internal.yaml Migrates the production internal pipeline to the governed 1ES Official template and updates artifact publishing/signing integration accordingly.
Suppressed comments (1)

build-tools/automation/yaml-templates/stage-package-tests.yaml:120

  • ⚠️ maintainability — Same as above for the second macOS APK test job: enabling usePublicSetupTemplate changes implicit defaults (JDK version selection and useAgentJdkPath) compared to setup-test-environment.yaml. Passing them explicitly here helps preserve behavior while avoiding the private checkout template dependency.
    - ${{ if eq(parameters.usePublicSetupTemplate, true) }}:
      - template: /build-tools/automation/yaml-templates/setup-test-environment-public.yaml
        parameters:
          use1ESTemplate: ${{ parameters.use1ESTemplate }}

jonathanpeppers and others added 4 commits August 26, 2026 15:36
Keep the existing DefaultJavaSdkMajorVersion and agent JDK behavior when package tests use the governed-compatible checkout path.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Qualify repository-local templates explicitly now that the root pipeline extends the external 1ES template repository.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Exclude android-tools and maui from source scanning because they are external build and test inputs, matching the existing official pipeline behavior.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Drop the unused android-tools resource and use the already-required MAUI repository to preserve multi-checkout path semantics.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@jonathanpeppers

jonathanpeppers commented Aug 26, 2026 •

Copy link
Copy Markdown
Member Author

Internal validation completed: dnceng/internal build 3059110, testing commit 31c4146ae.

Disk cleanup is validated: all 22 hosted macOS test jobs completed with zero low-disk warnings, and all downstream Windows/Linux/signing jobs passed. The run finished with 37 passed jobs and six unrelated/baseline failures:

  • Five macOS MSBuild shards each failed one BuildTest.CheckLint* case. The same five cases failed in non-governed main build 3058331.
  • Emulator shard 9 failed before setup because recursive Git submodule checkout failed.

The change uses the existing remove_dotnet setup option for hosted macOS test matrices, removing preinstalled SDKs/runtimes before installing the selected SDK. The existing warning gate remains unchanged. No migration-related or cleanup-related failure was observed.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 60a6ba42-9755-43ac-8d02-5380d84c5fd0
Remove preinstalled .NET SDKs before installing the pipeline-selected SDK in internal hosted macOS test matrices. This preserves the existing warning gate while preventing test artifact downloads from crossing the agent's low-disk threshold.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 60a6ba42-9755-43ac-8d02-5380d84c5fd0
@jonathanpeppers

Copy link
Copy Markdown
Member Author

Internal validation completed: dnceng/internal build 3059110 tested commit 31c4146ae.

The disk cleanup worked: all 22 hosted macOS test jobs completed with zero low-disk warnings, and all Windows, Linux, and signing jobs passed. The six failed jobs were unrelated to this change: five BuildTest.CheckLint* failures exactly matched non-governed main build 3058331, and one emulator shard failed during recursive submodule checkout. No migration-related or cleanup-related failure was observed.

@jonathanpeppers

Copy link
Copy Markdown
Member Author

The JDK 25/lint test failures on internal are hopefully fixed by:

@jonathanpeppers jonathanpeppers added the ready-to-review This PR is ready to review/merge, I think any CI failures are just flaky (ignorable). label Aug 27, 2026
@jonathanpeppers

Copy link
Copy Markdown
Member Author

@dalexsoto review

Replace otherwise-unused MAUI checkouts with dotnet/macios-devtools while retaining MAUI where integration jobs consume it. Document that the extra checkout preserves the s/android multi-repository layout.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 60a6ba42-9755-43ac-8d02-5380d84c5fd0
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 60a6ba42-9755-43ac-8d02-5380d84c5fd0
@jonathanpeppers
jonathanpeppers merged commit cf93dff into main Aug 28, 2026
42 of 44 checks passed
@jonathanpeppers
jonathanpeppers deleted the jonathanpeppers-govern-internal-pipeline branch August 28, 2026 18:05
@github-actions github-actions Bot locked and limited conversation to collaborators Sep 28, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

ready-to-review This PR is ready to review/merge, I think any CI failures are just flaky (ignorable).

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants