Skip to content

Add portable linux source build leg #75546

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
mmitche merged 15 commits into from
Oct 6, 2022
Merged

Add portable linux source build leg #75546

Show file tree
Hide file tree
Changes from 11 commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
72625f7
Add portable linux source build leg
mmitche Sep 7, 2022
d8e009d
Invert conditional
mmitche Sep 13, 2022
eac193f
Set the platform name so that log artifacts don't overlap
mmitche Sep 13, 2022
3a2dca5
Try something to set asset manifest name
mmitche Sep 13, 2022
65ccd91
Merge remote-tracking branch 'upstream/main' into add-portable-linux-leg
mmitche Sep 20, 2022
6811513
Respond to feedback
mmitche Sep 20, 2022
a5bf093
Update arcade
mmitche Sep 20, 2022
cac3d1f
Merge remote-tracking branch 'upstream/main' into add-portable-linux-leg
mmitche Oct 3, 2022
1630623
Move insert to end
mmitche Oct 3, 2022
0e1fab4
Fix up non-portable parameter
mmitche Oct 3, 2022
bf026ca
Fix up target rid
mmitche Oct 3, 2022
e73ec54
Merge remote-tracking branch 'upstream/main' into add-portable-linux-leg
mmitche Oct 4, 2022
8e4039a
- Run only linux-x64 portable officially
mmitche Oct 4, 2022
84643a7
Move PR leg into dev-innnerloop
mmitche Oct 4, 2022
3b6b4d1
Remove helix group in innerloop
mmitche Oct 5, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 9 additions & 8 deletions eng/pipelines/common/global-build-job.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,8 @@ parameters:
shouldContinueOnError: false
dependOnEvaluatePaths: false
isOfficialBuild: false
buildingOnSourceBuildImage: false
isSourceBuild: false
isNonPortableSourceBuild: false
runtimeFlavor: 'coreclr'
runtimeVariant: ''
helixQueues: ''
Expand Down Expand Up @@ -113,9 +114,9 @@ jobs:
value: ''

- name: _sclEnableCommand
${{ if eq(parameters.buildingOnSourceBuildImage, true) }}:
${{ if eq(parameters.isSourceBuild, true) }}:
value: scl enable llvm-toolset-7.0 --
${{ if ne(parameters.buildingOnSourceBuildImage, true) }}:
${{ if ne(parameters.isSourceBuild, true) }}:
value: ''

- ${{ each variable in parameters.variables }}:
Expand All @@ -136,15 +137,15 @@ jobs:
artifact: Mono_Offsets_${{monoCrossAOTTargetOS}}
path: '$(Build.SourcesDirectory)/artifacts/obj/mono/offsetfiles'

- ${{ if eq(parameters.buildingOnSourceBuildImage, true) }}:
- ${{ if eq(parameters.isSourceBuild, true) }}:
- template: /eng/common/templates/steps/source-build.yml
parameters:
platform:
buildScript: $(_sclEnableCommand) $(Build.SourcesDirectory)$(dir)build$(scriptExt)
nonPortable: true
# Use a custom RID that isn't in the RID graph here to validate we don't break the usage of custom rids that aren't in the graph.
targetRID: banana.24-x64
Copy link
Member

@ViktorHofer ViktorHofer Oct 4, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You remove the custom RID, but we want to keep it to make sure that we can build with arbitrary RIDs.

Copy link
Member Author

@mmitche mmitche Oct 4, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then we probably need another leg, or one that only runs in PRs? Preference?

Copy link
Member Author

@mmitche mmitche Oct 4, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ViktorHofer @tmds @MichaelSimons I made some tweaks:

  • Build only linux-x64 portable officially (nothing else flows)
  • Add missing PR legs for linux-x64 (to avoid official breaks)
  • Add rolling CI for Banana and centos7

nonPortable: ${{ parameters.isNonPortableSourceBuild }}
targetRID: ${{ parameters.targetRid }}
runtimeOS: linux
name: ${{ parameters.platform }}

- ${{ if in(parameters.osGroup, 'OSX', 'iOS', 'tvOS', 'MacCatalyst') }}:
- script: $(Build.SourcesDirectory)/eng/install-native-dependencies.sh ${{ parameters.osGroup }} ${{ parameters.archType }} azDO
Expand All @@ -163,7 +164,7 @@ jobs:
filePath: $(Build.SourcesDirectory)/eng/pipelines/mono/update-machine-certs.ps1

# Build
- ${{ if eq(parameters.buildingOnSourceBuildImage, false) }}:
- ${{ if eq(parameters.isSourceBuild, false) }}:
- script: $(_sclEnableCommand) $(Build.SourcesDirectory)$(dir)build$(scriptExt) -ci -arch ${{ parameters.archType }} $(_osParameter) ${{ parameters.buildArgs }} $(_officialBuildParameter) $(_crossBuildPropertyArg) $(_cxx11Parameter) $(_richCodeNavigationParam) $(_buildDarwinFrameworksParameter) $(_overrideTestScriptWindowsCmdParameter)
displayName: Build product
${{ if eq(parameters.useContinueOnErrorDuringBuild, true) }}:
Expand Down
30 changes: 28 additions & 2 deletions eng/pipelines/common/platform-matrix.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -279,7 +279,32 @@ jobs:
disableClrTest: true
${{ insert }}: ${{ parameters.jobParameters }}

# Linux x64 Source Build
# Centos 7 x64 Source Build

- ${{ if containsValue(parameters.platforms, 'SourceBuild_Centos7_x64') }}:
- template: xplat-setup.yml
parameters:
jobTemplate: ${{ parameters.jobTemplate }}
helixQueuesTemplate: ${{ parameters.helixQueuesTemplate }}
variables: ${{ parameters.variables }}
osGroup: Linux
archType: x64
targetRid: centos.7-x64
platform: Centos7_Linux_x64
shouldContinueOnError: ${{ parameters.shouldContinueOnError }}
container:
image: centos-7-source-build-20210714125450-5d87b80
registry: mcr
jobParameters:
runtimeFlavor: ${{ parameters.runtimeFlavor }}
stagedBuild: ${{ parameters.stagedBuild }}
buildConfig: ${{ parameters.buildConfig }}
helixQueueGroup: ${{ parameters.helixQueueGroup }}
isSourceBuild: true
isNonPortableSourceBuild: true
${{ insert }}: ${{ parameters.jobParameters }}

# Portable Linux x64 Source Build

- ${{ if containsValue(parameters.platforms, 'SourceBuild_Linux_x64') }}:
- template: xplat-setup.yml
Expand All @@ -300,8 +325,9 @@ jobs:
stagedBuild: ${{ parameters.stagedBuild }}
buildConfig: ${{ parameters.buildConfig }}
helixQueueGroup: ${{ parameters.helixQueueGroup }}
isSourceBuild: true
isNonPortableSourceBuild: false
${{ insert }}: ${{ parameters.jobParameters }}
buildingOnSourceBuildImage: true

# Linux s390x

Expand Down
16 changes: 15 additions & 1 deletion eng/pipelines/runtime-official.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -367,6 +367,20 @@ stages:
#
# Build Sourcebuild leg
#
- template: /eng/pipelines/common/platform-matrix.yml
parameters:
jobTemplate: /eng/pipelines/common/global-build-job.yml
buildConfig: Release
helixQueueGroup: ci
platforms:
- SourceBuild_Centos7_x64
jobParameters:
nameSuffix: Centos7SourceBuild
extraStepsTemplate: /eng/pipelines/common/upload-intermediate-artifacts-step.yml
extraStepsParameters:
name: SourceBuildPackages
timeoutInMinutes: 95

- template: /eng/pipelines/common/platform-matrix.yml
parameters:
jobTemplate: /eng/pipelines/common/global-build-job.yml
Expand All @@ -375,7 +389,7 @@ stages:
platforms:
- SourceBuild_Linux_x64
jobParameters:
nameSuffix: SourceBuild
nameSuffix: PortableSourceBuild
extraStepsTemplate: /eng/pipelines/common/upload-intermediate-artifacts-step.yml
extraStepsParameters:
name: SourceBuildPackages
Expand Down