Add arcade-powered-source-build local and ci builds

[omajid]

This enables 'source-build', which makes it easier to build the entire shipping .NET SDK from source.

This is the first and second step of arcade-powered-source-build: https://github.com/dotnet/source-build/blob/master/Documentation/planning/arcade-powered-source-build/README.md

See dotnet/sourcelink#692 for a similar PR, that this is based on.

[codecov]

Codecov Report

Merging #4930 (437b074) into main (9e5f533) will increase coverage by 0.04%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             main    #4930      +/-   ##
==========================================
+ Coverage   95.54%   95.59%   +0.04%     
==========================================
  Files        1185     1188       +3     
  Lines      272465   272671     +206     
  Branches    16428    16472      +44     
==========================================
+ Hits       260338   260649     +311     
+ Misses      10001     9895     -106     
- Partials     2126     2127       +1     

[omajid]

This fixes this error I was seeing:

roslyn-analyzers/artifacts/source-build/self/src/src/Utilities/Compiler/WellKnownTypeProvider.cs(87,10): error CS0122: 'PerformanceSensitiveAttribute' is inaccessible due to its protection level [roslyn-analyzers/artifacts/source-build/self/src/src/Microsoft.CodeAnalysis.Analyzers/Core/Microsoft.CodeAnalysis.Analyzers.csproj]

I have no idea what the consequences of this change might be.

[sharwell]

This error would be reported if the build incorrectly omitted Microsoft.CodeAnalysis.PerformanceSensitiveAnalyzers, which is a required analyzer package.

[omajid]

I got some feedback about the CS warnings (safe to disable), but not sure about the AD ones... looks like the VB analogue of CS

[sharwell]

These should be moved to a conditionally-included .globalconfig in eng/config/globalconfigs. See dotnet/roslyn for an example:

https://github.com/dotnet/roslyn/tree/main/eng/config/globalconfigs

[omajid]

This duplicates the contents eng/common/templates/jobs/source-build.yaml. Is there some way to just delegate to that (but only for source-build)?

[mavasani]

@jmarolf Can you please review?

[sharwell]

AD0001 should not be in this list.

[sharwell]

CS8600 and many others in this list fall under nullable. If there is intent to disable all NRT warnings, all items under the nullable identifier umbrella should be removed from this list, and nullable used by name instead.

[sharwell]

IDE0005 should not be in this list.

[sharwell]

❗ We should not patch RoslynAnalyzers.sln. If you need to build a subset of projects, create a SourceBuild.slnf solution filter, which is derived from RoslynAnalyzers.sln and only contains the projects needed for the build.

[sharwell]

This patch is going to break every time we update the code analysis version. We should find a different approach.

[sharwell]

netstandard1.3 is the correct target for this package. Builds using netstandard2.0 would not be correct.

[sharwell]

❗ None of the *.Setup.csproj projects should be included in the solution filter which is used for source build. As such, no changes are expected to any of them.

[sharwell]

😕 Why the suppression here?

[sharwell]

The patches here are producing a totally different build than the repository. If we aren't going to build the real branch properly, we need to make a new branch which contains the code used for source build so it's clear the two build outputs are not equivalent. This branch would not need or use any patches.

[omajid]

@crummel @dseefeld @MichaelSimons any thoughts about the separate branching?

[omajid]

I am going to echo something @dagood wrote here:

I think we should merge this PR as-is, and fix up the patches and this functional issue as followup. This PR represents the state of dotnet/source-build, it's not necessary to solve every source-build issue right here and now.

I dont think a separate branch will work. https://github.com/dotnet/source-build/blob/master/Documentation/planning/arcade-powered-source-build/implementation-plan.md#stage-descriptions says:

• Merging into main/master strongly preferred.
• If a dev branch is created, the repo team must merge it into main/master before ending stage (3).

The end goal is that the exact same git commit that produces official releases can be used for source-build.

[dseefeld]

The patches here are producing a totally different build than the repository. If we aren't going to build the real branch properly, we need to make a new branch which contains the code used for source build so it's clear the two build outputs are not equivalent. This branch would not need or use any patches.

I echo @omajid's comments above. Creating a branch defeats the purpose of what we're trying to do with Arcade-powered source-build. The goal is to build source-build alongside the official build to ensure that all repos can be built from source, rather than doing this after the fact during a release. The patches represent changes that the source-build team has made to be able to build the repo with source-build. The idea is that these would be merged in and maintained in the main code base and that an additional source-build PR validation build would be added to exercise them.

[omajid]

@mavasani @jmarolf @sharwell Hey folks, any updates? Are you totally opposed to our approach of merging in these patches and then coming back later to clean them up?

[sharwell]

I don't believe the proposed approach is maintainable for individual repositories (i.e. the division of work pushes too many changes from devops to individual repository contributors). This impairs the ability to be a proper open source project with external contributors, and the value to those contributors is minimal if any. I would prefer the design be revisited before trying to add it to the main branch.

[sharwell]

It's not clear why source build would be different from normal builds here. We have warnings treated as errors in our build, and the build is clean. If these warnings start to show up for source build, then source build is by definition not matching our actual build and cannot be considered valid.

[sharwell]

❔ Can we move the source build supporting files to a subfolder?

[sharwell]

There are so many problems with the proposed patches, it's really difficult for me to see a viable path from this proposal forward to something that works. It certainly seems easier to just remove all of them, and work through the resulting issues by making the build machine match repository expectations rather than trying to update the code to a different reality. The number/severity of deviations in the resulting artifacts certainly warrants a new branch, if not an entirely different fork.

[omajid]

Thanks for merging this! Please let me know how you want to coordinate a cleanup.