Building with a dotnet not on the PATH is broken

[eerhardt]

This is basically a duplicate of #69023, but I can't re-open that issue, so I logging a new one.

The latest builds of .NET 8 Preview7 are now broken if you xcopy/unzip/untar the bits to some dotnet "hive" that is not on the $PATH. This scenario needs to work.

Repro Steps

1. Have a machine with .NET 7 installed in %PROGRAMFILES%
2. Download and unzip the latest .NET 8 SDK https://github.com/dotnet/installer#table to somewhere not on the $PATH. Ex. C:\dotnet.
3. In a new directory issue the following commands:

C:\dotnet\dotnet.exe new console
C:\dotnet\dotnet.exe build

Expected results

The build should succeed

Actual results

C:\dotnet\sdk\8.0.100-rc.1.23371.3\Roslyn\Microsoft.CSharp.Core.targets(80,5): error : You must install or update .NET to run this application. [C:\Do
tNetTest\Net8Console\Net8Console.csproj]
C:\dotnet\sdk\8.0.100-rc.1.23371.3\Roslyn\Microsoft.CSharp.Core.targets(80,5): error :  [C:\DotNetTest\Net8Console\Net8Console.csproj]
C:\dotnet\sdk\8.0.100-rc.1.23371.3\Roslyn\Microsoft.CSharp.Core.targets(80,5): error : App: C:\dotnet\sdk\8.0.100-rc.1.23371.3\Roslyn\bincore\csc.dll
[C:\DotNetTest\Net8Console\Net8Console.csproj]
C:\dotnet\sdk\8.0.100-rc.1.23371.3\Roslyn\Microsoft.CSharp.Core.targets(80,5): error : Architecture: x64 [C:\DotNetTest\Net8Console\Net8Console.csproj
]
C:\dotnet\sdk\8.0.100-rc.1.23371.3\Roslyn\Microsoft.CSharp.Core.targets(80,5): error : Framework: 'Microsoft.NETCore.App', version '8.0.0-preview.7.23
368.2' (x64) [C:\DotNetTest\Net8Console\Net8Console.csproj]
C:\dotnet\sdk\8.0.100-rc.1.23371.3\Roslyn\Microsoft.CSharp.Core.targets(80,5): error : .NET location: C:\Program Files\dotnet\ [C:\DotNetTest\Net8Cons
ole\Net8Console.csproj]
C:\dotnet\sdk\8.0.100-rc.1.23371.3\Roslyn\Microsoft.CSharp.Core.targets(80,5): error :  [C:\DotNetTest\Net8Console\Net8Console.csproj]
C:\dotnet\sdk\8.0.100-rc.1.23371.3\Roslyn\Microsoft.CSharp.Core.targets(80,5): error : The following frameworks were found: [C:\DotNetTest\Net8Console
\Net8Console.csproj]
C:\dotnet\sdk\8.0.100-rc.1.23371.3\Roslyn\Microsoft.CSharp.Core.targets(80,5): error :   3.1.32 at [C:\Program Files\dotnet\shared\Microsoft.NETCore.A
pp] [C:\DotNetTest\Net8Console\Net8Console.csproj]
C:\dotnet\sdk\8.0.100-rc.1.23371.3\Roslyn\Microsoft.CSharp.Core.targets(80,5): error :   6.0.0-preview.5.21301.5 at [C:\Program Files\dotnet\shared\Mi
crosoft.NETCore.App] [C:\DotNetTest\Net8Console\Net8Console.csproj]
C:\dotnet\sdk\8.0.100-rc.1.23371.3\Roslyn\Microsoft.CSharp.Core.targets(80,5): error :   6.0.19 at [C:\Program Files\dotnet\shared\Microsoft.NETCore.A
pp] [C:\DotNetTest\Net8Console\Net8Console.csproj]
C:\dotnet\sdk\8.0.100-rc.1.23371.3\Roslyn\Microsoft.CSharp.Core.targets(80,5): error :   6.0.20 at [C:\Program Files\dotnet\shared\Microsoft.NETCore.A
pp] [C:\DotNetTest\Net8Console\Net8Console.csproj]
C:\dotnet\sdk\8.0.100-rc.1.23371.3\Roslyn\Microsoft.CSharp.Core.targets(80,5): error :   7.0.8 at [C:\Program Files\dotnet\shared\Microsoft.NETCore.Ap
p] [C:\DotNetTest\Net8Console\Net8Console.csproj]
C:\dotnet\sdk\8.0.100-rc.1.23371.3\Roslyn\Microsoft.CSharp.Core.targets(80,5): error :   7.0.9 at [C:\Program Files\dotnet\shared\Microsoft.NETCore.Ap
p] [C:\DotNetTest\Net8Console\Net8Console.csproj]
C

Original issue

          At the moment this behavior is "By Design". The compiler will use the `dotnet` which is available on `$PATH`. 

Various parts of dotnet/sdk seem to set the DOTNET_HOST_PATH environment variable, and this is no longer being used.

That is an implementation detail of our internal architecture that leaked into the shipping product. Very confident our conclusion is going to be that this needs to be removed from the product.

If you feel a better appproach is in order here please put the discussion on dotnet/runtime#88754. That is the design we are following here.

Originally posted by @jaredpar in #69023 (comment)

[ericstj]

I second what @eerhardt is reporting here. Looks like the CLI was explicitly setting DOTNET_HOST_PATH in the shipping product to communicate which dotnet to use from MSBuild:
https://github.com/dotnet/sdk/blob/996439eb9a1fdd7988a48bdb6f7156e2691c8706/src/Cli/Microsoft.DotNet.Cli.Utils/MSBuildForwardingAppWithoutLogging.cs#L44

It looks like this was introduced here: dotnet/cli#7311 #21237
Related: dotnet/sdk#20, dotnet/sdk#8592
It might be that DOTNET_HOST_PATH was always specifically for this scenario 🤔