Makefile updates to get dotnet/fsharp building on mono #1

baronfel · 2019-05-17T22:40:18Z

Goals are to be able to build the whole compiler toolchain on latest stable mono with a recent msbuild, for net472 (I guess?).

We need to replicate the make && make install capabilities of the fsharp/fsharp makefile.

Vague plan of attack:

get current state building for net472 on stable mono + msbuild
enable tests to some degree
port over install logic
add this flow to the vsts ci platform

…eters

not currently working, though if you manually run msbuild /t:Restore followed by msbuild /t:Build /p:TargetFramework=net472 for that project, it all works great.

last step is to translate the dotnet test commands to msbuild /t:Test equivalents

baronfel · 2019-05-18T15:56:05Z

As of now we can build and test on mono 5.20 stable 🎆

baronfel · 2019-05-18T15:56:52Z

FSharpBuild.Directory.Build.props

    <FrameworkPathOverride Condition="'$(TargetFramework)' == 'net47'">$(MonoLibFolder)/4.7-api</FrameworkPathOverride>
    <FrameworkPathOverride Condition="'$(TargetFramework)' == 'net471'">$(MonoLibFolder)/4.7.1-api</FrameworkPathOverride>
-    <FrameworkPathOverride Condition="'$(TargetFramework)' == 'net472'">$(MonoLibFolder)/4.7.2-api</FrameworkPathOverride>
+    <FrameworkPathOverride Condition="'$(TargetFramework)' == 'net472'">$(MonoLibFolder)/4.7.2-api;$(MonoLibFolder)/4.7.2-api/Facades</FrameworkPathOverride>


this can be reverted I believe. the stable msbuild targets for mono handle setting facades from what I can see now, and this property just ends up being reverted if you look at a detailed log.

baronfel · 2019-05-18T15:57:43Z

Makefile

-	$(DotNetExe) build src/fsharp/FSharp.Build/FSharp.Build.fsproj -f netstandard2.0 -c Proto
-	$(DotNetExe) build src/fsharp/fsc/fsc.fsproj -f netcoreapp2.1 -c Proto
+proto:
+	$(RestoreCommand) $(NF472) src/buildtools/buildtools.proj 


virtually all changes to this file revolve around using pure msbuild equivalents to previous dotnet sdk commands.

baronfel · 2019-05-18T15:58:26Z

eng/Versions.props

    <!-- version numbers from files -->
    <RoslynVersion>$([System.IO.File]::ReadAllText('$(MSBuildThisFileDirectory)..\RoslynPackageVersion.txt').Trim())</RoslynVersion>
    <!-- System.* packages -->
+    <SystemCollectionsVersion>4.3.0</SystemCollectionsVersion>


initially I thought that adding a version here and flowing through a packagereference would be what we needed to get some projects to build on net472 for mono, but in the end plain References where what we needed.

baronfel · 2019-05-18T15:59:54Z

src/buildtools/buildtools.targets


    <PropertyGroup>
-        <FsLexPath Condition="'$(FsLexPath)' == ''">$(ArtifactsDir)\Bootstrap\fslex.dll</FsLexPath>
+        <FsLexPath Condition="'$(FsLexPath)' == '' And '$(MonoPackaging)' == 'false' ">$(ArtifactsDir)\Bootstrap\netcoreapp2.1\fslex.dll</FsLexPath>


changes to this file are around making fxlex/fsyacc invoke correctly on mono/unix. Directories have changed too and need to be validated (ie building by the makefile we don't get the Bootstrap subdirectory. if this is desired we should solve the artifactsdir problem then adjust these paths.

baronfel · 2019-05-18T16:00:15Z

src/buildtools/fslex/fslex.fsproj

  <PropertyGroup>
    <OutputType>Exe</OutputType>
-    <TargetFramework>netcoreapp2.1</TargetFramework>
+    <TargetFrameworks>net472;netcoreapp2.1</TargetFrameworks>


need to be able to invoke these tools on mono, so they build for net472 now.

baronfel · 2019-05-18T16:00:33Z

src/fsharp/FSharp.Build/FSharp.Build.fsproj

  <PropertyGroup>
    <OutputType>Library</OutputType>
    <TargetFrameworks>net472;netcoreapp2.1</TargetFrameworks>
-    <TargetFrameworks Condition="'$(OS)' == 'Unix'">netcoreapp2.1</TargetFrameworks>


This was removed for several projects in order to enable mono builds

baronfel · 2019-05-18T16:00:52Z

src/fsharp/FSharp.Build/FSharp.Build.fsproj


  <ItemGroup Condition="'$(TargetFramework)' == 'net472'">
    <PackageReference Include="Microsoft.VisualFSharp.Type.Providers.Redist" Version="$(MicrosoftVisualFSharpTypeProvidersRedistVersion)" />
+    <Reference Include="System.Collections" />


Should this be strongly-versioned? (same question for the other instances)

baronfel · 2019-05-18T16:02:02Z

src/fsharp/FSharp.Build/FSharp.Build.fsproj

    <PackageReference Include="Microsoft.Build.Tasks.Core" Version="$(MicrosoftBuildTasksCoreVersion)" />
    <PackageReference Include="Microsoft.Build.Utilities.Core" Version="$(MicrosoftBuildUtilitiesCoreVersion)" />
    <PackageReference Include="Microsoft.Win32.Registry" Version="$(MicrosoftWin32RegistryVersion)" />
+    <PackageReference Include="System.Collections" Version="4.3.0" />


Initially I wanted to use a packagereference for System.Collections like other assemblies do, but this didn't work for two reasons: I couldn't use the version from Versions.props (it never got picked up?) and the packagereference didn't delegate and add the appropriate gac reference.

baronfel · 2019-05-18T16:02:31Z

src/fsharp/FSharp.Compiler.Private/FSharp.Compiler.Private.fsproj


  <ItemGroup Condition="'$(TargetFramework)' == 'net472'">
    <Reference Include="ISymWrapper, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" />
+    <Reference Include="System.Collections" />


same story here for these Reference nodes. Should they be strongly-referenced and/or is there a package equivalent?

baronfel · 2019-05-18T16:03:18Z

src/fsharp/FSharp.Core/FSharp.Core.fsproj

    <AllowCrossTargeting>true</AllowCrossTargeting>
    <DefineConstants>$(DefineConstants);FSHARP_CORE</DefineConstants>
    <DefineConstants Condition="'$(Configuration)' == 'Proto'">BUILDING_WITH_LKG;$(DefineConstants)</DefineConstants>
+    <DefineConstants Condition="'$(MonoPackaging)' == 'true'">BUILD_FROM_SOURCE;$(DefineConstants)</DefineConstants>


How is BUILD_FROM_SOURCE specified with the official builds? I couldn't replicate it with a command line flag and without this we get errors in prim-types.fsi where the inref/byref types are defined and used.

baronfel · 2019-05-18T16:06:46Z

@dsyme @cartermp @KevinRansom I'm starting work to make the main repo build on mono and wanted to isolate the changes as much as possible. Can you all look over the changes above (which are the ones most relevant to building and testing the repository on mono)? These should be the ones with the most overlap with the dotnet-sdk-based build chain that's already established, and the remainder of the work would be porting the install make target, which is more a question of file organization than build process IMO. I'd also be amenable to breaking any or all of this out in parts to upstream eg jsut the build-and-test portions in hopes of getting a mono-based ci build integrated before we began working on the mono-install tasks.

baronfel · 2019-05-18T18:31:45Z

I added a dockerfile and realized that you can only call the VsTest target on .net sdk 3 previews, per this statement from the vstest project readme:

The Test Platform currently ships as part Visual Studio 2017, and in the .NET Core Tools Preview 3.

So I've removed the test step from the plan for now. The remaining steps work just fine though, and can be run locally via make or in a docker mono 5.20 environment via docker build -f mono.Dockerfile .

apply makefile patch to include paths for VS v16.0 fsharp locations

…ono/blob/master/packaging/MacSDK/patches/fsharp-noinstall.patch

…SDK/patches/fsharp-custom-prefix.patch to allow for custom prefixes

baronfel · 2019-05-19T15:25:19Z

I added a few mono patches that they'd added to address increased mono msbuild compatibility to msbuild v16, as well as allowing for a custom PREFIX in the makefile (nice for testing locally without blowing away your local install).

I also got testing working for FSharp.Core.UnitTests and FSharp.Build.UnitTests on mono by discovering and running nunit directly (though right now it's using the latest nunit.consolerunner package present and I'd like to pull that version from Versions.props or whereever is correct.

I'm still iterating on the install logic right now, trying to replicate the current directory layout structure as much as possible

baronfel · 2019-05-19T15:25:37Z

Tagging @ninjarobot on this as well for input and status updates.

baronfel added 5 commits May 17, 2019 15:26

bring over mono directory updates from fsharp/fsharp

7453406

update makefile to use msbuild and flags instead of sdk and cli param…

54364b8

…eters

try to get system.collection reference honored by FSharp.Build.fsproj

541ba7c

not currently working, though if you manually run msbuild /t:Restore followed by msbuild /t:Build /p:TargetFramework=net472 for that project, it all works great.

proto/restore/build all work on mono 5.20 now

c4f93b0

last step is to translate the dotnet test commands to msbuild /t:Test equivalents

replicate msbuild targets for --no-build and -l

d906c81

baronfel force-pushed the makefile-updates-for-mono branch from fae0f54 to d906c81 Compare May 18, 2019 15:55

baronfel commented May 18, 2019

View reviewed changes

Add dockerfile for replicable environment while iterating

129e9dc

baronfel added 5 commits May 18, 2019 18:31

start including sections from the mono build for location defintions

4838d8f

apply makefile patch to include paths for VS v16.0 fsharp locations

Apply patch from mono packaging repository: https://github.com/mono/m…

ebc63f3

…ono/blob/master/packaging/MacSDK/patches/fsharp-noinstall.patch

applying patch https://github.com/mono/mono/blob/master/packaging/Mac…

033a425

…SDK/patches/fsharp-custom-prefix.patch to allow for custom prefixes

add in previous copies and start iterating on making paths correct

a55d861

Add new TFM for net45 since fsharp.core builds that as a baseline now

f37ef5e

baronfel force-pushed the makefile-updates-for-mono branch from 04d15b2 to 6f85e74 Compare May 19, 2019 15:20

run tests via nunit directly on mono

6f1bde0

baronfel force-pushed the makefile-updates-for-mono branch from 6f85e74 to 6f1bde0 Compare May 19, 2019 15:22

baronfel added 2 commits May 21, 2019 20:57

get makefile working for everything except old targetframeworkprofiles

a656980

Fix casing

9c3c48d

baronfel mentioned this pull request May 31, 2019

Fixed build tools paths dotnet/fsharp#6911

Closed

ninjarobot mentioned this pull request Jun 5, 2019

FSharp 4.6 Github Release dotnet/fsharp#6943

Closed

akoeplinger mentioned this pull request Nov 7, 2019

Bump F# to 4.7 mono/mono#16763

Closed

Makefile updates to get dotnet/fsharp building on mono #1

Are you sure you want to change the base?

Makefile updates to get dotnet/fsharp building on mono #1

Uh oh!

Conversation

baronfel commented May 17, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

baronfel commented May 18, 2019

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

baronfel commented May 18, 2019

Uh oh!

baronfel commented May 18, 2019

Uh oh!

baronfel commented May 19, 2019

Uh oh!

baronfel commented May 19, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

baronfel commented May 17, 2019 •

edited

Loading