[wasm-mt] Issues with enabling threads in .NET 7 RC 1

[lambdageek]

Using the wasm-experimental workload it shoudl be possible to create multi-threaded wasmbrowser projects in .NET 7 RC1

Issues:

• First group - fixed on main by [wasm-mt] Fix pack/build issues in threaded builds #75162 and on release/7.0 by [release/7.0][wasm-mt] Fix pack/build issues in threaded builds #75171
  • WasmGenerateAppBundle doesn't create multi-threaded builds with WasmEnableThreads - only WasmBuildNative does - due to dotnet.worker.js missing from the runtime pack
  • WorkloadManifest.targets.in uses WasmEnableThreading instead of WasmEnableThreads
  • The CoreLibs in the multithread and perftrace build variants don't define the correct feature flags.
  • dotnet new wasmbrowser creates browser.csproj, not <dirname>.csproj
• dotnet new wasmbrowser does not create runtimeconfig.template.json required for dotnet run this looks like a local caching issue
• [wasm-mt] Emscripten SDK runtime pack is missing multi-threaded static libraries #75263

Detailed reproduction steps

1. Download a net7 RC 1 nightly tar.gz from dotnet/installer (I used the "Release/7.0.1xx-rc1 (7.0.x Runtime)" column)
2. Unpack into ${HOME}/work/net7-nightly
3. Set DOTNET_ROOT and PATH:

   export DOTNET_ROOT="${HOME}/work/net7-nightly"
   export PATH="${DOTNET_ROOT}:${PATH}"

4. Create the directory ${HOME}/work/net7-playground and add the following into ${HOME}/work/net7-playground/NuGet.config

   <configuration>
     <packageSources>
       <add key="dotnet7" value="https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet7/nuget/v3/index.json" />
     </packageSources>
   </configuration>

5. From the net7-playground directory (so that NuGet.config is in the current dir), install the wasm-experimental workload:

   cd ~/work/net7-playground
   dotnet workload install wasm-experimental

6. Make a sample project directory ${HOME}/work/net7-playground/hithread and create a wasmbrowser project:

   cd ~/work/net7-playground
   mkdir hithread
   cd hithread
   dotnet new wasmbrowser

7. Bug 1 note that there is browser.csproj not hithread.csproj - other templates create a .csproj file with the same name as the directory.
8. Bug 2 create the missing runtimeconfig.template.json with teh following content

   {
       "wasmHostProperties": {
           "perHostConfig": [
               {
                   "name": "browser",
                   "html-path": "index.html",
                   "Host": "browser"
               }
           ]
       }
   }

9. Add the follwoing to your browser.csproj

   <PropertyGroup>
     <WasmEnableThreads>true</WasmEnableThreads>
     <WasmEnableThreading>true</WasmEnableThreading>
     <WasmBuildNative>true</WasmBuildNative>
   </PropertyGroup>

   Bug 3 note that WorkloadManifest.targets.in has a typo and uses WasmEnableThreading instead of WasmEnableThreads
10. Run dotnet build

$ dotnet build
MSBuild version 17.4.0-preview-22416-02+5d102ae37 for .NET
Determining projects to restore...
Restored /Users/alklig/work/net7-playground/hithread/browser.csproj (in 2 ms).
/Users/alklig/work/net7-nightly/sdk/7.0.100-rc.2.22425.5/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.RuntimeIdentifierInference.targets(219,5): message NETSDK1057: You are using a preview version of .NET. See: https://aka.ms/dotnet-support-policy [/Users/alklig/work/net7-playground/hithread/browser.csproj]
browser -> /Users/alklig/work/net7-playground/hithread/bin/Debug/net7.0/browser-wasm/browser.dll
Compiling native assets with emcc with -O0. This may take a while ...
[2/3] corebindings.c -> corebindings.o [took 0.384s]
[1/3] pinvoke.c -> pinvoke.o [took 0.384s]
[3/3] driver.c -> driver.o [took 0.416s]
Linking for initial memory $(EmccInitialHeapSize)=536870912 bytes. Set this msbuild property to change the value.
Linking with emcc with -O0. This may take a while ...
...
Exception: FROZEN_CACHE is set, but cache file is missing: "sysroot/lib/wasm32-emscripten/libGL-mt.a" (in cache root path "/Users/alklig/work/net7-nightly/packs/Microsoft.NET.Runtime.Emscripten.3.1.12.Sdk.osx-x64/8.0.0-alpha.1.22415.5/tools/emscripten/cache")
/Users/alklig/work/net7-nightly/packs/Microsoft.NET.Runtime.WebAssembly.Sdk/7.0.0-rc.1.22422.12/Sdk/WasmApp.Native.targets(422,5): error MSB3073: The command "emcc "@/Users/alklig/work/net7-nightly/packs/Microsoft.NETCore.App.Runtime.Mono.multithread.browser-wasm/7.0.0-rc.1.22422.12/runtimes/browser-wasm/native/src/emcc-default.rsp" "@/Users/alklig/work/net7-nightly/packs/Microsoft.NETCore.App.Runtime.Mono.multithread.browser-wasm/7.0.0-rc.1.22422.12/runtimes/browser-wasm/native/src/emcc-link.rsp" "@/Users/alklig/work/net7-playground/hithread/obj/Debug/net7.0/browser-wasm/wasm/for-build/emcc-link.rsp"" exited with code 1. [/Users/alklig/work/net7-playground/hithread/browser.csproj]

Bug 4 We removed the *-mt.a static libraries in our EMSDK pack
9. Copy *-mt.a from an upstream EMSDK install into the emsdk pack:

cp ~/work/dotnet-runtime/runtime/src/mono/wasm/emsdk/upstream/emscripten/cache/sysroot/lib/wasm32-emscripten/*-mt.a ~/work/net7-nightly/packs/Microsoft.NET.Runtime.Emscripten.3.1.12.Sdk.osx-x64/8.0.0-alpha.1.22415.5/tools/emscripten/cache/sysroot/lib/wasm32-emscripten

10. dotnet build should now succeed.
    dotnet run shoudl serve the app. Open the URL in Chrome and open dev tools and hit reload. You should see something like this in the console:

MONO_WASM: worker initializing essential C exports and APIs
MONO_WASM: worker initializing essential C exports and APIs
MONO_WASM: worker initializing essential C exports and APIs
MONO_WASM: worker initializing essential C exports and APIs
dotnet.js:5 MONO_WASM: afterLoadWasmModuleToWorker added message event handler Worker {onmessage: ƒ, onerror: ƒ}
dotnet.js:5 MONO_WASM: afterLoadWasmModuleToWorker added message event handler Worker {onmessage: ƒ, onerror: ƒ}
dotnet.js:5 MONO_WASM: afterLoadWasmModuleToWorker added message event handler Worker {onmessage: ƒ, onerror: ƒ}
dotnet.js:5 MONO_WASM: afterLoadWasmModuleToWorker added message event handler Worker {onmessage: ƒ, onerror: ƒ}
dotnet.js:5 MONO_WASM: worker initializing essential C exports and APIs
dotnet.js:5 MONO_WASM: worker initializing essential C exports and APIs
dotnet.js:5 MONO_WASM: worker initializing essential C exports and APIs
dotnet.js:5 MONO_WASM: worker initializing essential C exports and APIs
dotnet.js:3 mono_wasm_runtime_ready fe00e07a-5519-4dfe-b35a-f867dbaf2e28
dotnet.js:2013 Hello, Console!

11. Clean the bin and obj directories.
    Change the browser.csproj like this:

<PropertyGroup>
  <WasmEnableThreads>true</WasmEnableThreads>
  <WasmEnableThreading>true</WasmEnableThreading>
  <WasmGenerateAppBundle>true</WasmGenerateAppBundle>
</PropertyGroup>

That is, replace WasmBuildNative by WasmGenerateAppBundle
12. run dotnet build again. Note that dotnet.worker.js isn't in the AppBundle directory. When running
the app prints 404s when trying to load it. From the Chrome DevTools console:

GET http://127.0.0.1:9000/dotnet.worker.js 404 (Not Found)
dotnet.worker.js:1          GET http://127.0.0.1:9000/dotnet.worker.js 404 (Not Found)
dotnet.worker.js:1          GET http://127.0.0.1:9000/dotnet.worker.js 404 (Not Found)
dotnet.worker.js:1          GET http://127.0.0.1:9000/dotnet.worker.js 404 (Not Found)
dotnet.js:5 MONO_WASM: afterLoadWasmModuleToWorker added message event handler Worker {onmessage: ƒ, onerror: ƒ}
dotnet.js:5 MONO_WASM: afterLoadWasmModuleToWorker added message event handler Worker {onmessage: ƒ, onerror: ƒ}
dotnet.js:5 MONO_WASM: afterLoadWasmModuleToWorker added message event handler Worker {onmessage: ƒ, onerror: ƒ}
dotnet.js:5 MONO_WASM: afterLoadWasmModuleToWorker added message event handler Worker {onmessage: ƒ, onerror: ƒ}

Tagging subscribers to 'arch-wasm': @lewing
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Using the wasm-experimental workload it shoudl be possible to create multi-threaded wasmbrowser projects in .NET 7 RC1

Issues:

• dotnet new wasmbrowser creates browser.csproj, not <dirname>.csproj
• dotnet new wasmbrowser does not create runtimeconfig.template.json required for dotnet run
• WorkloadManifest.targets.in uses WasmEnableThreading instead of WasmEnableThreads
• EMSDK nuget is missing *-mt.a libs needed for multi-threaded builds due to Delete selected libs from cache to shrink size emsdk#43
• WasmGenerateAppBundle doesn't create multi-threaded builds with WasmEnableThreads - only WasmBuildNative does

Detailed reproduction steps

1. Download a net7 RC 1 nightly tar.gz from dotnet/installer (I used the "Release/7.0.1xx-rc1 (7.0.x Runtime)" column)
2. Unpack into ${HOME}/work/net7-nightly
3. Set DOTNET_ROOT and PATH:

   export DOTNET_ROOT="${HOME}/work/net7-nightly"
   export PATH="${DOTNET_ROOT}:${PATH}"

4. Create the directory ${HOME}/work/net7-playground and add the following into ${HOME}/work/net7-playground/NuGet.config

   <configuration>
     <packageSources>
       <add key="dotnet7" value="https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet7/nuget/v3/index.json" />
     </packageSources>
   </configuration>

5. From the net7-playground directory (so that NuGet.config is in the current dir), install the wasm-experimental workload:

   cd ~/work/net7-playground
   dotnet workload install wasm-experimental

6. Make a sample project directory ${HOME}/work/net7-playground/hithread and create a wasmbrowser project:

   cd ~/work/net7-playground
   mkdir hithread
   cd hithread
   dotnet new wasmbrowser

7. Bug 1 note that there is browser.csproj not hithread.csproj - other templates create a .csproj file with the same name as the directory.
8. Bug 2 create the missing runtimeconfig.template.json with teh following content

   {
       "wasmHostProperties": {
           "perHostConfig": [
               {
                   "name": "browser",
                   "html-path": "index.html",
                   "Host": "browser"
               }
           ]
       }
   }

9. Add the follwoing to your browser.csproj

   <PropertyGroup>
     <WasmEnableThreads>true</WasmEnableThreads>
     <WasmEnableThreading>true</WasmEnableThreading>
     <WasmBuildNative>true</WasmBuildNative>
   </PropertyGroup>

   Bug 3 note that WorkloadManifest.targets.in has a typo and uses WasmEnableThreading instead of WasmEnableThreads
10. Run dotnet build

$ dotnet build
MSBuild version 17.4.0-preview-22416-02+5d102ae37 for .NET
Determining projects to restore...
Restored /Users/alklig/work/net7-playground/hithread/browser.csproj (in 2 ms).
/Users/alklig/work/net7-nightly/sdk/7.0.100-rc.2.22425.5/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.RuntimeIdentifierInference.targets(219,5): message NETSDK1057: You are using a preview version of .NET. See: https://aka.ms/dotnet-support-policy [/Users/alklig/work/net7-playground/hithread/browser.csproj]
browser -> /Users/alklig/work/net7-playground/hithread/bin/Debug/net7.0/browser-wasm/browser.dll
Compiling native assets with emcc with -O0. This may take a while ...
[2/3] corebindings.c -> corebindings.o [took 0.384s]
[1/3] pinvoke.c -> pinvoke.o [took 0.384s]
[3/3] driver.c -> driver.o [took 0.416s]
Linking for initial memory $(EmccInitialHeapSize)=536870912 bytes. Set this msbuild property to change the value.
Linking with emcc with -O0. This may take a while ...
...
Exception: FROZEN_CACHE is set, but cache file is missing: "sysroot/lib/wasm32-emscripten/libGL-mt.a" (in cache root path "/Users/alklig/work/net7-nightly/packs/Microsoft.NET.Runtime.Emscripten.3.1.12.Sdk.osx-x64/8.0.0-alpha.1.22415.5/tools/emscripten/cache")
/Users/alklig/work/net7-nightly/packs/Microsoft.NET.Runtime.WebAssembly.Sdk/7.0.0-rc.1.22422.12/Sdk/WasmApp.Native.targets(422,5): error MSB3073: The command "emcc "@/Users/alklig/work/net7-nightly/packs/Microsoft.NETCore.App.Runtime.Mono.multithread.browser-wasm/7.0.0-rc.1.22422.12/runtimes/browser-wasm/native/src/emcc-default.rsp" "@/Users/alklig/work/net7-nightly/packs/Microsoft.NETCore.App.Runtime.Mono.multithread.browser-wasm/7.0.0-rc.1.22422.12/runtimes/browser-wasm/native/src/emcc-link.rsp" "@/Users/alklig/work/net7-playground/hithread/obj/Debug/net7.0/browser-wasm/wasm/for-build/emcc-link.rsp"" exited with code 1. [/Users/alklig/work/net7-playground/hithread/browser.csproj]

Bug 4 We removed the *-mt.a static libraries in our EMSDK pack
9. Copy *-mt.a from an upstream EMSDK install into the emsdk pack:

cp ~/work/dotnet-runtime/runtime/src/mono/wasm/emsdk/upstream/emscripten/cache/sysroot/lib/wasm32-emscripten/*-mt.a ~/work/net7-nightly/packs/Microsoft.NET.Runtime.Emscripten.3.1.12.Sdk.osx-x64/8.0.0-alpha.1.22415.5/tools/emscripten/cache/sysroot/lib/wasm32-emscripten

10. dotnet build should now succeed.
    dotnet run shoudl serve the app. Open the URL in Chrome and open dev tools and hit reload. You should see something like this in the console:

MONO_WASM: worker initializing essential C exports and APIs
MONO_WASM: worker initializing essential C exports and APIs
MONO_WASM: worker initializing essential C exports and APIs
MONO_WASM: worker initializing essential C exports and APIs
dotnet.js:5 MONO_WASM: afterLoadWasmModuleToWorker added message event handler Worker {onmessage: ƒ, onerror: ƒ}
dotnet.js:5 MONO_WASM: afterLoadWasmModuleToWorker added message event handler Worker {onmessage: ƒ, onerror: ƒ}
dotnet.js:5 MONO_WASM: afterLoadWasmModuleToWorker added message event handler Worker {onmessage: ƒ, onerror: ƒ}
dotnet.js:5 MONO_WASM: afterLoadWasmModuleToWorker added message event handler Worker {onmessage: ƒ, onerror: ƒ}
dotnet.js:5 MONO_WASM: worker initializing essential C exports and APIs
dotnet.js:5 MONO_WASM: worker initializing essential C exports and APIs
dotnet.js:5 MONO_WASM: worker initializing essential C exports and APIs
dotnet.js:5 MONO_WASM: worker initializing essential C exports and APIs
dotnet.js:3 mono_wasm_runtime_ready fe00e07a-5519-4dfe-b35a-f867dbaf2e28
dotnet.js:2013 Hello, Console!

11. Clean the bin and obj directories.
    Change the browser.csproj like this:

<PropertyGroup>
  <WasmEnableThreads>true</WasmEnableThreads>
  <WasmEnableThreading>true</WasmEnableThreading>
  <WasmGenerateAppBundle>true</WasmGenerateAppBundle>
</PropertyGroup>

That is, replace WasmBuildNative by WasmGenerateAppBundle
12. run dotnet build again. Note that you get a single-threaded app
(dotnet.worker.js isn't in the AppBundle directory, when running
the app it doesn't print messages about initializing workers to
the console)
Bug 5 WasmGenerateAppBundle doesn't generate a multi-threaded app.

Author:	lambdageek
Assignees:	-
Labels:	arch-wasm, untriaged, area-Build-mono
Milestone:	-

[lambdageek]

@lewing @steveisok @radical @radekdoulik

[lambdageek]

@radekdoulik @lewing I guess our biggest problem is going to be

EMSDK nuget is missing *-mt.a libs needed for multi-threaded builds

because that runs up against the nuget size limit?

[radical]

@radekdoulik @lewing I guess our biggest problem is going to be

EMSDK nuget is missing *-mt.a libs needed for multi-threaded builds

because that runs up against the nuget size limit?

I can take care of the rest.

[lambdageek]

A bit more detail on teh last point: it seems like WasmGenerateAppBundle does work - except that the dotnet.worker.js file is missing from the multithread (and perftrace, I guess) runtime pack, so it doesn't get copied into the AppBundle

[lambdageek]

Maybe for net7 RC2 we can focus on WasmGenerateAppBundle+WasmEnableThreads as the "golden path" - that way we don't have to put the *-mt.a libs back in the Emscripten nuget yet. Anyone who wants to P/Invoke native code will be broken, but simple experiments with threading will work.

for net7 RTM we probably should have the *-mt.a stuff back in the Emscripten nuget, but if that's hard, maybe WasmBuildNative+WasmEnableThreads will have to wait for net8

[lambdageek]

Ah there's one more issue:

public class MyClass {
    [MethodImpl(MethodImplOptions.NoInlining)]
    public static string CallMeFromJS()
    {
        new Thread(Loop).Start();

        return "Hello, World!";
    }

    public static void Loop() {
        for (int i = 0; i < 5; ++i) {
            Console.WriteLine ("Thread ping!");
            Thread.Sleep (100);
        }
    }
}

results in

Which means we didn't build CoreLib for threading. Probably needs to check for MonoWasmBuildVariant when defining FeatureWasmThreads in the CoreLib proj. I'll start a PR