Add Rx.NET 2023 roadmap, and tooling update ADR

[idg10]

As #1868 states, one of the steps in bringing Rx.NET 'back to life' is to "Publish a Roadmap for 2023 (covering the .NET 8.0 LTS release train) & beyond".

This PR addresses that by setting out a road map. It also provides an Architectural Decision Record (ADR) describing the first proposed step (updating the Rx.NET) in more detail, along with all the context that informs that part of our proposals.

Resolves #1880

[idg10]

I don't know to what extent any of Rx's former contributors continue to have any interest in this project, but I thought it would be worth drawing these proposals to their attention. There might be historical folk knowledge that I don't have that would point against some of these suggestions.

So to that end, I am tagging @bartdesmet @danielcweber @akarnokd @clairernovotny @shiftkey @bash @quinmars @RxDave @Daniel-Svensson @mattpodwysocki @ghuntley @LeeCampbell @SimonCropp @anaisbetts @NewellClark @RyotaMurohoshi @M-Zuber @briangru @marodev @aelij @theodorzoulias @ryanwersal @cdmihai @cabauman @dotMorten @joelhoisko @ltrzesniewski @hez2010 @PJB3005 @jkotas @eugbaranov @NickDarvey @ts2do @olevett @alfredmyers @TobBrandt-Work @NickCraver @tcberryoxinst @rytswd @Dixin @khellang @jamesdoran @MichalStrehovsky @najuqvt @martsyn @sblom @georgis in case any of them have an opinion on this.

(I think that's everyone who ever made a commit in this repo that had a technical impact on Rx. Apologies if I missed anyway.)

[danielcweber]

I don't know to what extent any of Rx's former contributors continue to have any interest in this project

I'm planning to adopt Rx.Async in some form once it gets a release and would happily pick up contributing to this repo again (bug fixes, etc.). I still have write access to this repo from my batch of work back in 2018/9.

[Trinitek]

I'm curious, which camera API specifically? CameraCaptureUI?

[idg10]

I was using CaptureElement in XAML and a whole bunch of types from Windows.Media.Capture.

[HowardvanRooijen]

See PR #1882

[HowardvanRooijen]

See PR #1883

[Fijo]

About platform independent rx, test sharing and Unity

I just wanted to leave some information here in regards to IL2CPP compatibility and testing with Unity. In an attempt to reuse as much of the test code from this repository as possible and I've created a pretty unorthodox automation toolchain as part of the Rx.Unity repo.

The main reason I am making this post is because I saw you, talking about sharing tests so adoptions for different platforms (like UWP) can keep running them after they've been moved out of the main repository. Thus I thought I'd share my approach to give you some insight, inspiration and in hopes for a good solution that'll be easily maintainable for everyone.

Before I forget thank you so much for the work you've been putting in to bring Rx.Net back on track. I really appreciate it.

While that (Rx.Unity) needs updating again, which I am still committed to getting back to.
I'm also planning to do some more work on there once it's possible to use it along with the official System.Reactive NuGet Packages.

Why did I have to run all the tests?
Mostly to figure out what works with IL2CPP and in Unity in general.

How the build works (only focusing on what's relevant for Rx.net):

1. I clone the Rx.Net sources into a folder.
2. Patching the sources (described below).
3. Custom System.Reactive Build (described below).
4. Sideloading partial classes from tests (described below).
5. Including migrated UniRx-Tests to verify support for people migrating
6. Building and running tests in Unity (described below).

Patching the sources

Build on gulp I've scripted out a few tasks to transform the tests from the dotnet/reactive repo so they run within Unity.

Here are some of the things being done to rx sources themselves:

Patch System.Reactive.csproj

• TargetFramework => netstandard2.0
• LangVersion => 9
• DefineConstants => NO_NULLABLE_ATTRIBUTES
• SignAssembly => false

Patch './Rx.NET/Source/src/System.Reactive/TaskObservable.cs'

• Removing the [AsyncMethodBuilder(typeof(TaskObservableMethodBuilder<>))]-Attribute (Apparently this was unsupported / caused issues - I'll have to look into that again to find out more.)

Patch System.Reactive - Entire File replacements

I replace the following files with different ones or my own (Links lead to the replaced file):

• ./Rx.NET/Source/src/System.Reactive/Directory.build.props
• ./Rx.NET/Source/src/System.Reactive/Properties/InternalsVisibleTo.cs
• ./Rx.NET/Source/src/System.Reactive/Concurrency/SchedulerDefaults.cs (no longer relevant once this issue is addressed)
• ./Rx.NET/Source/src/global.json (just tooling stuff; workaround character / subject to change)

Patch XUnit to NUnit

Unitys tooling, as well as the runtime test runner I'm using only really works with NUnit. Adding too many additional dependencies is kind of painful (it would propobly work for assertations and I think I'll look if that's an option for when I'm adapting the now MS Test based sources.). Luckily a few replacements were enough to cover everything being used within the tests. Asyncronous Tests the trickiest here.
For more see xunit2nunitTransformer-Method.

Patches on Tests

• ThreadPoolScheduler.Instance is replaced with Rx.Unity.Concurrency.ThreadPoolOnlyScheduler.Instance in most test sources, as that's generally preferred when using Rx.Unity.
• public class is replaced with public partial class in most test sources, to allow additional setup / teardown logic to be provided by additional files.
• Various Tests are blacklisted by either making those methods private (via search and replace) or by excluding those files. The goal is ofc to get as much as possible to work in Unity but certain things just aren't there yet or would be a lot more difficult to troubleshoot. Alot of them cause the Tests to end up hanging up as well.

Custom System.Reactive Build

Building is done with dotnet build in ./Rx.NET/Source/src/System.Reactive.

=> Once the whole SchedulerDefaults.cs thing is fixed I think the whole InternalsVisibleTo.cs required for the tests is the only real thing preventing me from just using the NuGet Package of System.Reactive.

I'm only using the netstandard2.0 build for now. The latest version of Unity supports up to netstandard2.1, today. They are however working on a [https://forum.unity.com/threads/net-6-support.1059992/](full move to dotnet6) AFAIK.

Sideloading partial classes from tests

Alot of the test classes are being made partial and they are sideloaded with a custom SetUp/ TearDown-Methods specified in additional sources.
The main purpose of this is generally to switch the static instances within SchedulerDefaults to the more Unity compatible ones and to change them back on teardown.
In a few instances additional static methods are also provided (which are never actually used) but provide a workaround to force the AOT Linker to generate certain generic classes with the required specific type parameters as in rare cases it fails to make correct predictions. Having all the generic type combination being generated used to be a requirement for IL2CPP to work. Recently Unity have added support for generic type creation at runtime in IL2CPP but it is slow.

Building and running tests in Unity

Only test code and the source for Rx.Unity is build within Unity itself but not System.Reactive itself. The required files are copied into the appropriate folder by my script.

Unity tests are built and run in a via github actions. To ensure correct behaviour with IL2CPP an actual game is built including some framework that enables just running tests in that game. Then those tests are run via console. For Mac, iOS, Android and WebGL this isn't possible (or in the case of Mac I didn't get it to run that - I don't have a Mac so I couldn't troubleshoot it easily). Here an artefact with the build and a limited expiration is generated which can be manually downloaded, installed and run to verify all tests are working.

@idg10

[HowardvanRooijen]

@Fijo thanks very much for sharing that. We're very keen to enable better support for Unity, but it's an ecosystem we're unfamiliar with. If you're happy, we'll reach out to get your feedback when we get to those backlog items (most likely as part of v7 / v8)

[Thaina]

@HowardvanRooijen Are there any progress about changing default scheduler or anything to support other platform like unity at all?

[HowardvanRooijen]

Hi @Thaina

There are a number of things in flight at the moment, that we're trying to land in time for the .NET 9 launch (and .NET 6.0 going out of support, as well as giving Ix some TLC #2135 #2131 #2102) and general tidy up #2113 #2092 #2134

There was a big discussion around packaging... which would solve some of the problems we're facing #2038 - but I don't think we got to a consensus.

We've been blocked for a couple of months as the .NET Foundation has mandated a move to Managed Identities for code signing auth. This is a great idea, but since Rx was the first project to attempt this, it took a while to work out how to get the config right (dotnet-foundation/projects#387). Ian got that working this week (#2167), with the help of Chris Sfanos from the DNF, so progress is moving ahead again.

Ian has been looking at getting Async Rx into a better state:

• Add tests to AsyncRx.NET #1900 - specifically using Source Generators to transform the existing Rx test suite - Add tests to AsyncRx.NET #1900 (comment)
• Feature Request: Make AsyncGate.LockAsync cancellable #2148 - Feature Request: Make AsyncGate.LockAsync cancellable #2148 (comment)

We're hoping to do another yearly update either as part of .NET Conf or released independently at the same time.

[Thaina]

Thank you for your response

[HowardvanRooijen]

@Thaina - you're welcome! There's lots of stuff going on, to try and get all the bits ready for the next release. Hopefully we'll be able to make this a little clearer and more cohesive over the next month as we head towards the November milestone.

[khellang]

Thanks for the hard work you're putting into this, @HowardvanRooijen and team ❤️

[HowardvanRooijen]

@khellang it's all @idg10 hard work!

[Thaina]

@HowardvanRooijen Are there any progress about changing default scheduler or anything to support other platform like unity at all?

Next month of September seem so long

[HowardvanRooijen]

@idg10 is currently working on this and should have an update very soon (within the next week or so)

[Thaina]

Thank you very much

[Thaina]

@idg10 is currently working on this and should have an update very soon (within the next week or so)

@HowardvanRooijen Next week seem like forever

[HowardvanRooijen]

Are you keeping track of #2268 ?

Ian's current focus is getting the v7 release out with .NET 10 and recording a talk for .NET Conf 2025.

[Thaina]

@HowardvanRooijen Last week I have ask @idg10 and he said like it was actually unrelated

#2211

[HowardvanRooijen]

I think we're missing some useful context. For the avoidance of doubt, the v7 release is the thing we've been working towards for 2.5 years and that's to de-couple the platform specific dependencies, which should help solve a lot of these cross platform (Uno / Avalonia / Unity) problems. We're a couple of weeks away from this happening (.NET 10 release). Once this is done it will free up a HUGE amount of cognitive bandwidth, and will also clean up a number of open PRs which explored alternative approaches, which are cluttering up the repo.

I would, as Ian suggests create a new feature request / issue, so this isn't buried in a 2.5 year old complex issue, and please be as detailed as possible, as Unity is not a workload we have any experience with.

A sample project we can use to repro, with examples of the scheduler changes you propose would be very useful.

[Thaina]

@HowardvanRooijen My expectation of this project is that, I don't actually want this project to have anything to do with unity at all. I just want it to be compatible with .NET Standard 2.0 or 2.1 and so it's dll or even the code itself could be imported into unity directly

And then the most important things is to have #1628 (comment). One way to remove all kind of dependency workload from reactive system is to allow us to specify default scheduler by ourselves for each platform we work with. So that I could always implement scheduler for each platform specifically