Notification tokens don't keep collections alive

[nirinchev]

When subscribing for notifications, there's an expectation that you'll receive notifications until you dispose of the token. In reality, if you don't keep a reference to the collection, the collection finalizer will run and unsubscribe from notifications. Reported on the forum.

[tekr]

Just to clarify and expand on my forum post, the collection subscription remains active if the calling code holds on to the token returned from subscription (not surprising since the implementation of NotificationToken holds a reference to the collection, preventing it being GC'd).

The Rx convention however is that it's not necessary to hold on to a returned subscription token if there's no need to unsubscribe. I.e. GC of the token shouldn't affect a subscription in any way. <-- (This last bit is not a necessary consequence of the previous statement)

[nirinchev]

That doesn't sound right though - if you don't hold a reference to the token and expect that it'll not get unsubscribed when GC'd, you're essentially asking for this callback to be invoked forever. I don't have much experience with Rx but as far as I remember, you'd need to either retain the subscription token or the observable source (in this case, the Realm collection). If the issue is that you're not retaining any references and expecting a subscription callback to be invoked for the lifetime of the application, then I'd say this goes against the design we've adopted and would be a recipe for memory leaks (imagine a UI control that subscribes for PropertyChanged but doesn't unsubscribe in its finalizer).

[tekr]

if you don't hold a reference to the token and expect that it'll not get unsubscribed when GC'd, you're essentially asking for this callback to be invoked forever."

Well, in the case of Realm events, at least until the Realm is closed or the object is deleted. And in the case of Rx, until the observable terminates. For example I think users have a reasonable expectation that the following should keep firing at least as long as _realm is valid/open:

   _realm.All<Foo>().AsRealmCollection().CollectionChanged += HandleCollectionChanged;

But as I understand it, currently the above handler will just stop being called at some non-deterministic time, and I believe you are suggesting that one should have to do this instead:

   _fooCollection = _realm.All<Foo>().AsRealmCollection();    // Save to instance variable
   _fooCollection.CollectionChanged += HandleCollectionChanged;

And I think this surprises Joe user. "I fetched a 'live' collection (lifted that term from Realm docs) and subscribed to its changed event. Why isn't it firing?!"

as far as I remember, you'd need to either retain the subscription token or the observable source (in this case, the Realm collection)

This generally isn't the case. Although I haven't yet found the definitive word on it (just the final section of this), I'm confident that it's by design to avoid issues like the above. To illustrate, all the below Rx subscriptions stay alive across continual GCs in release build, without holding any reference to the observable or subscription token:

   Observable.Interval(TimeSpan.FromSeconds(1)).Subscribe(o => Console.WriteLine("Foo"));

   Observable.Timer(TimeSpan.FromSeconds(10)).Subscribe(o => Console.WriteLine($"Bar"));

   Observable.Range(0, int.MaxValue).SubscribeOn(NewThreadScheduler.Default).Subscribe(_ =>
   {
       Console.WriteLine($"Baz");
       Thread.Sleep(1000);
   });

The third case is interesting because there's no timer to inadvertently keep the observable alive via a callback.

There's a sample project here indicating the Rx implementation keeps the returned token alive until the user explictly Dispose()s it or the observable terminates.

I wonder if Realm could possibly try a similar approach; e.g. hold references to subscription tokens (which keeps relevant Realm objects alive) per-Realm until they or the Realm are explicitly Dispose()d.

recipe for memory leaks (imagine a UI control that subscribes for PropertyChanged but doesn't unsubscribe in its finalizer).

Sure, but no doubt you'd agree that a) that's clearly a bug in the UI control, and b) the same leak occurs today as soon as the client keeps a long-lived reference to the observed Realm object, which is not an unusual thing to do. (Also to clarify, I assume you meant Dispose() rather than finalizer, since the latter wouldn't be invoked here).

[nirinchev]

Again, I don't have much experience with Rx, but with regular .NET events, there's certainly no expectation that an event subscription will prevent an event source from getting garbage collected (the opposite is not the case though - i.e. an event source will keep a subscriber alive). And the fact that an event source is GC-ed implies that no events will fire from it. You can refer to this SO answer for some practical example of this in action.

I decided to experiment a little more with Rx and this small app seems to confirm the same behavior with Rx's FromEventPattern:

class Source
{
    public event EventHandler MyEvent;

    ~Source()
    {
        Console.WriteLine("Source collected");
    }
}

static async Task Main(string[] args)
{
    var source = new WeakReference<Source>(new Source());
    Observable.FromEventPattern(
        x =>
        {
            if (source.TryGetTarget(out var liveSource))
            {
                Console.WriteLine("Subscribed to source");
                liveSource.MyEvent += x;
            }
        },
        x =>
        {
            if (source.TryGetTarget(out var liveSource))
            {
                Console.WriteLine("Unsubscribed to source");
                liveSource.MyEvent -= x;
            }
        })
        .Subscribe(_ => Console.WriteLine("Event emitted"));


    Console.WriteLine("Some GC loops");

    for (var i = 0; i < 100; i++)
    {
        GC.Collect();
        await Task.Delay(10);
    }

    Console.WriteLine($"After GC loops. Source is {(source.TryGetTarget(out _) ? "alive" : "dead")}");}

The output is

Subscribed to source
Some GC loops
Source collected
After GC loops. Source is dead

As you can see, I'm subscribing to an event source without retaining the source itself and that doesn't prevent the source from being collected. I don't believe there's a reasonable expectation that collected objects will emit events, so at some point events will stop.

And I think this surprises Joe user. "I fetched a 'live' collection (lifted that term from Realm docs) and subscribed to its changed event. Why isn't it firing?!"

I'd like to challenge that - I think users generally have an expectation that Realm objects and collections behave similarly to the framework types. This means that if no one keeps a reference to a Realm object/collection, it should be eligible for GC regardless of whether there's a subscriber somewhere or not. Additionally, in the docs page, in the first snippet, there's an explicit comment saying // Observe collection notifications. Retain the token to keep observing. and in the second snippet, the collection is clearly stored as a class-level field.

To be clear, I don't mean to invalidate your experience and it's very possible that our behavior doesn't follow precisely Rx conventions. That being said, Rx is used by a much smaller portion of our userbase compared to people using classic databinding with CollectionChanged/PropertyChanged events. We have to optimize for the majority use case and the majority expects collections not to leak when you subscribe to them.