MSC4075: MatrixRTC m.rtc.notification Call Ringing and Notifications

[toger5]

Rendered

To-do:

• How do you decline a call? MSC4310
• Implementations using revised format from 04635eb (#4075)
  • Ruma MatrixRTC: Add updated m.rtc.notification event (depreceate m.call.notify) ruma/ruma#2199
  • js-sdk Allow sending notification events when starting a call matrix-js-sdk#4826
• Implementation using revised formats with relations 7c996bb (#4075)
  • js-sdk Allow sending notification events when starting a call matrix-js-sdk#4826
  • Ruma MatrixRTC: Add updated m.rtc.notification event (depreceate m.call.notify) ruma/ruma#2199
• Implementation using revised formats with lifetime 6362c23 (#4075)
  • js-sdk Allow sending notification events when starting a call matrix-js-sdk#4826
  • Ruma MatrixRTC: Add updated m.rtc.notification event (depreceate m.call.notify) ruma/ruma#2199
• This looks very similar to m.call.invite but with much less info. Have we taken into account learnings from this existing capability?
  • edit by @toger5: looking into this it seems that the lifetime concept makes sense. It allows the sending client to know/control when the receiver stops ringing. Additional precautions need to take place to not allow for infinite rings. This is part of 6362c23 (#MSC4075)

Dependencies:

• MSC4143: MatrixRTC #4143

Implementations:

• Element Web
  • sending rings: Add ringing for matrixRTC matrix-react-sdk#11870
  • ringing: Add ringing for matrixRTC matrix-react-sdk#11870
• Ruma
  • notify event support: Add CallNotify event as described by MSC4075 ruma/ruma#1704
• Element X
  • android sending rings
  • android ringing
  • ios sending rings
  • ios ringing
• Extensible events fallback
  • Element web
  • Element X ios
  • Element X android

[uhoreg]

@toger5 You've added some implementations in the PR description. Do these implement the MSC in full, or are there parts of the MSC that are still lacking implementation?

[toger5]

Yes it's implemented in full. At least on the platforms shown in the description (EW).

There are optional configurations that are still missing. (Allowing to send a notification to a specific subset of users) but the API for this is already part of the SDK. There is no ui for this however and we are not sure we want that in EW.

[Half-Shot]

Noting that I'm playing around with the idea of including a media_hint field that gives a soft indication of whether the call is a "audio" or "video" (+audio) call. This is just to indicate between trusted callers that they can start with audio or video off, but the actual call semantics remain the same and they may switch at will. The intention is so that we can indicate this properly in notifications.

Suggested change

	"notification_type": "ring | notification",
	"notification_type": "ring | notification",
	"m.call.intent": "audio | video | something-else",

[Half-Shot]

In practice this is now in our implementation and has certainly been proven a useful addition

[BillCarsonFr]

Looks like the current implementation in js-sdk uses m.call.intenthttps://github.com/matrix-org/matrix-js-sdk/blob/4d0d32307eb4f1ce1fb65080fcca704f5bdedc31/src/matrixrtc/types.ts#L150

[BillCarsonFr]

Is it required?

[BillCarsonFr]

Also in this MSC it is never talking about calls, it is talking about rtc sessions. So maybe the naming is bad?
Maybe it should just be a rtc_intent that depends on the actual session beeing notified? like voice_call | video_call | white_board_invite ...

[Susurrus]

Ruma currently implements this with call_intent. It's useful to be able to devide if you notify someone that there's a "call" or a "video call" incoming.

[Susurrus]

I'm working on an implementation for Fractal (https://gitlab.gnome.org/World/fractal/-/issues/1720) and I believe this should be the event to anchor a rendering of the call in the timeline. If we're in a MatrixRTC world, what else would we use?

[toger5]

The long term plan on no matrixRTC timeline rendering is to use the m.rtc.member events they mark join and leaves.
A time based overlap computation of join/leave intervals (member sessions) will then be used to compute a historic call.

Similar to how we currently group other timeline events in clients (e.g. UserA and 10 other users join the room)
a new item would combine all the session information (UserA and 8 others participated in a call from X.XX until Y.YY)

[toger5]

But using the notification event is an acceptable stop gap. There are discussions if we want a specific notification as part of the m.call MSC instead of a general purpose m.notification event.

(mostly because that would allow a notification acknowlege response, which is needed for a reliable ring feedback UX)

So pls dont put too much time into the m.rtc.notification based approach. Make it simple.
Feel free to explore the "time based overlap computation of join/leave intervals" approach. It probably needs work in the rust sdk since this would be where the code should life.

If you look into this I would be very happy to be kept in the loop (ping me once a plan is laid out in an issue). (I already did some consideration on this topic and it would be unfortunate to waste resources due to a lack of communication)

[Susurrus]

I think it should be added to the MSC then, since it mentions this shouldn't be rendered, that it points to an MSC that contains the part that would be rendered. Otherwise this is confusing looking at this from the application side.

[BillCarsonFr]

Just to add that, regardless of push rules, having a separate event for ringing can also allow to easily configure Power Level to decide who can ring or not in a room

[Susurrus]

I think also if it's not a room mention then it should be ignored, right?

Tho I'm a bit confused by the mentions how they're used here, since they're an optional field on the message. So a client could send one with no mentions according to this spec, but every other client should ignore it then; is there a reason to allow that? I'd think m.mentions should be required with either room or have at least one user listed.