MSC4195: MatrixRTC Transport using LiveKit Backend by hughns · Pull Request #4195 · matrix-org/matrix-spec-proposals

hughns · 2024-09-16T10:58:01Z

Rendered

Implementation

Backend: MatrixRTC Authorisation Service
- https://github.com/element-hq/lk-jwt-service/releases/tag/v0.4.1
Client: Element Call
- https://github.com/element-hq/element-call/releases/tag/v0.17.0-rc.1

Dependencies:

MSC4143: MatrixRTC #4143

To-do:

Define error responses for POST /get_token

proposals/4195-matrixrtc-livekit.md

networkException · 2025-05-18T14:39:51Z

proposals/4195-matrixrtc-livekit.md

+
+The homeserver restriction could be applied by checking the `matrix_server_name` field of the OpenID token before validating the token.
+
+The `room_id` could be validated too, and checking that the Matrix user from the OpenID token is a member of the room.


From my understanding there's no API that allows room_id validation. Without that implementing access control in a federated environment seems infeasible however.

Given that all members in a session use one SFU via oldest_membership, just validating against matrix_server_name (authenticated using TLS / OpenID) would require allowlisting any server that users might want to join from.

Ideally access control could check either for an allowlisted server or for an existing session with the oldest member being on an allowlisted server, gating an SFU to sessions being started by their "own" users.

The implementation in element-hq/lk-jwt-service#67 does allow limiting creation of new SFU rooms based on servers in a given allowlist, but still allows arbitrary users to join

With the context provided in element-hq/synapse#18967, I believe we can do better by not using ID tokens or a lk-jwt-service altogether.

Instead of a client authenticating against all remote SFUs using an ID token, an extended /transports Client-Server API can return matching credentials, obtained by the user's homeserver over federation. This removes the need for the somewhat bespoke ID token mechanism (which needs to do server name lookup over HTTP and DNS as a fallback. By design that also doesn't play nicely with MAS and OIDC native Matrix), reusing existing server to server authentication. Crucially, this allows for proper per-room tokens.

…seudonymous `livekit_alias` Derivation"

…urity considerations

onestacked · 2025-11-10T20:27:09Z

proposals/4195-matrixrtc-livekit.md

+End-to-end encryption is mapped into the LiveKit frame level encryption mechanism described
+[here](https://github.com/livekit/livekit/issues/1035).
+
+Where a shared password is used by the application it is used as the `string` input to the LiveKit
+key derivation function (which uses PBKDF2) and all participants use the same derived key for
+encryption and decryption.
+
+Where a per-participant key is used it is imported as the byte array input to the LiveKit key
+derivation function (which uses HKDF). The `index` field of the `m.rtc.encryption_keys` event is
+used as the key index for the key provider.
+
+On receipt of the `m.rtc.encryption_keys` event the application can associate the received key with
+the LiveKit participant identity by calculating the pseudonymous LiveKit participant identity as
+described above.


I would like more details on how to implement this.
I've been trying to get it to work in matrix-jukebox, but had no success in getting compatibility with Element Call.
(I'm implementing the old version of the spec without slots and multiSFU for now)

…llisions.

Airyzz · 2026-02-03T05:04:25Z

proposals/4195-matrixrtc-livekit.md

+encryption and decryption.
+
+Where a per-participant key is used it is imported as the byte array input to the LiveKit key
+derivation function (which uses HKDF). The `index` field of the `m.rtc.encryption_keys` event is


It seems like multiple LiveKit SDKs (Rust, Flutter) only support using PBKDF2, so using HKDF here is going to limit compatibility with non-web based clients out of the box

See:
livekit/rust-sdks#796
livekit/client-sdk-flutter#974

This also affects the SDKs using Rust-FFI (like python and maybe others)

turt2live · 2026-02-10T03:04:52Z

This is blocked on MSC4143's block: #4143 (comment)

Placeholder

e5cd38e

hughns changed the title ~~MatrixRTC using LiveKit backend~~ MSC4195: MatrixRTC using LiveKit backend Sep 16, 2024

Allocate MSC number

9fcfe3e

turt2live added voip proposal A matrix spec change proposal client-server Client-Server API kind:core MSC which is critical to the protocol's success needs-implementation This MSC does not have a qualifying implementation for the SCT to review. The MSC cannot enter FCP. labels Sep 25, 2024

Latest

6072744

hughns force-pushed the hughns/matrixrtc-livekit branch from 054a23c to 6072744 Compare December 17, 2024 07:22

anoadragon453 reviewed Jan 9, 2025

View reviewed changes

proposals/4195-matrixrtc-livekit.md Show resolved Hide resolved

inducer mentioned this pull request Apr 10, 2025

Allow restricting the set of home servers for which tokens are issued element-hq/lk-jwt-service#95

Closed

networkException reviewed May 18, 2025

View reviewed changes

ara4n added the matrix-2.0 Required for Matrix 2.0 label Sep 5, 2025

turt2live added this to Spec Core Team Workflow Sep 5, 2025

github-project-automation bot moved this to Tracking for review in Spec Core Team Workflow Sep 5, 2025

fkwp changed the title ~~MSC4195: MatrixRTC using LiveKit backend~~ MSC4195: MatrixRTC Transport using LiveKit Backend Oct 14, 2025

rewrite addressing a lot of feedback from offline discussions.

d93480c

fkwp marked this pull request as ready for review October 14, 2025 13:16

fkwp added 4 commits October 21, 2025 10:41

Address concern wrt. reliance of this MSC on the LiveKit protocol

394d044

add section about error responses

02e0783

address: error handling and information disclosure

d93eb77

cleanup structure

1567fd4

fkwp mentioned this pull request Nov 4, 2025

Implementation of MSC4195: MatrixRTC Transport using LiveKit Backend element-hq/lk-jwt-service#128

Merged

fkwp added 6 commits November 7, 2025 10:40

Add short intro

59af163

simplify livekit_alias section

1509c49

typo

04fa36c

add high level diagram to show the auth process

087515c

rename /sfu/get to more descriptive /get_token

9ed3677

Add section about the delegation of delayed events

3aca20e

fkwp added 5 commits November 7, 2025 16:03

M_INVALID_PARAM -> M_BAD_JSON

68f8868

fix example JSON snippets

fb0c402

"Pseudonymous livekit_alias" --> "Source of truly_random_bits for P…

b3b1688

…seudonymous `livekit_alias` Derivation"

make clear its the matrix room

5ee153a

remove alternative section as the topic is already handled in the sec…

870337b

…urity considerations

onestacked reviewed Nov 10, 2025

View reviewed changes

zecakeh mentioned this pull request Nov 24, 2025

client-api: Add unstable support for RTC transports discovery endpoint ruma/ruma#2298

Merged

fkwp and others added 4 commits December 8, 2025 11:32

add information about delegation of delayed events

a584a9e

namespace the pseudonymous LiveKit participant identity to prevent co…

ad88a5a

…llisions.

Specify the encoding of participant identities

a9fa009

Encode all hashes as unpadded base64

156da9d

turt2live added 00-weekly-pings Tracking for weekly pings in the SCT office. 00 to make it first in the labels list. and removed needs-implementation This MSC does not have a qualifying implementation for the SCT to review. The MSC cannot enter FCP. labels Jan 30, 2026

turt2live moved this from Tracking for review to Proposed for FCP readiness in Spec Core Team Workflow Jan 30, 2026

Airyzz reviewed Feb 3, 2026

View reviewed changes

turt2live added blocked Something needs to be done before action can be taken on this PR/issue. and removed 00-weekly-pings Tracking for weekly pings in the SCT office. 00 to make it first in the labels list. labels Feb 10, 2026

Update transport type to match current implementations

d5ef7e1

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

MSC4195: MatrixRTC Transport using LiveKit Backend#4195

MSC4195: MatrixRTC Transport using LiveKit Backend#4195
hughns wants to merge 24 commits intomatrix-org:mainfrom
hughns:hughns/matrixrtc-livekit

hughns commented Sep 16, 2024 •

edited by turt2live

Loading

Uh oh!

Uh oh!

networkException May 18, 2025

Uh oh!

networkException May 18, 2025

Uh oh!

networkException Dec 8, 2025 •

edited

Loading

Uh oh!

onestacked Nov 10, 2025

Uh oh!

Airyzz Feb 3, 2026

Uh oh!

onestacked Feb 6, 2026

Uh oh!

turt2live commented Feb 10, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

9 participants


		The homeserver restriction could be applied by checking the `matrix_server_name` field of the OpenID token before validating the token.

		The `room_id` could be validated too, and checking that the Matrix user from the OpenID token is a member of the room.

Conversation

hughns commented Sep 16, 2024 • edited by turt2live Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

networkException May 18, 2025

Choose a reason for hiding this comment

Uh oh!

networkException May 18, 2025

Choose a reason for hiding this comment

Uh oh!

networkException Dec 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

onestacked Nov 10, 2025

Choose a reason for hiding this comment

Uh oh!

Airyzz Feb 3, 2026

Choose a reason for hiding this comment

Uh oh!

onestacked Feb 6, 2026

Choose a reason for hiding this comment

Uh oh!

turt2live commented Feb 10, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

9 participants

hughns commented Sep 16, 2024 •

edited by turt2live

Loading

networkException Dec 8, 2025 •

edited

Loading