Matrix v2 by Josue-T · Pull Request #439 · YunoHost-Apps/synapse_ynh

Josue-T · 2024-02-26T21:05:20Z

Matrix V2 Implementation

This is more a tracking page and discussion about Matrix v2 implementation than a pull request.

The target here is to implement all component of matrix V2 so:

~~Matrix sliding sync proxy: https://github.com/matrix-org/sliding-sync~~ Replaced by the synapse native implementation.
Element call: https://github.com/vector-im/element-call (ready for testing in Element call backend #528)
Matrix authentication service: https://github.com/matrix-org/matrix-authentication-service

The idea here is to provide for user how want a beta version of synapse which will implement theses new feature. Note that this branch will be updated in same time than the testing so user can easily install the branch and get the update as same as than for testing.

frequently asked questions

Why not merging this and making this available to more people ?

Because some component are still really experimental on matrix side. ~~On sliding proxy projet side we have currently theses issues:~~

This project are built to run only with new libc version which is not available on debian bullseye. A chroot was made to bypass this issue but it's not a clean way to implement this. The best solution about this is to wait for debian bookworm on yunohost.
~~The build of this project are available on for arm64 and amd64 architecture so it's really restricted.~~

How can I use this branch and experiment theses feature ?

As same as for testing you can install the app with this command:

sudo yunohost app install https://github.com/YunoHost-Apps/synapse_ynh/tree/matrix_v2

And you can also update the app with this command:

sudo yunohost app upgrade synapse -u https://github.com/YunoHost-Apps/synapse_ynh/tree/matrix_v2

Can I install this without breaking my instance ?

Short answer: the stability of this branch will be same than testing

Long answer: As same as for testing we try to avoid any issue with these branch and try to keep it stable. But it still be a community project and we can't guarantee the stability of the code. Note also that the stability depends of the upstream code.

When will this be merged into master ?

It depends of the development on package side but also of the upstream on matrix/element side so for now it's difficult to say. In all case the idea is not to merge before all component are ready and tested on the package.

Why adding element-call into this package as there are already a package with this

The idea here is to provide a package with all main feature of matrix configured directly into synapse so there is no more setup to do.

Can I switch back to master or testing branch after doing an upgrade from this branch ?

Nothing was designed for this, so there are a huge risk to break something or that something don't work any-more if you do this. But note that this branch in long term will be merged into testing than master so you won't be blocked if you move to this branch.

TODO

Finish Sliding sync proxy Sliding proxy #440
Implement Element call ~~(require Add possibility to declare subdomain requirement in app ressources YunoHost/issues#2563 to be implemented to implement this)~~
Implement Matrix authentication service (require App multiple db management for packaging v2 YunoHost/issues#2245, Add possibility to declare subdomain requirement in app ressources YunoHost/issues#2563 and Use YunoHost as an identity provider? YunoHost/issues#676 to be implemented to implement this)
Drop Debian bullseye support and clean-up sliding sync proxy implementation (when Yunohost will support officially debian bookworm).

lapineige · 2024-03-08T11:31:21Z

I want to congratulate @Josue-T for the huge recent work on both these improvements and work on packaging v2 #426 and various fixes 🎉 👏
Thanks also to all people contributing to this :)

Thatoo · 2024-03-08T11:47:38Z

Thank you so much @Josue-T for your incredible amount of work these last days.

centralscrutinizer21 · 2024-03-08T14:18:08Z

Adding my thanks to @Josue-T as well. I am going to test this branch and face the wrath of the 3 friends I convinced to get on my matrix instance 😄

centralscrutinizer21 · 2024-03-08T18:31:19Z

Installation went fine, upgrading not so much.
I still can't get to use Element x maybe something related to this warning?
https://paste.yunohost.org/agadaqileg
Also, do I need to open ports 8008 8009 ?

Josue-T · 2024-03-08T19:09:11Z

Normally all port are already open. But well what happen wrong. The upgrade process or the service was not working after the upgrade ?

centralscrutinizer21 · 2024-03-08T19:11:37Z

The service is running fine. Of course the two bridges (whatsapp & telegram) are not working anymore.
As I said I tried to login with Element X . Ii recognise the server as valid, but then it cannot reach it.
From the logs it seems that there could be some syntax erro maybe?

Josue-T · 2024-03-08T20:37:36Z

Do you have some log of theses bridges ?

centralscrutinizer21 · 2024-03-09T06:59:54Z

Sorry, my bad. I had a second application on the same domain, and that screwed up the installation.
Now it's working. Where do I find the procedure to move synapse to a different domain keeping the same user handle?

Josue-T · 2024-03-09T09:15:21Z

I created an issue about this here: #443

centralscrutinizer21 · 2024-03-09T09:36:43Z

Yes, thank you I found it and succesfully moved synapse to a different domain.
I am just not very proficient with the syntax of /etc/ssowat/conf.json.persistent can you show me how it should look?

Josue-T · 2024-03-09T09:50:55Z

About change url and domain constraints issue please post your message on #443

nathanael-h · 2024-03-19T08:08:10Z

Hello, I successfully upgraded to this branch. The Sliding Sync proxy's service is enabled and running well. As far as I see Synapse and the slinding sync proxy are installed on the same domain. Unfortunately I can't make Element X on Android connect to my server.

My matrix server is running at matrix.domain.fr and my main domain is domain.fr the .well-known are advertised here :

curl https://domain.fr/.well-known/matrix/client 
{
        "m.homeserver": { "base_url": "https://matrix.domain.fr" },
        "org.matrix.msc3575.proxy": { "url": "https://matrix.domain.fr"},
        "im.vector.riot.jitsi": {"preferredDomain": "jitsi.riot.im"},
        "im.vector.riot.e2ee": {"default": true }
    }%

When I try to connect Element X to domain.fr I have a "Nous n'avons pas pu atteindre ce serveur d'accueil. Vérifiez que vous avez correctement saisi l'URL du serveur d'accueil. Si l'URL est correcte contactez votre administrateur... etc."
I see this entries in logs nignx :

/var/log/nginx/domain.fr-access.log:90.66.163.140        - - [18/Mar/2024:18:17:03 +0100] "GET /.well-known/matrix/client HTTP/2.0" 200 273 "-" "Element X/0.4.5 (Fairphone FP3; Android 12; e_FP3-user 12 SQ3A.220705.004 eng.root.20240220.205127 dev-keys; Sdk TODO)"
/var/log/nginx/matrix.domain.fr-access.log:90.66.163.140 - - [18/Mar/2024:18:17:03 +0100] "GET /_matrix/client/versions HTTP/2.0" 200 929 "-" "Element X/0.4.5 (Fairphone FP3; Android 12; e_FP3-user 12 SQ3A.220705.004 eng.root.20240220.205127 dev-keys; Sdk TODO)"
/var/log/nginx/matrix.domain.fr-access.log:90.66.163.140 - - [18/Mar/2024:18:17:03 +0100] "GET /_matrix/client/v3/login HTTP/2.0" 200 170 "-" "Element X/0.4.5 (Fairphone FP3; Android 12; e_FP3-user 12 SQ3A.220705.004 eng.root.20240220.205127 dev-keys; Sdk TODO)"

I'd like to help you with this pull request and adding the sliding sync proxy, by the way thank you for all you've already done.

Josue-T · 2024-03-19T08:19:01Z

Hello, I think you should define the server as matrix.domain.fr and not domain.fr ? Normally this should works.

nathanael-h · 2024-03-19T09:15:10Z

I tried to login with matrix.domain.fr URL in Element X, but it says "Error this homeserver does not support sliding sync".

In nginx logs I see this :

root@domain:/etc/nginx/conf.d# grep "Element X" /var/log/nginx/*.log
/var/log/nginx/domain.fr-access.log:90.66.163.140 - - [19/Mar/2024:10:05:55 +0100] "GET /yunohost/sso/?r=aHR0cHM6Ly9tYXRyaXguaXNpZG9ydXxxxxxxxxxxx4vbWF0cml4L2NsaWVudA== HTTP/2.0" 200 3077 "-" "Element X/0.4.5 (Fairphone FP3; Android 12; e_FP3-user 12 SQ3A.220705.004 eng.root.20240220.205127 dev-keys; Sdk TODO)"
/var/log/nginx/domain.fr-access.log:90.66.163.140 - - [19/Mar/2024:10:06:05 +0100] "GET /yunohost/sso/?r=aHR0cHM6Ly9tYXRyaXguaXNpZG9ydXxxxxxxxxxxx4vbWF0cml4L2NsaWVudA== HTTP/2.0" 200 3077 "-" "Element X/0.4.5 (Fairphone FP3; Android 12; e_FP3-user 12 SQ3A.220705.004 eng.root.20240220.205127 dev-keys; Sdk TODO)"
/var/log/nginx/domain.fr-access.log:90.66.163.140 - - [19/Mar/2024:10:06:08 +0100] "GET /yunohost/sso/?r=aHR0cHM6Ly9tYXRyaXguaXNpZG9ydXxxxxxxxxxxx4vbWF0cml4L2NsaWVudA== HTTP/2.0" 200 3077 "https://matrix.domain.fr/.well-known/matrix/client" "Element X/0.4.5 (Fairphone FP3; Android 12; e_FP3-user 12 SQ3A.220705.004 eng.root.20240220.205127 dev-keys; Sdk TODO)"


/var/log/nginx/matrix.domain.fr-access.log:90.66.163.140 - - [19/Mar/2024:10:05:55 +0100] "GET /.well-known/matrix/client HTTP/2.0" 302 138 "-" "Element X/0.4.5 (Fairphone FP3; Android 12; e_FP3-user 12 SQ3A.220705.004 eng.root.20240220.205127 dev-keys; Sdk TODO)"
/var/log/nginx/matrix.domain.fr-access.log:90.66.163.140 - - [19/Mar/2024:10:06:05 +0100] "GET /.well-known/matrix/client HTTP/2.0" 302 138 "-" "Element X/0.4.5 (Fairphone FP3; Android 12; e_FP3-user 12 SQ3A.220705.004 eng.root.20240220.205127 dev-keys; Sdk TODO)"
/var/log/nginx/matrix.domain.fr-access.log:90.66.163.140 - - [19/Mar/2024:10:06:05 +0100] "GET /.well-known/matrix/client HTTP/2.0" 302 138 "-" "Element X/0.4.5 (Fairphone FP3; Android 12; e_FP3-user 12 SQ3A.220705.004 eng.root.20240220.205127 dev-keys; Sdk TODO)"
/var/log/nginx/matrix.domain.fr-access.log:90.66.163.140 - - [19/Mar/2024:10:06:07 +0100] "GET /.well-known/matrix/client HTTP/2.0" 302 138 "-" "Element X/0.4.5 (Fairphone FP3; Android 12; e_FP3-user 12 SQ3A.220705.004 eng.root.20240220.205127 dev-keys; Sdk TODO)"
/var/log/nginx/matrix.domain.fr-access.log:90.66.163.140 - - [19/Mar/2024:10:06:08 +0100] "GET /_matrix/client/versions HTTP/2.0" 200 929 "-" "Element X/0.4.5 (Fairphone FP3; Android 12; e_FP3-user 12 SQ3A.220705.004 eng.root.20240220.205127 dev-keys; Sdk TODO)"
/var/log/nginx/matrix.domain.fr-access.log:90.66.163.140 - - [19/Mar/2024:10:06:08 +0100] "GET /_matrix/client/versions HTTP/2.0" 200 929 "-" "Element X/0.4.5 (Fairphone FP3; Android 12; e_FP3-user 12 SQ3A.220705.004 eng.root.20240220.205127 dev-keys; Sdk TODO)"
/var/log/nginx/matrix.domain.fr-access.log:90.66.163.140 - - [19/Mar/2024:10:06:08 +0100] "GET /_matrix/client/v3/login HTTP/2.0" 200 170 "-" "Element X/0.4.5 (Fairphone FP3; Android 12; e_FP3-user 12 SQ3A.220705.004 eng.root.20240220.205127 dev-keys; Sdk TODO)"
root@domain:/etc/nginx/conf.d#

And if I curl https://matrix.domain.fr/.well-known/matrix/client I have a 302 to SSO ; indeed the .well-know/matrix/client is on the main yunohost domain, i.e. domain.fr... It looks like the Element X app is first calling the .well-know/matrix/client path on the URL I set as the homeserver URL, then nginx says 302 go to SSO, then it fails.

I see no entry in sliding-sync service's journal.

Josue-T · 2024-03-19T22:36:35Z

Hello,

Did some test, the issue is linked that Element X don't support CAS and if password authentication is disabled it don't works. By d42f646 it should work now.

nathanael-h · 2024-03-20T11:20:40Z

I confirm it works: I could log in Element X, indeed with username and password!

…orrectly

Updated from this doc: https://willlewis.co.uk/blog/posts/deploy-element-call-backend-with-synapse-and-docker-compose/?ref=element.io

Josue-T · 2025-04-23T23:06:18Z

Hello,

I've a good and a bad news 😄 The good one is that element call backend is now supported in matrix_v2 and I plan to merge this PR if everything is OK in some week.

And the bad one is that the MAS implementation will take a while mainly because of the OIDC integration which will be quite long so I'll won't wait on this work to close this PR.

Thatoo · 2025-04-24T04:20:31Z

That's already an awesome news. Thank you so much for this work done.
It means we'll be able to use ElementX from now on and won't have element call discontinuation (matrix.org closing their free SFU soon). That's the most important for now.
Thank you again.
MAS can indeed come later. There isn't yet any deadline for ElementX not supporting old authentication in favour of MAS.

Josue-T · 2025-05-07T23:21:01Z

!testme

yunohost-bot · 2025-05-07T23:21:05Z

🚀

Add setting to allow request on localhost

Josue-T · 2025-05-09T06:49:00Z

!testme

yunohost-bot · 2025-05-09T06:49:03Z

Alrighty!

This was linked to issues Feb 26, 2024

OpenID Connect #411

Open

sliding sync proxy? #395

Closed

Josue-T mentioned this pull request Mar 8, 2024

sliding sync proxy? #395

Closed

Josue-T added 3 commits March 8, 2024 12:09

First implementation of matrix sliding proxy

3a1d10d

Make it less warning on remove

61add5e

fix test

8b4d2e1

Josue-T force-pushed the matrix_v2 branch from 3069137 to 8b4d2e1 Compare March 8, 2024 11:10

Fix upgrade with old 3pids_email values

632ecc4

Josue-T added 2 commits March 9, 2024 11:20

Merge branch 'testing' into matrix_v2

1b522bd

Merge branch 'testing' into matrix_v2

1efee60

Josue-T and others added 2 commits March 22, 2024 12:13

Force enable password authentication as some client need it to work c…

507ac9d

…orrectly

Auto-update README

72b21bf

Josue-T force-pushed the matrix_v2 branch from b207ea7 to af846cd Compare March 22, 2024 11:15

Merge branch 'testing' into matrix_v2

24fb003

Josue-T and others added 7 commits April 22, 2025 23:40

Fix upgrade

080ed2a

Update homeserver config

6c01b3d

Updated from this doc: https://willlewis.co.uk/blog/posts/deploy-element-call-backend-with-synapse-and-docker-compose/?ref=element.io

Move env config for lk-jwt to environement file

6bb0213

fix tests

bee2eb4

Improve livekit config

7df15aa

Put SFU in subpath

77d137a

Auto-update READMEs

117665e

Josue-T marked this pull request as ready for review April 23, 2025 23:06

Josue-T added 3 commits April 28, 2025 22:48

Cleanup sliding sync well-known

8e94b46

Rework well-known and fix support for some edge cases

af29d40

Fix from CI

1615027

Josue-T mentioned this pull request May 2, 2025

Support for the i386 and armv7 architectures may be discontinued #535

Open

Josue-T and others added 6 commits May 7, 2025 21:21

Fix auto update

11ff119

Avoid systemd dependency to postgresql

a603e73

Fix checksum

0b2fbed

Upgrade synapse to 1.129.0

fdbab09

Merge remote-tracking branch 'upstream/testing' into matrix_v2

bda575d

Auto-update READMEs

0b35dc9

Josue-T and others added 4 commits May 8, 2025 01:22

Merge branch 'testing' into matrix_v2

2f0943b

Auto-update READMEs

d750b39

Add setting to allow request on localhost

f6bfa54

Merge pull request #540 from YunoHost-Apps/allow_request_localhost

259f280

Add setting to allow request on localhost

Josue-T merged commit 63ccda4 into testing May 12, 2025
1 check passed

Josue-T deleted the matrix_v2 branch May 12, 2025 06:26

Conversation

Josue-T commented Feb 26, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Matrix V2 Implementation

frequently asked questions

Why not merging this and making this available to more people ?

How can I use this branch and experiment theses feature ?

Can I install this without breaking my instance ?

When will this be merged into master ?

Why adding element-call into this package as there are already a package with this

Can I switch back to master or testing branch after doing an upgrade from this branch ?

TODO

Uh oh!

lapineige commented Mar 8, 2024

Uh oh!

Thatoo commented Mar 8, 2024

Uh oh!

centralscrutinizer21 commented Mar 8, 2024

Uh oh!

centralscrutinizer21 commented Mar 8, 2024

Uh oh!

Josue-T commented Mar 8, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

centralscrutinizer21 commented Mar 8, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Josue-T commented Mar 8, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

centralscrutinizer21 commented Mar 9, 2024

Uh oh!

Josue-T commented Mar 9, 2024

Uh oh!

centralscrutinizer21 commented Mar 9, 2024

Uh oh!

Josue-T commented Mar 9, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nathanael-h commented Mar 19, 2024

Uh oh!

Josue-T commented Mar 19, 2024

Uh oh!

nathanael-h commented Mar 19, 2024

Uh oh!

Josue-T commented Mar 19, 2024

Uh oh!

nathanael-h commented Mar 20, 2024

Uh oh!

Josue-T commented Apr 23, 2025

Uh oh!

Thatoo commented Apr 24, 2025

Uh oh!

Josue-T commented May 7, 2025

Uh oh!

yunohost-bot commented May 7, 2025

Uh oh!

Josue-T commented May 9, 2025

Uh oh!

yunohost-bot commented May 9, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

10 participants

Josue-T commented Feb 26, 2024 •

edited

Loading

Josue-T commented Mar 8, 2024 •

edited

Loading

centralscrutinizer21 commented Mar 8, 2024 •

edited

Loading

Josue-T commented Mar 8, 2024 •

edited

Loading

Josue-T commented Mar 9, 2024 •

edited

Loading