Skip to content

Adds Element Call support, call UI elements, and rudimentary widget support#2335

Closed
GigiaJ wants to merge 258 commits intocinnyapp:devfrom
GigiaJ:dev
Closed

Adds Element Call support, call UI elements, and rudimentary widget support#2335
GigiaJ wants to merge 258 commits intocinnyapp:devfrom
GigiaJ:dev

Conversation

@GigiaJ
Copy link
Copy Markdown
Contributor

@GigiaJ GigiaJ commented May 23, 2025

Description

Adds Element Call support for both the npm embedded package as well as a hosted instance.
Adds a level of rudimentary widget support (not entirely sure of the extent even needed).
Adds calling interface elements such as a nav bar status section with a mute/unmute button, video enable/disable button, and hang up button. Icons are somewhat of a placeholder, but are relatively fitting.
Added a segmentation between Text rooms and Call rooms and provided an icon for call rooms (need to add the visibility icon support such as public/private/etc)
Added a --disabled for now-- icon for calling in DMs in the room header
Added dual iframe state juggling to provide lobby support akin to Discord where a user might wish to view another call's lobby or text channel
Added a text channel toggle to show or hide the room text channel
Added user icons and usernames to appear in the room nav to provide call activity visibility akin to Discord's previous interface implementation
Added mobile support for calling and text channel swapping and made the default behavior on mobile simply place you in the lobby and never in a call (does currently pop on your camera and mic, but is not joined to a call.)
Adjusted Room Header to account for a call room vs text room and behave accordingly

** Added a call or text room creation option in the room creation menu
Probably some other bits I'm forgetting, apologies. I feel like this is at a good enough stage to open a dialogue and move forward with, so I'm opening a PR.

Fixes #
#528

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

Checklist:

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas (I did, but comments didn't match the code style of the project. Happy to retroactively add them though.)
  • I have made corresponding changes to the documentation (I'd happily add docs pertaining to calling or simply hosting the standalone element call and SFU)
  • My changes generate no new warnings (The iframes probably will always generate some level as widgets are created and destroyed)

GigiaJ added 30 commits April 14, 2025 04:28
add groundwork for intercepting the read_events so we can correctly process them (not sure what is wrong but I believe there is a weird set of version mismatches between the embedded app, cinny, and the matrix widget api)
Adds a separator for rooms based on voice vs text rooms
still need to fix embedded behavior (standalone works for testing rn)
Copy link
Copy Markdown

@CEbbinghaus CEbbinghaus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I cannot comment much on most of the UI code since I'm not overly familiar with frontend dev.

One big bit of UX to note is that unlike element, only "call" rooms have call functionality which sucks since it requires moving to entirely different rooms almost completely dedicated to calling.

Instead element allows for calling in any room via the top call button. It would be great if cinny could support this too. Especially given cinny (with this PR) isn't even able to accept a call initiated in a DM from a element client.

Comment on lines +130 to +136
const errcodes = [
'M_UNKNOWN',
'M_BAD_JSON',
'M_ROOM_IN_USE',
'M_INVALID_ROOM_STATE',
'M_UNSUPPORTED_ROOM_VERSION',
];
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A lot of these changes are simply formatting changes, Probably best to reset all purely cosmetic changes to declutter the PR and make reviewing easier. Best to make stylistic changes in a separate PR

const showRoom = roomToUnread.has(roomId) || roomId === selectedRoomId;
if (showRoom) return false;
return true;
const showRoomAnyway =
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why rename this to showRoomAnyways. showRoom is probably sufficient.

count: hierarchy.length,
getScrollElement: () => scrollRef.current,
estimateSize: () => 0,
estimateSize: () => 32,
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pardon the novice question but what is the purpose of estimateSize and why are we changing it to 32?

Comment on lines +223 to +235
{!isSpace && (
<SettingTile
title="Room type"
options={
<SegmentControl
selected={roomType}
segments={[{ text: 'Text' }, { text: 'Call' }]}
onSelect={setRoomType}
/>
}
content={<Text variant="b3">Select the type of room.</Text>}
/>
)}
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apologies for the question but are matrix rooms locked into only one of the two options?

After having tested it there is always the ability to chat in a room with calls. and on element it is always possible to call in a room with chat (not so for cinny it seems), so why the arbitrary limit (note that element does support "call" rooms which I didn't know existed.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note however that I haven't yet found a way to create a call only room in element...

Copy link
Copy Markdown

@shootie22 shootie22 Feb 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

edit: May only be a thing on the Element client.

I believe you may only be able to create a call-only room inside of a space. You click on the + button next to the space name, then New video room:
image
The result is a room which has the Video icon and text displayed under its name, to indicate it's a video room. The difference is that on desktop, it auto-throws you into the Element Call UI as soon as you click on the room. The room does still have a chat as usual.
image

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmm that would make sense. Having call specific rooms I'n spaces seems sensible. Less so for just any odd room being created

Copy link
Copy Markdown

@CEbbinghaus CEbbinghaus Feb 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As it turns out element is able to create calls rooms outside of spaces too. But regardless, While call rooms being supported in cinny is awesome. Calls in arbitrary rooms & DMs is the primary feature Most will be looking for out of this PR

@shootie22
Copy link
Copy Markdown

shootie22 commented Feb 4, 2026

A few things which I see are holding this back:

  • Conflicts must be resolved
  • No voice call & video call functionality exists in DMs, only within spaces

Decisions and implementations would probably be needed for:

  • Voice & video calls only work on designated "video rooms" within spaces. Would being able to call any room be desirable? (like on Element) What would that look like?
    I would argue supporting this is important in order to maintain compatibility with other clients.
  • UI & functionality needed for DM calling
  • Is keeping the Element Call embedded UI active while in a voice chat desirable? It uses up a good amount of screen space without adding much. If removed, functionality for displaying screen sharing, hand raising and reactions would be needed.
    A simple option I see is having a setting to hide / show it by default (perhaps also a button to do it from the channel UI). Maybe show it when someone is sharing their screen. Showing reactions, hand raises and screen sharing could be shown as a badge next to the name of the user doing the action in voice chat.

But from a practical point of view, the conflicts are the first blocker. This PR is 9mo old, I'm unsure how easy it would be to resolve them. Would it be an easier and faster option to try to reimplement the changes in the latest version of Cinny?

<RoomViewHeader />
<Box grow="Yes">
<CallView room={room} />
{room.isCallRoom() && screenSize === ScreenSize.Desktop && (
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think most people would like all rooms to have a call icon, not just call rooms. Although, only call rooms should have the widget shown in the chat window by default.

Copy link
Copy Markdown

@shootie22 shootie22 Feb 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As an alternative, I propose:

  • Every room has the regular # icon by default, indicating it's a text chat
  • When someone initiates a call in any room, the # icon changes into a speaker icon, indicating there's an active call.

I think that would be cleaner.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that doesn't stop the need for a button in the top bar of every chat
image
(around here)

to initiate a voice call even in non-call channels.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

100% agree, I misunderstood what you were referring to.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No stress. Had a feeling. That's why we clarify 😄

"ua-parser-js": "1.0.35"
},
"devDependencies": {
"@element-hq/element-call-embedded": "0.12.2",
Copy link
Copy Markdown

@wrenix wrenix Feb 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"@element-hq/element-call-embedded": "0.12.2",
"@element-hq/element-call-embedded": "0.16.3",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if you bump the element call version you will need to change src/app/features/call/SmallWidget.ts a bit on line 59

replacing hideHeader: 'true', with header: 'none', because hideheader doesnt exist anymore

and adding confineToRoom: 'true', to hide the weird "call lobby" ui that they automatically put you in when you leave the call

"jotai": "2.6.0",
"linkify-react": "4.1.3",
"linkifyjs": "4.1.3",
"matrix-widget-api": "1.11.0",
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"matrix-widget-api": "1.11.0",
"matrix-widget-api": "1.17.0",

"@atlaskit/pragmatic-drag-and-drop-auto-scroll": "1.3.0",
"@atlaskit/pragmatic-drag-and-drop-hitbox": "1.0.3",
"@fontsource/inter": "4.5.14",
"@matrix-org/react-sdk-module-api": "2.5.0",
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@YoJames2019
Copy link
Copy Markdown
Contributor

YoJames2019 commented Feb 8, 2026

would love to help with this and have some code changes but I cannot suggest changes to a file that isnt included in this pr for some reason.
I'm very unfamiliar with contributing to public repositories on github so if someone could help me out a bit with the github side of things (or even just point to a resource that would help get me up to speed) i would really appreciate it.
(i would just ask my questions here but dont want to clog up the pr with unnecessary dialogue)

Edit: figured most of it out

@YoJames2019
Copy link
Copy Markdown
Contributor

YoJames2019 commented Feb 10, 2026

The previous implementation of the actual call/lobby display was a bit jank imo, so I made a bunch of improvements to it and replaced the element call lobby with a custom one
heres the current state of it, just currently waiting for GigiaJ to review the changes

@YoJames2019
Copy link
Copy Markdown
Contributor

cinny-call.mp4

@YoJames2019
Copy link
Copy Markdown
Contributor

another change is that the ui shows the call controls even while not in call, so if you want to join a call muted you can do that

@Mondrethos
Copy link
Copy Markdown

(i would just ask my questions here but dont want to clog up the pr with unnecessary dialogue)

Ask your questions here or in the relevant issue.

@JadedBlueEyes
Copy link
Copy Markdown

@YoJames2019 you can make a pull request to this user's branch on their repo, or just create a new PR to cinny itself

@HmonsterG
Copy link
Copy Markdown

What's the progress on this looking like, as im sure many people will be considering a migration from discord to cinny

@YoJames2019
Copy link
Copy Markdown
Contributor

YoJames2019 commented Feb 10, 2026

@YoJames2019 you can make a pull request to this user's branch on their repo, or just create a new PR to cinny itself

I made a pr to GigiaJs branch, waiting on review and approval, if anyone else wants to go review it and suggest changes to the code it is very welcome

@YoJames2019
Copy link
Copy Markdown
Contributor

Currently starting work on resolving conflicts with my branch and the current cinny dev branch and getting it up to date

@CogumeloNoGIT
Copy link
Copy Markdown

I wonder what will happen with cinny-desktop since tauri doesn't really support webrtc on linux

@hazre
Copy link
Copy Markdown
Contributor

hazre commented Feb 10, 2026

I think the cinny-desktop tauri v2 PR would probably fix that.

@Mondrethos
Copy link
Copy Markdown

@YoJames2019 you can make a pull request to this user's branch on their repo, or just create a new PR to cinny itself

I made a pr to GigiaJs branch, waiting on review and approval, if anyone else wants to go review it and suggest changes to the code it is very welcome

You might have to PR against cinny. I don’t think GigiaJs has been around in over 6 months but I could be wrong.

@hazre
Copy link
Copy Markdown
Contributor

hazre commented Feb 10, 2026

Yeah we might need a fork of GigiaJ's PR that has been rebased against upstream. So basically a new PR. I wanted to do that if we didn't get any response from GigiaJ but looks like @YoJames2019 might be ahead of me

@YoJames2019
Copy link
Copy Markdown
Contributor

YoJames2019 commented Feb 11, 2026

Yeah we might need a fork of GigiaJ's PR that has been rebased against upstream. So basically a new PR. I wanted to do that if we didn't get any response from GigiaJ but looks like @YoJames2019 might be ahead of me

I have literally 0 experience with this type of thing so it will probably take me a lot longer to figure out how this works.

Here's my fork of this pr with all the changes i made above, you're very welcome to try to rebase it against upstream and make a new pr against cinny if you have experience since i dont see it getting pulled in to this pr anytime soon on account of GigiaJ being MIA
https://github.com/YoJames2019/cinny/tree/GigiaJ-patch-2

@YoJames2019
Copy link
Copy Markdown
Contributor

after that we can actually work on direct message call support and polishing it up

@LinuxinaBit
Copy link
Copy Markdown

LinuxinaBit commented Feb 11, 2026

This looks so nice.
The only thing that really needs the polish on the UI front seems to be the persistent call bar, to integrate it better with the rest of Cinny's UI.
Everything else looks basically perfectly done. Great job! ❤️

@shootie22
Copy link
Copy Markdown

Yeah we might need a fork of GigiaJ's PR that has been rebased against upstream. So basically a new PR. I wanted to do that if we didn't get any response from GigiaJ but looks like @YoJames2019 might be ahead of me

I have literally 0 experience with this type of thing so it will probably take me a lot longer to figure out how this works.

Here's my fork of this pr with all the changes i made above, you're very welcome to try to rebase it against upstream and make a new pr against cinny if you have experience since i dont see it getting pulled in to this pr anytime soon on account of GigiaJ being MIA

https://github.com/YoJames2019/cinny/tree/GigiaJ-patch-2

I've had a very busy week but I will try to get some time on the weekend to jump onto this. 🤞

@Doc-4
Copy link
Copy Markdown

Doc-4 commented Feb 11, 2026

These additions are exactly what I need to get friends to move over, excellent PR! The desktop version performance is rough on Nvidia Linux due to having to use WEBKIT_DISABLE_DMABUF_RENDERER=1, but the PWA is fine for now, and will likely see these features sooner. Thanks for your hard work!

@hazre
Copy link
Copy Markdown
Contributor

hazre commented Feb 11, 2026

Yeah we might need a fork of GigiaJ's PR that has been rebased against upstream. So basically a new PR. I wanted to do that if we didn't get any response from GigiaJ but looks like @YoJames2019 might be ahead of me

I have literally 0 experience with this type of thing so it will probably take me a lot longer to figure out how this works.

Here's my fork of this pr with all the changes i made above, you're very welcome to try to rebase it against upstream and make a new pr against cinny if you have experience since i dont see it getting pulled in to this pr anytime soon on account of GigiaJ being MIA https://github.com/YoJames2019/cinny/tree/GigiaJ-patch-2

I've done it, here's the new PR #2599

@kfiven
Copy link
Copy Markdown
Collaborator

kfiven commented Feb 14, 2026

Closing in favor of #2599

@kfiven kfiven closed this Feb 14, 2026
@github-actions github-actions bot locked and limited conversation to collaborators Feb 14, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.