-
Notifications
You must be signed in to change notification settings - Fork 434
MSC4425: Ephemeral media #4425
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
MSC4425: Ephemeral media #4425
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,117 @@ | ||
| # MSC4425: Ephemeral media | ||
|
|
||
| Media is typically expected to be included in events and thus exist for a length of time approaching | ||
| "forever". In the meantime, use cases like [MSC4268](https://github.com/matrix-org/matrix-spec-proposals/pull/4268) | ||
| are coming up which use the media repo for non-event (temporary) data storage. | ||
|
|
||
| This proposal introduces a mechanism for a media uploader to indicate that their upload should be | ||
| considered ephemeral. An endpoint to delete media (making it truly ephemeral) is also introduced. | ||
|
|
||
| Future proposals like [MSC3911](https://github.com/matrix-org/matrix-spec-proposals/pull/3911) and | ||
| [MSC4322](https://github.com/matrix-org/matrix-spec-proposals/pull/4322) may introduce additional | ||
| mechanics for ensuring in-event media deletion/redaction. | ||
|
|
||
|
|
||
| ## Proposal | ||
|
|
||
| On [`POST /_matrix/media/v1/create`](https://spec.matrix.org/v1.17/client-server-api/#post_matrixmediav1create) | ||
| and [`POST /_matrix/media/v3/upload`](https://spec.matrix.org/v1.17/client-server-api/#post_matrixmediav3upload), | ||
| a new `ephemeral` boolean query string parameter is added. `ephemeral` defaults to `false`. | ||
|
|
||
| When `ephemeral` is `true`, the uploader can later delete the media with a new endpoint: | ||
|
|
||
| ``` | ||
| DELETE /_matrix/client/v1/media/{origin}/{mediaId} | ||
| Authorization: ... | ||
|
|
||
| <empty request body> | ||
| ``` | ||
|
|
||
| This new endpoint requires authentication, can be rate limited, and returns the following: | ||
|
|
||
| * 200 OK + empty JSON object - The media was deleted or flagged as deleted on the server. | ||
| * 404 `M_NOT_FOUND` - The media was already deleted or was not found. | ||
| * 403 `M_FORBIDDEN` - The media was not uploaded by the user, was not found, or is not ephemeral. | ||
|
|
||
| The new endpoint is not available to guests because guests cannot upload media in the first place. | ||
|
|
||
| **Note**: Notifications to other servers/users are not sent when media is deleted. Read on for details | ||
| about cache expiration. | ||
|
|
||
| **Note**: Thumbnails for ephemeral media are also ephemeral themselves. They MUST be deleted at the | ||
| same time as the media object. | ||
|
|
||
| Over federation, ephemeral media is denoted in the media metadata during [download](https://spec.matrix.org/v1.17/server-server-api/#get_matrixfederationv1mediadownloadmediaid) | ||
| and [thumbnailing](https://spec.matrix.org/v1.17/server-server-api/#get_matrixfederationv1mediathumbnailmediaid), | ||
| like so: | ||
|
|
||
| ``` | ||
| Content-Type: multipart/mixed; boundary=gc0p4Jq0M2Yt08jU534c0p | ||
|
|
||
| --gc0p4Jq0M2Yt08jU534c0p | ||
| Content-Type: application/json | ||
|
|
||
| {"ephemeral":true} | ||
|
|
||
| --gc0p4Jq0M2Yt08jU534c0p | ||
| Content-Type: text/plain | ||
|
|
||
| Media content goes here | ||
| --gc0p4Jq0M2Yt08jU534c0p | ||
| ``` | ||
|
|
||
| Like the Client-Server API changes, `ephemeral` is optional and defaults to `false`. Values which aren't | ||
| booleans are considered `false` too. | ||
|
|
||
| When a server downloads ephemeral media (or its thumbnail) over federation, it MUST NOT cache that | ||
| response for longer than 1 day. This is to ensure two things: | ||
|
|
||
| 1. Later, when the media is deleted, the remote server also implicitly deletes it. | ||
| 2. Users cannot cause high bandwidth usage by cache-busting the media repo on upload. | ||
|
|
||
| Servers MAY be interested in applying harsher rate limits on ephemeral media than "regular" media to | ||
| further reduce resource consumption risks. | ||
|
|
||
|
|
||
| ## Potential issues | ||
|
|
||
| The new delete endpoint is not technically idempotent - clients will need to expect a 200, 403, or | ||
| 404 status code to confidently believe they have made the delete request. | ||
|
|
||
| Servers will likely need to track that they used the media ID even if the media has been deleted. | ||
| Otherwise, they might reuse the media ID and inappropriately grant access to the wrong user. | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. ..or we just make the media ID the hash of the media and don't have to worry about this? This is content-based addressing effectively.
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. That causes conflicts with several other media-related changes. Namely, anything which needs multiple owners, like events. |
||
|
|
||
| If ephemeral media is used in an event, that event might "break" for users later. This has larger | ||
| impact for media which is shared in multiple events, like custom emoji/stickers. Clients SHOULD NOT | ||
| use this proposal to upload media destined for an event. | ||
|
|
||
|
|
||
| ## Alternatives | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Rather than using a boolean, could the uploader specify a lifetime for the media after which servers could automatically purge it?
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The current use cases don't have a known lifetime for the media objects.
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The only currently listed use case is #4268 and based on matrix-org/matrix-rust-sdk#5113 (comment) this might be catered to with a static lifetime. Are there any other intended use cases that are currently known?
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'm not confident that picking a timer for invites is going to be successful. Like the (arbitrary) 1 day cache on this proposal, it risks keeping data for too long, well after the invite is accepted and processed. Previously QR code login used a similar media-based system, but has since changed. It also wouldn't work best if time bound.
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Ok, fair enough. Was just thinking that it might be an easy solution for the cache lifetime problem. But I see your reasoning.
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'm 100% with @Johennes on this. A lifetime specified at upload time is:
The use case in MSC4268 is indeed time-limited due to things like "how long after having accepted an invite will you try to access the key bundle" which is being set to ~1 day in the rust SDK. The key bundle is encrypted, so there isn't a pressing need to delete it NOWNOWNOW as soon as it is no longer needed. I'd rather have guaranteed deletion and have to hold onto the data for a bit longer than potentially having to hold onto the data forever if the client doesn't hit DELETE.
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Guaranteed deletion is not easier to implement: it requires actually sending out deletion requests to servers/users who may have downloaded it. Other proposals offer that functionality. Servers can also just delete ephemeral media at will, with or without client interaction. It's ephemeral, so it should be safe to purge. I'm not sure this is part of the proposal right now, but it should be. I feel a stronger use case is needed to change this proposal to use a timestamp.
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Why would you though? If it's lifetime-based, you can simply share the expiry timestamp, and therefore don't need to send any deletion request to other servers or even users. Yes, faulty clients/servers will potentially not delete it, but that's unavoidable. We only care about the compliant scenario.
But it's not safe to purge if it's a key bundle? It's unsafe to purge, but we may want to reserve the right to do that anyway more like?
I feel there needs to be stronger justification for a manual DELETE, particularly given both @Johennes and myself feel that for the MSC4268 use case an expiry time suffices.
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We can also say the earliest a server can delete something is a day after upload, to match the (currently arbitrary) 1 day cache. Aside from missing a strong use case, this proposal also just hasn't been open long enough to warrant a shift in how it works. Ideally, it gets implemented first to see how it works in practice. Or, it doesn't get implemented, which is effectively a downvote for the MSC. |
||
|
|
||
| This proposal is complimentary to proposals which have more sophisticated deletion or redaction | ||
| mechanisms. For example, proposals which link media to the events they are sent within. | ||
|
|
||
|
|
||
| ## Security considerations | ||
|
|
||
| If the cache value is too low, both remote and origin servers may experience abnormally high bandwidth | ||
| usage. Rate limits can help mitigate this. | ||
|
|
||
| If the cache value is too high, a later delete might get missed. Capping the server's value at 1 day | ||
| is a relatively arbitrary choice, but balances the likelihood of needing to make a second download | ||
| request with the ability for the media to disappear. | ||
|
|
||
| Clients might decide to flag all media, whether in events or otherwise, as ephemeral. If undesirable, | ||
| servers can impose stricter rate limits as they see fit. | ||
|
|
||
|
|
||
| ## Unstable prefix | ||
|
|
||
| While this proposal is not considered stable, implementations should use `org.matrix.msc4425.ephemeral` | ||
| instead of `ephemeral` and `DELETE /_matrix/client/unstable/org.matrix.msc4425/{origin}/{mediaId}` in | ||
| place of the stable endpoint. | ||
|
|
||
|
|
||
| ## Dependencies | ||
|
|
||
| This proposal has no direct dependencies. | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Implementation requirements: