Skip to content
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
117 changes: 117 additions & 0 deletions proposals/4425-ephemeral-media.md
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Implementation requirements:

  • Server (uploading)
  • Server (downloading)
  • Client (using)

Original file line number Diff line number Diff line change
@@ -0,0 +1,117 @@
# MSC4425: Ephemeral media

Media is typically expected to be included in events and thus exist for a length of time approaching
"forever". In the meantime, use cases like [MSC4268](https://github.com/matrix-org/matrix-spec-proposals/pull/4268)
are coming up which use the media repo for non-event (temporary) data storage.

This proposal introduces a mechanism for a media uploader to indicate that their upload should be
considered ephemeral. An endpoint to delete media (making it truly ephemeral) is also introduced.

Future proposals like [MSC3911](https://github.com/matrix-org/matrix-spec-proposals/pull/3911) and
[MSC4322](https://github.com/matrix-org/matrix-spec-proposals/pull/4322) may introduce additional
mechanics for ensuring in-event media deletion/redaction.


## Proposal

On [`POST /_matrix/media/v1/create`](https://spec.matrix.org/v1.17/client-server-api/#post_matrixmediav1create)
and [`POST /_matrix/media/v3/upload`](https://spec.matrix.org/v1.17/client-server-api/#post_matrixmediav3upload),
a new `ephemeral` boolean query string parameter is added. `ephemeral` defaults to `false`.

When `ephemeral` is `true`, the uploader can later delete the media with a new endpoint:

```
DELETE /_matrix/client/v1/media/{origin}/{mediaId}
Authorization: ...

<empty request body>
```

This new endpoint requires authentication, can be rate limited, and returns the following:

* 200 OK + empty JSON object - The media was deleted or flagged as deleted on the server.
* 404 `M_NOT_FOUND` - The media was already deleted or was not found.
* 403 `M_FORBIDDEN` - The media was not uploaded by the user, was not found, or is not ephemeral.

The new endpoint is not available to guests because guests cannot upload media in the first place.

**Note**: Notifications to other servers/users are not sent when media is deleted. Read on for details
about cache expiration.

**Note**: Thumbnails for ephemeral media are also ephemeral themselves. They MUST be deleted at the
same time as the media object.

Over federation, ephemeral media is denoted in the media metadata during [download](https://spec.matrix.org/v1.17/server-server-api/#get_matrixfederationv1mediadownloadmediaid)
and [thumbnailing](https://spec.matrix.org/v1.17/server-server-api/#get_matrixfederationv1mediathumbnailmediaid),
like so:

```
Content-Type: multipart/mixed; boundary=gc0p4Jq0M2Yt08jU534c0p

--gc0p4Jq0M2Yt08jU534c0p
Content-Type: application/json

{"ephemeral":true}

--gc0p4Jq0M2Yt08jU534c0p
Content-Type: text/plain

Media content goes here
--gc0p4Jq0M2Yt08jU534c0p
```

Like the Client-Server API changes, `ephemeral` is optional and defaults to `false`. Values which aren't
booleans are considered `false` too.

When a server downloads ephemeral media (or its thumbnail) over federation, it MUST NOT cache that
response for longer than 1 day. This is to ensure two things:

1. Later, when the media is deleted, the remote server also implicitly deletes it.
2. Users cannot cause high bandwidth usage by cache-busting the media repo on upload.

Servers MAY be interested in applying harsher rate limits on ephemeral media than "regular" media to
further reduce resource consumption risks.


## Potential issues

The new delete endpoint is not technically idempotent - clients will need to expect a 200, 403, or
404 status code to confidently believe they have made the delete request.

Servers will likely need to track that they used the media ID even if the media has been deleted.
Otherwise, they might reuse the media ID and inappropriately grant access to the wrong user.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

..or we just make the media ID the hash of the media and don't have to worry about this? This is content-based addressing effectively.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That causes conflicts with several other media-related changes. Namely, anything which needs multiple owners, like events.


If ephemeral media is used in an event, that event might "break" for users later. This has larger
impact for media which is shared in multiple events, like custom emoji/stickers. Clients SHOULD NOT
use this proposal to upload media destined for an event.


## Alternatives
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rather than using a boolean, could the uploader specify a lifetime for the media after which servers could automatically purge it?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The current use cases don't have a known lifetime for the media objects.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The only currently listed use case is #4268 and based on matrix-org/matrix-rust-sdk#5113 (comment) this might be catered to with a static lifetime.

Are there any other intended use cases that are currently known?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not confident that picking a timer for invites is going to be successful. Like the (arbitrary) 1 day cache on this proposal, it risks keeping data for too long, well after the invite is accepted and processed.

Previously QR code login used a similar media-based system, but has since changed. It also wouldn't work best if time bound.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, fair enough. Was just thinking that it might be an easy solution for the cache lifetime problem. But I see your reasoning.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm 100% with @Johennes on this. A lifetime specified at upload time is:

  • easier to implement
  • guarantees expiry (whereas we can't guarantee the client will hit DELETE)

The use case in MSC4268 is indeed time-limited due to things like "how long after having accepted an invite will you try to access the key bundle" which is being set to ~1 day in the rust SDK. The key bundle is encrypted, so there isn't a pressing need to delete it NOWNOWNOW as soon as it is no longer needed. I'd rather have guaranteed deletion and have to hold onto the data for a bit longer than potentially having to hold onto the data forever if the client doesn't hit DELETE.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Guaranteed deletion is not easier to implement: it requires actually sending out deletion requests to servers/users who may have downloaded it. Other proposals offer that functionality.

Servers can also just delete ephemeral media at will, with or without client interaction. It's ephemeral, so it should be safe to purge. I'm not sure this is part of the proposal right now, but it should be.

I feel a stronger use case is needed to change this proposal to use a timestamp.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Guaranteed deletion is not easier to implement: it requires actually sending out deletion requests to servers/users who may have downloaded it.

Why would you though? If it's lifetime-based, you can simply share the expiry timestamp, and therefore don't need to send any deletion request to other servers or even users. Yes, faulty clients/servers will potentially not delete it, but that's unavoidable. We only care about the compliant scenario.

Servers can also just delete ephemeral media at will, with or without client interaction. It's ephemeral, so it should be safe to purge.

But it's not safe to purge if it's a key bundle? It's unsafe to purge, but we may want to reserve the right to do that anyway more like?

I feel a stronger use case is needed to change this proposal to use a timestamp.

I feel there needs to be stronger justification for a manual DELETE, particularly given both @Johennes and myself feel that for the MSC4268 use case an expiry time suffices.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can also say the earliest a server can delete something is a day after upload, to match the (currently arbitrary) 1 day cache.

Aside from missing a strong use case, this proposal also just hasn't been open long enough to warrant a shift in how it works. Ideally, it gets implemented first to see how it works in practice. Or, it doesn't get implemented, which is effectively a downvote for the MSC.


This proposal is complimentary to proposals which have more sophisticated deletion or redaction
mechanisms. For example, proposals which link media to the events they are sent within.


## Security considerations

If the cache value is too low, both remote and origin servers may experience abnormally high bandwidth
usage. Rate limits can help mitigate this.

If the cache value is too high, a later delete might get missed. Capping the server's value at 1 day
is a relatively arbitrary choice, but balances the likelihood of needing to make a second download
request with the ability for the media to disappear.

Clients might decide to flag all media, whether in events or otherwise, as ephemeral. If undesirable,
servers can impose stricter rate limits as they see fit.


## Unstable prefix

While this proposal is not considered stable, implementations should use `org.matrix.msc4425.ephemeral`
instead of `ephemeral` and `DELETE /_matrix/client/unstable/org.matrix.msc4425/{origin}/{mediaId}` in
place of the stable endpoint.


## Dependencies

This proposal has no direct dependencies.