Expire sliding sync connections#19211
Merged
erikjohnston merged 16 commits intodevelopfrom Nov 25, 2025
Merged
Conversation
When checking which might have had updates, instead of just checking the in-memory cache of recent data also check the DB tables. This gives a more accurate view of how much data there is to send down when either a) the position is old or b) the server has recently restarted and so has a smaller cache.
It's generally faster for clients to reset the sliding sync connection than to try and send lots of data down on the existing connection. As a heuristic we arbitrarily choose to reset the connection if a) there are more than 100 rooms with updates and b) the connection hasn't been used for at least an hour.
| def delete_old_sliding_sync_connections_txn(txn: LoggingTransaction) -> None: | ||
| sql = """ | ||
| DELETE FROM sliding_sync_connections | ||
| WHERE last_used_ts IS NOT NULL AND last_used_ts < ? |
Contributor
There was a problem hiding this comment.
Feels like we should also have a background job that fills in last_used_ts for all of the existing rows so we can clean them all up at some point.
Could be split out to another PR.
Member
Author
There was a problem hiding this comment.
Yeah, I am a bit undecided whether next release we just delete all connections with a null last_used_ts and take care of the problem that way. So happy to punt to a separate PR.
|
|
||
| -- Add a timestamp for when the sliding sync connection position was last used, | ||
| -- only updated with a small granularity. | ||
| ALTER TABLE sliding_sync_connections ADD COLUMN last_used_ts BIGINT; |
Contributor
There was a problem hiding this comment.
Does last_used_ts need an index to be efficient with the delete query?
Member
Author
There was a problem hiding this comment.
Have added a comment:
-- Note: We don't add an index on this column to allow HOT updates on PostgreSQL
-- to reduce the cost of the updates to the column. c.f.
-- https://www.postgresql.org/docs/current/storage-hot.html
--
-- We do query this column directly to find expired connections, but we expect
-- that to be an infrequent operation and a sequential scan should be fine.This doesn't work on SQLite, causing the tests to fail. I don't think we actually need this now I think about it more. Any DELETE on the rows will already conflict with the SELECT, so we don't need to acquire stronger row locks.
MadLittleMods
approved these changes
Nov 24, 2025
Co-authored-by: Eric Eastwood <erice@element.io>
Member
Author
|
Thanks @MadLittleMods! |
Member
|
Great! Thanks. |
reivilibre
added a commit
that referenced
this pull request
Dec 11, 2025
The team has decided to deprecate and stop publishing python wheels for MacOS. Synapse docker images will continue to work on MacOS, as will building Synapse from source (though note this requires a Rust compiler). Admins using the unstable [MSC2666](matrix-org/matrix-spec-proposals#2666) endpoint (`/_matrix/client/unstable/uk.half-shot.msc2666/user/mutual_rooms`), please check [the relevant section in the upgrade notes](https://github.com/element-hq/synapse/blob/develop/docs/upgrade.md#upgrading-to-v11440) as this release contains changes that disable that endpoint by default. No significant changes since 1.144.0rc1. Admins using the unstable [MSC2666](matrix-org/matrix-spec-proposals#2666) endpoint (`/_matrix/client/unstable/uk.half-shot.msc2666/user/mutual_rooms`), please check [the relevant section in the upgrade notes](https://github.com/element-hq/synapse/blob/develop/docs/upgrade.md#upgrading-to-v11440) as this release contains changes that disable that endpoint by default. - Add experimentatal implememntation of [MSC4380](matrix-org/matrix-spec-proposals#4380) (invite blocking). ([\#19203](#19203)) - Allow restarting delayed event timeouts on workers. ([\#19207](#19207)) - Fix a bug in the database function for fetching state deltas that could result in unnecessarily long query times. ([\#18960](#18960)) - Fix v12 rooms when running with `use_frozen_dicts: True`. ([\#19235](#19235)) - Fix bug where invalid `canonical_alias` content would return 500 instead of 400. ([\#19240](#19240)) - Fix bug where `Duration` was logged incorrectly. ([\#19267](#19267)) - Document in the `--config-path` help how multiple files are merged - by merging them shallowly. ([\#19243](#19243)) - Stop building release wheels for MacOS. ([\#19225](#19225)) - Improve event filtering for Simplified Sliding Sync. ([\#17782](#17782)) - Export `SYNAPSE_SUPPORTED_COMPLEMENT_TEST_PACKAGES` environment variable from `scripts-dev/complement.sh`. ([\#19208](#19208)) - Refactor `scripts-dev/complement.sh` logic to avoid `exit` to facilitate being able to source it from other scripts (composable). ([\#19209](#19209)) - Expire sliding sync connections that are too old or have too much pending data. ([\#19211](#19211)) - Require an experimental feature flag to be enabled in order for the unstable [MSC2666](matrix-org/matrix-spec-proposals#2666) endpoint (`/_matrix/client/unstable/uk.half-shot.msc2666/user/mutual_rooms`) to be available. ([\#19219](#19219)) - Prevent changelog check CI running on @dependabot's PRs even when a human has modified the branch. ([\#19220](#19220)) - Auto-fix trailing spaces in multi-line strings and comments when running the lint script. ([\#19221](#19221)) - Move towards using a dedicated `Duration` type. ([\#19223](#19223), [\#19229](#19229)) - Improve robustness of the SQL schema linting in CI. ([\#19224](#19224)) - Add log to determine whether clients are using `/messages` as expected. ([\#19226](#19226)) - Simplify README and add ESS Getting started section. ([\#19228](#19228), [\#19259](#19259)) - Add a unit test for ensuring associated refresh tokens are erased when a device is deleted. ([\#19230](#19230)) - Prompt user to consider adding future deprecations to the changelog in release script. ([\#19239](#19239)) - Fix check of the Rust compiled code being outdated when using source checkout and `.egg-info`. ([\#19251](#19251)) - Stop building macos wheels in CI pipeline. ([\#19263](#19263)) * Bump Swatinem/rust-cache from 2.8.1 to 2.8.2. ([\#19244](#19244)) * Bump actions/checkout from 5.0.0 to 6.0.0. ([\#19213](#19213)) * Bump actions/setup-go from 6.0.0 to 6.1.0. ([\#19214](#19214)) * Bump actions/setup-python from 6.0.0 to 6.1.0. ([\#19245](#19245)) * Bump attrs from 25.3.0 to 25.4.0. ([\#19215](#19215)) * Bump docker/metadata-action from 5.9.0 to 5.10.0. ([\#19246](#19246)) * Bump http from 1.3.1 to 1.4.0. ([\#19249](#19249)) * Bump pydantic from 2.12.4 to 2.12.5. ([\#19250](#19250)) * Bump pyopenssl from 25.1.0 to 25.3.0. ([\#19248](#19248)) * Bump rpds-py from 0.28.0 to 0.29.0. ([\#19216](#19216)) * Bump rpds-py from 0.29.0 to 0.30.0. ([\#19247](#19247)) * Bump sentry-sdk from 2.44.0 to 2.46.0. ([\#19218](#19218)) * Bump types-bleach from 6.2.0.20250809 to 6.3.0.20251115. ([\#19217](#19217)) * Bump types-jsonschema from 4.25.1.20250822 to 4.25.1.20251009. ([\#19252](#19252))
erikjohnston
added a commit
that referenced
this pull request
Jan 22, 2026
When we change the `required_state` config for a room in sliding sync, we insert a new entry into the `sliding_sync_connection_required_state` table. As the sliding sync connection advances we can accrue a lot of stale entries, so let's clear those out. This is a sort of follow on from #19211 --------- Co-authored-by: Eric Eastwood <erice@element.io>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
We add some logic to expire sliding sync connections if they get old or if there is too much pending data to return.
The values of the constants are picked fairly arbitrarily, these are currently:
Reviewable commit-by-commit
Spawning from #19175