Skip to content

Unify Complement developer docs #19518

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
MadLittleMods merged 8 commits into from
Mar 3, 2026
Merged

Unify Complement developer docs #19518

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
082dcc3
Unify Complement docs
MadLittleMods Mar 3, 2026
d151533
Add changelog
MadLittleMods Mar 3, 2026
56401ec
Less our
MadLittleMods Mar 3, 2026
314fc96
Point to the new place
MadLittleMods Mar 3, 2026
493dc98
Move database access docs
MadLittleMods Mar 3, 2026
23bdfab
Fix typo
MadLittleMods Mar 3, 2026
63672e4
Merge branch 'develop' into madlittlemods/unify-complement-docs
MadLittleMods Mar 3, 2026
a6f9db3
"asyncio-backed reactor for Twisted"
MadLittleMods Mar 3, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions changelog.d/19518.doc
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Unify Complement developer docs.
49 changes: 45 additions & 4 deletions complement/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,7 @@ ensure everything works at a holistic level.
## Setup

Nothing beyond a [normal Complement
setup](https://github.com/matrix-org/complement?tab=readme-ov-file#running) (just Go and
Docker).
setup](https://github.com/matrix-org/complement#running) (just Go and Docker).


## Running tests
Expand All @@ -28,14 +27,39 @@ scripts-dev/complement.sh ./tests/csapi/... -run TestRoomCreate/Parallel/POST_/c
scripts-dev/complement.sh ./tests/... -run 'TestRoomCreate/Parallel/POST_/createRoom_makes_a_(.*)'
```

Typically, if you're developing the Synapse and Complement tests side-by-side, you will
run something like this:
It's often nice to develop on Synapse and write Complement tests at the same time.
Here is how to run your local Synapse checkout against your local Complement checkout.

```shell
# To run a specific test
COMPLEMENT_DIR=../complement ./scripts-dev/complement.sh ./tests/csapi/... -run TestRoomCreate
```

The above will run a monolithic (single-process) Synapse with SQLite as the database.
For other configurations, try:

- Passing `POSTGRES=1` as an environment variable to use the Postgres database instead.
- Passing `WORKERS=1` as an environment variable to use a workerised setup instead. This
option implies the use of Postgres.
- If setting `WORKERS=1`, optionally set `WORKER_TYPES=` to declare which worker types
you wish to test. A simple comma-delimited string containing the worker types
defined from the `WORKERS_CONFIG` template in
[here](https://github.com/element-hq/synapse/blob/develop/docker/configure_workers_and_start.py#L54).
A safe example would be `WORKER_TYPES="federation_inbound, federation_sender,
synchrotron"`. See the [worker documentation](../workers.md) for additional
information on workers.
- Passing `ASYNCIO_REACTOR=1` as an environment variable to use the asyncio-backed
reactor with Twisted instead of the default one.
- Passing `PODMAN=1` will use the [podman](https://podman.io/) container runtime,
instead of docker.
- Passing `UNIX_SOCKETS=1` will utilise Unix socket functionality for Synapse, Redis,
and Postgres(when applicable).
Comment on lines +38 to +56
Copy link
Copy Markdown
Contributor Author

@MadLittleMods MadLittleMods Mar 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved from docs/development/contributing_guide.md with wrapping


To increase the log level for the tests, set `SYNAPSE_TEST_LOG_LEVEL`, e.g:
```sh
SYNAPSE_TEST_LOG_LEVEL=DEBUG COMPLEMENT_DIR=../complement ./scripts-dev/complement.sh -run TestRoomCreate
```
Comment on lines +58 to +61
Copy link
Copy Markdown
Contributor Author

@MadLittleMods MadLittleMods Mar 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved from docs/development/contributing_guide.md (just with a different test to match the other examples here, TestRoomCreate)



### Running in-repo tests

Expand All @@ -52,3 +76,20 @@ To run the in-repo Complement tests, use the `--in-repo` command line argument.
# Similarly, you can also use `-run` to specify all or part of a specific test path to run
scripts-dev/complement.sh --in-repo ./tests/... -run TestIntraShardFederation
```

### Access database for homeserver after Complement test runs.

If you're curious what the database looks like after you run some tests, here are some
steps to get you going in Synapse:

1. In your Complement test comment out `defer deployment.Destroy(t)` and replace with
`defer time.Sleep(2 * time.Hour)` to keep the homeserver running after the tests
complete
1. Start the Complement tests
1. Find the name of the container, `docker ps -f name=complement_` (this will filter for
just the Complement related Docker containers)
1. Access the container replacing the name with what you found in the previous step:
`docker exec -it complement_1_hs_with_application_service.hs1_2 /bin/bash`
1. Install sqlite (database driver), `apt-get update && apt-get install -y sqlite3`
1. Then run `sqlite3` and open the database `.open /conf/homeserver.db` (this db path
comes from the Synapse homeserver.yaml)
Comment on lines +80 to +95
Copy link
Copy Markdown
Contributor Author

@MadLittleMods MadLittleMods Mar 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved from docs/development/contributing_guide.md with wrapping

4 changes: 1 addition & 3 deletions docker/README-testing.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,11 +12,9 @@ Note that running Synapse's unit tests from within the docker image is not suppo

`scripts-dev/complement.sh` is a script that will automatically build
and run Synapse against Complement.
Consult the [contributing guide][guideComplementSh] for instructions on how to use it.
Consult our [Complement docs][https://github.com/element-hq/synapse/tree/develop/complement] for instructions on how to use it.


[guideComplementSh]: https://element-hq.github.io/synapse/latest/development/contributing_guide.html#run-the-integration-tests-complement

## Building and running the images manually

Under some circumstances, you may wish to build the images manually.
Expand Down
53 changes: 2 additions & 51 deletions docs/development/contributing_guide.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -334,46 +334,9 @@ For more details about other configurations, see the [Docker-specific documentat

## Run the integration tests ([Complement](https://github.com/matrix-org/complement)).

[Complement](https://github.com/matrix-org/complement) is a suite of black box tests that can be run on any homeserver implementation. It can also be thought of as end-to-end (e2e) tests.
See our [Complement docs](https://github.com/element-hq/synapse/tree/develop/complement)
for how to use the `./scripts-dev/complement.sh` test runner script.

It's often nice to develop on Synapse and write Complement tests at the same time.
Here is how to run your local Synapse checkout against your local Complement checkout.

(checkout [`complement`](https://github.com/matrix-org/complement) alongside your `synapse` checkout)
```sh
COMPLEMENT_DIR=../complement ./scripts-dev/complement.sh
```

To run a specific test file, you can pass the test name at the end of the command. The name passed comes from the naming structure in your Complement tests. If you're unsure of the name, you can do a full run and copy it from the test output:

```sh
COMPLEMENT_DIR=../complement ./scripts-dev/complement.sh -run TestImportHistoricalMessages
```

To run a specific test, you can specify the whole name structure:

```sh
COMPLEMENT_DIR=../complement ./scripts-dev/complement.sh -run TestImportHistoricalMessages/parallel/Historical_events_resolve_in_the_correct_order
```

The above will run a monolithic (single-process) Synapse with SQLite as the database. For other configurations, try:

- Passing `POSTGRES=1` as an environment variable to use the Postgres database instead.
- Passing `WORKERS=1` as an environment variable to use a workerised setup instead. This option implies the use of Postgres.
- If setting `WORKERS=1`, optionally set `WORKER_TYPES=` to declare which worker
types you wish to test. A simple comma-delimited string containing the worker types
defined from the `WORKERS_CONFIG` template in
[here](https://github.com/element-hq/synapse/blob/develop/docker/configure_workers_and_start.py#L54).
A safe example would be `WORKER_TYPES="federation_inbound, federation_sender, synchrotron"`.
See the [worker documentation](../workers.md) for additional information on workers.
- Passing `ASYNCIO_REACTOR=1` as an environment variable to use the Twisted asyncio reactor instead of the default one.
- Passing `PODMAN=1` will use the [podman](https://podman.io/) container runtime, instead of docker.
- Passing `UNIX_SOCKETS=1` will utilise Unix socket functionality for Synapse, Redis, and Postgres(when applicable).

To increase the log level for the tests, set `SYNAPSE_TEST_LOG_LEVEL`, e.g:
```sh
SYNAPSE_TEST_LOG_LEVEL=DEBUG COMPLEMENT_DIR=../complement ./scripts-dev/complement.sh -run TestImportHistoricalMessages
```

### Prettier formatting with `gotestfmt`

Expand All @@ -389,18 +352,6 @@ COMPLEMENT_DIR=../complement ./scripts-dev/complement.sh -json | gotestfmt -hide
(Remove `-hide successful-tests` if you don't want to hide successful tests.)


### Access database for homeserver after Complement test runs.

If you're curious what the database looks like after you run some tests, here are some steps to get you going in Synapse:

1. In your Complement test comment out `defer deployment.Destroy(t)` and replace with `defer time.Sleep(2 * time.Hour)` to keep the homeserver running after the tests complete
1. Start the Complement tests
1. Find the name of the container, `docker ps -f name=complement_` (this will filter for just the Compelement related Docker containers)
1. Access the container replacing the name with what you found in the previous step: `docker exec -it complement_1_hs_with_application_service.hs1_2 /bin/bash`
1. Install sqlite (database driver), `apt-get update && apt-get install -y sqlite3`
1. Then run `sqlite3` and open the database `.open /conf/homeserver.db` (this db path comes from the Synapse homeserver.yaml)


# 9. Submit your patch.

Once you're happy with your patch, it's time to prepare a Pull Request.
Expand Down
Loading