Compare commits

...

37 Commits

Author SHA1 Message Date
Erik Johnston
25c05d4fb4 Merge remote-tracking branch 'origin/develop' into erikj/rust_http_introspect 2025-06-09 09:54:43 +01:00
Emmanuel Ferdman
6521406a37 Migrate to assertEqual (#18488)
This small PR migrates from `unittest.assertEquals` to
`unittest.assertEqual` which is deprecated from Python2.7:
```python
DeprecationWarning: Please use assertEqual instead.
```

Signed-off-by: Emmanuel Ferdman <emmanuelferdman@gmail.com>
2025-06-06 16:14:09 +01:00
Will Hunt
6e600c986e Don't allow users to ignore themselves. (#18508)
Fixes the self-ignore issues we've being seeing of reports of by
ignoring bad requests from clients.
Fixes https://github.com/element-hq/synapse/issues/11963

Fix https://github.com/element-hq/element-web/issues/29969 although this
should also be fixed on the client to avoid confusing errors popping up
while rejecting invites.

Related to https://github.com/matrix-org/matrix-rust-sdk/issues/5073
2025-06-06 15:37:15 +01:00
Will Hunt
d285d76185 Recover an appservice if a successful ping occurs. (#18521)
Fixes https://github.com/element-hq/synapse/issues/14240

This scratches an itch that i've had for years. We regularly run into
the issue where (especially in development) appservices can go down for
a period and them come back up. The ping endpoint was introduced some
time ago which means Synapse can determine if an AS is up more or less
immediately, so we might as well use that to schedule transaction
redelivery.

I believe transaction scheduling logic is largely implementation
specific, so we should be in the clear to do this without any spec
changes.
2025-06-06 11:59:38 +00:00
Devon Hudson
919c362466 Remove destinations from sending if not whitelisted (#18484)
Co-authored-by: Andrew Morgan <andrew@amorgan.xyz>
2025-06-06 11:19:58 +00:00
Hugh Nimmo-Smith
82189cbde4 Export RatelimitOverride from ModuleApi (#18513) 2025-06-06 10:48:49 +00:00
Eric Eastwood
e80bc4b062 Distinguish all vs local events being persisted in the "Event Send Time Quantiles" graph (#18510)
(Applies to the Grafana graphs)

As discovered by @devonh, we use `synapse_storage_events_persisted_events_total` (which tracks *all* persisted events) for the "Events" rate in the "Event Send Time Quantiles" graph. This is pretty misleading as I would expect it to be the rate of events being sent given the graph title, "Event Send Time Quantiles".

Since the event persistence queues are shared for local and remote events from federation and will block local events being sent, I think it does still make sense to have the event persist rate. I've updated the graph to include the rate of "Local events being persisted" and the rate of "All events being persisted". I think this properly disambiguates and clarifies what the graph is trying to show.
2025-06-05 15:30:28 -05:00
Dirk Klimpel
865d43b4b3 docs: render missing docs for scheduled tasks admin api (#18516)
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
2025-06-05 15:02:40 +01:00
reivilibre
0b9f1757a7 Reduce disk wastage by cleaning up received_transactions older than 1 day, rather than 30 days. (#18310)
Clean up `received_transactions` older than 1 day, rather than 30 days \
Reduces disk waste by homeservers

Closes #6437

---------

Signed-off-by: Olivier 'reivilibre <oliverw@matrix.org>
2025-06-05 11:14:02 +00:00
Will Hunt
8010377a88 Add support for MSC4155 Invite filtering (#18288)
This implements
https://github.com/matrix-org/matrix-spec-proposals/pull/4155, which
adds support for a new account data type that blocks an invite based on
some conditions in the event contents.

---------

Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
2025-06-05 11:49:09 +01:00
Mateusz Reszka
586b82e580 Propose CAP_NET_BIND_SERVICE instead running Synapse with root (#18408)
There are alternative ways to use low numbered ports besides root. Users
might be mislead into thinking they should run Synapse with root
privileges.
2025-06-04 20:44:25 +00:00
Hugh Nimmo-Smith
9b2bc75ed4 Add ratelimit callbacks to module API to allow dynamic ratelimiting (#18458) 2025-06-04 12:09:11 +00:00
Hugh Nimmo-Smith
28f21b4036 Add user_may_send_state_event callback to spam checker module API (#18455) 2025-06-04 11:26:04 +00:00
Hugh Nimmo-Smith
379356c0ea Add media repository callbacks to module API to control media upload size (#18457)
Adds new callbacks for media related functionality:

- `get_media_config_for_user`
- `is_user_allowed_to_upload_media_of_size`
2025-06-04 11:33:10 +01:00
Hugh Nimmo-Smith
fbe7a898f0 Pass room_config argument to user_may_create_room spam checker module callback (#18486)
This PR adds an additional `room_config` argument to the
`user_may_create_room` spam checker module API callback.

It will continue to work with implementations of `user_may_create_room`
that do not expect the additional parameter.

A side affect is that on a room upgrade the spam checker callback is
called *after* doing some work to calculate the state rather than
before. However, I hope that this is acceptable given the relative
infrequency of room upgrades.
2025-06-04 11:30:45 +01:00
Olivier 'reivilibre
08a0506f48 Merge branch 'master' into develop 2025-06-03 15:18:56 +01:00
Olivier 'reivilibre
c47d8e0ee1 1.131.0 2025-06-03 14:37:27 +01:00
Hugh Nimmo-Smith
a4d8da7a1b Make user_type extensible and allow default user_type to be set (#18456) 2025-06-03 11:34:40 +00:00
Quentin Gliech
461571fcf2 Changelog fixes
Co-Authored-By: Andrew Morgan <andrew@amorgan.xyz>
2025-05-28 12:36:28 +02:00
Quentin Gliech
22db145da3 1.131.0rc1 2025-05-28 12:29:07 +02:00
Erik Johnston
7283b6bcd8 Only import twisted reactor after twisted has already been imported 2025-04-24 17:03:01 +01:00
Erik Johnston
661734111d Move twisted reactor import to static as well 2025-04-24 16:38:00 +01:00
Erik Johnston
f2bd5b042e Review comments 2025-04-24 16:31:44 +01:00
Erik Johnston
f501987677 Fix tests 2025-04-24 11:52:14 +01:00
Erik Johnston
48d2217f2d Review comments 2025-04-24 10:19:22 +01:00
Erik Johnston
540b46088a But not that far 2025-04-24 09:56:26 +01:00
Erik Johnston
d5845d5442 Bump clippy to get rid of spurious lint 2025-04-24 09:50:47 +01:00
Erik Johnston
a8a34bb811 Use rustls instead of openssl
This is because openssl cannot be used in a manylinux context due to
lack of stable ABI.
2025-04-24 09:48:54 +01:00
Erik Johnston
59473e3377 Update nightly 2025-04-23 16:20:43 +01:00
Erik Johnston
236d7a7ef0 Use vendored native-tls, and remove unused dep 2025-04-23 16:16:19 +01:00
Erik Johnston
61b7ed02d8 Remove spurious empty file 2025-04-23 15:44:46 +01:00
Erik Johnston
27da4ecde2 Bump CI rust toolchains 2025-04-23 15:38:22 +01:00
Erik Johnston
44a301a32f Newsfile 2025-04-23 15:31:52 +01:00
Erik Johnston
34c65ee95b Also inject opentracing headers 2025-04-23 15:28:46 +01:00
Erik Johnston
95f3b249c4 Logging/tracing 2025-04-23 15:19:03 +01:00
Erik Johnston
7254716b7e Bump minimum rust version 2025-04-23 15:16:35 +01:00
Erik Johnston
a20a0bcbe2 Rust impl 2025-04-23 15:02:16 +01:00
106 changed files with 3930 additions and 470 deletions

View File

@@ -85,7 +85,7 @@ jobs:
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- name: Install Rust
uses: dtolnay/rust-toolchain@e05ebb0e73db581a4877c6ce762e29fe1e0b5073 # 1.66.0
uses: dtolnay/rust-toolchain@c1678930c21fb233e4987c4ae12158f9125e5762 # 1.81.0
- uses: Swatinem/rust-cache@9d47c6ad4b02e050fd481d890b2ea34778fd09d6 # v2.7.8
- uses: matrix-org/setup-python-poetry@5bbf6603c5c930615ec8a29f1b5d7d258d905aa4 # v2.0.0
with:
@@ -149,7 +149,7 @@ jobs:
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- name: Install Rust
uses: dtolnay/rust-toolchain@e05ebb0e73db581a4877c6ce762e29fe1e0b5073 # 1.66.0
uses: dtolnay/rust-toolchain@c1678930c21fb233e4987c4ae12158f9125e5762 # 1.81.0
- uses: Swatinem/rust-cache@9d47c6ad4b02e050fd481d890b2ea34778fd09d6 # v2.7.8
- name: Setup Poetry
@@ -210,7 +210,7 @@ jobs:
with:
ref: ${{ github.event.pull_request.head.sha }}
- name: Install Rust
uses: dtolnay/rust-toolchain@e05ebb0e73db581a4877c6ce762e29fe1e0b5073 # 1.66.0
uses: dtolnay/rust-toolchain@c1678930c21fb233e4987c4ae12158f9125e5762 # 1.81.0
- uses: Swatinem/rust-cache@9d47c6ad4b02e050fd481d890b2ea34778fd09d6 # v2.7.8
- uses: matrix-org/setup-python-poetry@5bbf6603c5c930615ec8a29f1b5d7d258d905aa4 # v2.0.0
with:
@@ -227,7 +227,7 @@ jobs:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- name: Install Rust
uses: dtolnay/rust-toolchain@e05ebb0e73db581a4877c6ce762e29fe1e0b5073 # 1.66.0
uses: dtolnay/rust-toolchain@0d72692bcfbf448b1e2afa01a67f71b455a9dcec # 1.86.0
with:
components: clippy
- uses: Swatinem/rust-cache@9d47c6ad4b02e050fd481d890b2ea34778fd09d6 # v2.7.8
@@ -247,7 +247,7 @@ jobs:
- name: Install Rust
uses: dtolnay/rust-toolchain@56f84321dbccf38fb67ce29ab63e4754056677e0 # master (rust 1.85.1)
with:
toolchain: nightly-2022-12-01
toolchain: nightly-2025-04-23
components: clippy
- uses: Swatinem/rust-cache@9d47c6ad4b02e050fd481d890b2ea34778fd09d6 # v2.7.8
@@ -265,7 +265,7 @@ jobs:
uses: dtolnay/rust-toolchain@56f84321dbccf38fb67ce29ab63e4754056677e0 # master (rust 1.85.1)
with:
# We use nightly so that it correctly groups together imports
toolchain: nightly-2022-12-01
toolchain: nightly-2025-04-23
components: rustfmt
- uses: Swatinem/rust-cache@9d47c6ad4b02e050fd481d890b2ea34778fd09d6 # v2.7.8
@@ -362,7 +362,7 @@ jobs:
postgres:${{ matrix.job.postgres-version }}
- name: Install Rust
uses: dtolnay/rust-toolchain@e05ebb0e73db581a4877c6ce762e29fe1e0b5073 # 1.66.0
uses: dtolnay/rust-toolchain@c1678930c21fb233e4987c4ae12158f9125e5762 # 1.81.0
- uses: Swatinem/rust-cache@9d47c6ad4b02e050fd481d890b2ea34778fd09d6 # v2.7.8
- uses: matrix-org/setup-python-poetry@5bbf6603c5c930615ec8a29f1b5d7d258d905aa4 # v2.0.0
@@ -404,7 +404,7 @@ jobs:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- name: Install Rust
uses: dtolnay/rust-toolchain@e05ebb0e73db581a4877c6ce762e29fe1e0b5073 # 1.66.0
uses: dtolnay/rust-toolchain@c1678930c21fb233e4987c4ae12158f9125e5762 # 1.81.0
- uses: Swatinem/rust-cache@9d47c6ad4b02e050fd481d890b2ea34778fd09d6 # v2.7.8
# There aren't wheels for some of the older deps, so we need to install
@@ -519,7 +519,7 @@ jobs:
run: cat sytest-blacklist .ci/worker-blacklist > synapse-blacklist-with-workers
- name: Install Rust
uses: dtolnay/rust-toolchain@e05ebb0e73db581a4877c6ce762e29fe1e0b5073 # 1.66.0
uses: dtolnay/rust-toolchain@c1678930c21fb233e4987c4ae12158f9125e5762 # 1.81.0
- uses: Swatinem/rust-cache@9d47c6ad4b02e050fd481d890b2ea34778fd09d6 # v2.7.8
- name: Run SyTest
@@ -663,7 +663,7 @@ jobs:
path: synapse
- name: Install Rust
uses: dtolnay/rust-toolchain@e05ebb0e73db581a4877c6ce762e29fe1e0b5073 # 1.66.0
uses: dtolnay/rust-toolchain@c1678930c21fb233e4987c4ae12158f9125e5762 # 1.81.0
- uses: Swatinem/rust-cache@9d47c6ad4b02e050fd481d890b2ea34778fd09d6 # v2.7.8
- name: Prepare Complement's Prerequisites
@@ -695,7 +695,7 @@ jobs:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- name: Install Rust
uses: dtolnay/rust-toolchain@e05ebb0e73db581a4877c6ce762e29fe1e0b5073 # 1.66.0
uses: dtolnay/rust-toolchain@c1678930c21fb233e4987c4ae12158f9125e5762 # 1.81.0
- uses: Swatinem/rust-cache@9d47c6ad4b02e050fd481d890b2ea34778fd09d6 # v2.7.8
- run: cargo test

View File

@@ -1,3 +1,57 @@
# Synapse 1.131.0 (2025-06-03)
No significant changes since 1.131.0rc1.
# Synapse 1.131.0rc1 (2025-05-28)
### Features
- Add `msc4263_limit_key_queries_to_users_who_share_rooms` config option as per [MSC4263](https://github.com/matrix-org/matrix-spec-proposals/pull/4263). ([\#18180](https://github.com/element-hq/synapse/issues/18180))
- Add option to allow registrations that begin with `_`. Contributed by `_` (@hex5f). ([\#18262](https://github.com/element-hq/synapse/issues/18262))
- Include room ID in response to the [Room Deletion Status Admin API](https://element-hq.github.io/synapse/latest/admin_api/rooms.html#status-of-deleting-rooms). ([\#18318](https://github.com/element-hq/synapse/issues/18318))
- Add support for calling Policy Servers ([MSC4284](https://github.com/matrix-org/matrix-spec-proposals/pull/4284)) to mark events as spam. ([\#18387](https://github.com/element-hq/synapse/issues/18387))
### Bugfixes
- Prevent race-condition in `_maybe_retry_device_resync` entrance. ([\#18391](https://github.com/element-hq/synapse/issues/18391))
- Fix the `tests.handlers.test_worker_lock.WorkerLockTestCase.test_lock_contention` test which could spuriously time out on RISC-V architectures due to performance differences. ([\#18430](https://github.com/element-hq/synapse/issues/18430))
- Fix admin redaction endpoint not redacting encrypted messages. ([\#18434](https://github.com/element-hq/synapse/issues/18434))
### Improved Documentation
- Update `room_list_publication_rules` docs to consider defaults that changed in v1.126.0. Contributed by @HarHarLinks. ([\#18286](https://github.com/element-hq/synapse/issues/18286))
- Add advice for upgrading between major PostgreSQL versions to the database documentation. ([\#18445](https://github.com/element-hq/synapse/issues/18445))
### Internal Changes
- Fix a memory leak in `_NotifierUserStream`. ([\#18380](https://github.com/element-hq/synapse/issues/18380))
- Fix a couple type annotations in the `RootConfig`/`Config`. ([\#18409](https://github.com/element-hq/synapse/issues/18409))
- Explicitly enable PyPy builds in `cibuildwheel`s config to avoid it being disabled on a future upgrade to `cibuildwheel` v3. ([\#18417](https://github.com/element-hq/synapse/issues/18417))
- Update the PR review template to remove an erroneous line break from the final bullet point. ([\#18419](https://github.com/element-hq/synapse/issues/18419))
- Explain why we `flush_buffer()` for Python `print(...)` output. ([\#18420](https://github.com/element-hq/synapse/issues/18420))
- Add lint to ensure we don't add a `CREATE/DROP INDEX` in a schema delta. ([\#18440](https://github.com/element-hq/synapse/issues/18440))
- Allow checking only for the existence of a field in an SSO provider's response, rather than requiring the value(s) to check. ([\#18454](https://github.com/element-hq/synapse/issues/18454))
- Add unit tests for homeserver usage statistics. ([\#18463](https://github.com/element-hq/synapse/issues/18463))
- Don't move invited users to new room when shutting down room. ([\#18471](https://github.com/element-hq/synapse/issues/18471))
### Updates to locked dependencies
* Bump actions/setup-python from 5.5.0 to 5.6.0. ([\#18398](https://github.com/element-hq/synapse/issues/18398))
* Bump authlib from 1.5.1 to 1.5.2. ([\#18452](https://github.com/element-hq/synapse/issues/18452))
* Bump docker/build-push-action from 6.15.0 to 6.17.0. ([\#18397](https://github.com/element-hq/synapse/issues/18397), [\#18449](https://github.com/element-hq/synapse/issues/18449))
* Bump lxml from 5.3.0 to 5.4.0. ([\#18480](https://github.com/element-hq/synapse/issues/18480))
* Bump mypy-zope from 1.0.9 to 1.0.11. ([\#18428](https://github.com/element-hq/synapse/issues/18428))
* Bump pyo3 from 0.23.5 to 0.24.2. ([\#18460](https://github.com/element-hq/synapse/issues/18460))
* Bump pyo3-log from 0.12.3 to 0.12.4. ([\#18453](https://github.com/element-hq/synapse/issues/18453))
* Bump pyopenssl from 25.0.0 to 25.1.0. ([\#18450](https://github.com/element-hq/synapse/issues/18450))
* Bump ruff from 0.7.3 to 0.11.11. ([\#18451](https://github.com/element-hq/synapse/issues/18451), [\#18482](https://github.com/element-hq/synapse/issues/18482))
* Bump tornado from 6.4.2 to 6.5.0. ([\#18459](https://github.com/element-hq/synapse/issues/18459))
* Bump setuptools from 72.1.0 to 78.1.1. ([\#18461](https://github.com/element-hq/synapse/issues/18461))
* Bump types-jsonschema from 4.23.0.20241208 to 4.23.0.20250516. ([\#18481](https://github.com/element-hq/synapse/issues/18481))
* Bump types-requests from 2.32.0.20241016 to 2.32.0.20250328. ([\#18427](https://github.com/element-hq/synapse/issues/18427))
# Synapse 1.130.0 (2025-05-20)
### Bugfixes

1333
Cargo.lock generated

File diff suppressed because it is too large Load Diff

View File

@@ -1 +0,0 @@
Add `msc4263_limit_key_queries_to_users_who_share_rooms` config option as per [MSC4263](https://github.com/matrix-org/matrix-spec-proposals/pull/4263).

View File

@@ -1 +0,0 @@
Add option to allow registrations that begin with `_`. Contributed by `_` (@hex5f).

View File

@@ -1 +0,0 @@
Update `room_list_publication_rules` docs to consider defaults that changed in v1.126.0. Contributed by @HarHarLinks.

View File

@@ -0,0 +1 @@
Add support for [MSC4155](https://github.com/matrix-org/matrix-spec-proposals/pull/4155) Invite Filtering.

1
changelog.d/18310.misc Normal file
View File

@@ -0,0 +1 @@
Reduce disk wastage by cleaning up `received_transactions` older than 1 day, rather than 30 days.

View File

@@ -1 +0,0 @@
Include room ID in room deletion status response.

1
changelog.d/18357.misc Normal file
View File

@@ -0,0 +1 @@
Increase performance of introspecting access tokens when using delegated auth.

View File

@@ -1 +0,0 @@
Fix a memory leak in `_NotifierUserStream`.

View File

@@ -1 +0,0 @@
Add support for calling Policy Servers ([MSC4284](https://github.com/matrix-org/matrix-spec-proposals/pull/4284)) to mark events as spam.

View File

@@ -1 +0,0 @@
Prevent race-condition in `_maybe_retry_device_resync` entrance.

1
changelog.d/18408.doc Normal file
View File

@@ -0,0 +1 @@
Mention `CAP_NET_BIND_SERVICE` as an alternative to running Synapse as root in order to bind to a privileged port.

View File

@@ -1 +0,0 @@
Fix a couple type annotations in the `RootConfig`/`Config`.

View File

@@ -1 +0,0 @@
Explicitly enable PyPy builds in `cibuildwheel`s config to avoid it being disabled on a future upgrade to `cibuildwheel` v3.

View File

@@ -1 +0,0 @@
Update the PR review template to remove an erroneous line break from the final bullet point.

View File

@@ -1 +0,0 @@
Explain why we `flush_buffer()` for Python `print(...)` output.

View File

@@ -1 +0,0 @@
Fix the `tests.handlers.test_worker_lock.WorkerLockTestCase.test_lock_contention` test which could spuriously time out on RISC-V architectures due to performance differences.

View File

@@ -1 +0,0 @@
Fix admin redaction endpoint not redacting encrypted messages.

View File

@@ -1 +0,0 @@
Add lint to ensure we don't add a `CREATE/DROP INDEX` in a schema delta.

View File

@@ -1 +0,0 @@
Add advice for upgrading between major PostgreSQL versions to the database documentation.

View File

@@ -1 +0,0 @@
Bump ruff from 0.7.3 to 0.11.10.

View File

@@ -1 +0,0 @@
Allow checking only for the existence of a field in an SSO provider's response, rather than requiring the value(s) to check.

View File

@@ -0,0 +1 @@
Add user_may_send_state_event callback to spam checker module API.

View File

@@ -0,0 +1 @@
Support configuration of default and extra user types.

View File

@@ -0,0 +1 @@
Add new module API callbacks that allows overriding of media repository maximum upload size.

View File

@@ -0,0 +1 @@
Add a new module API callback that allows overriding of per user ratelimits.

View File

@@ -1 +0,0 @@
Bump tornado from 6.4.2 to 6.5.0.

View File

@@ -1 +0,0 @@
Bump pyo3 from 0.23.5 to 0.24.2.

View File

@@ -1 +0,0 @@
Add unit tests for homeserver usage statistics.

View File

@@ -1 +0,0 @@
Don't move invited users to new room when shutting down room.

1
changelog.d/18484.bugfix Normal file
View File

@@ -0,0 +1 @@
Remove destinations from sending if not whitelisted.

View File

@@ -0,0 +1 @@
Pass room_config argument to user_may_create_room spam checker module callback.

1
changelog.d/18508.bugfix Normal file
View File

@@ -0,0 +1 @@
Prevent users from adding themselves to their own ignore list.

1
changelog.d/18510.misc Normal file
View File

@@ -0,0 +1 @@
Distinguish all vs local events being persisted in the "Event Send Time Quantiles" graph (Grafana).

1
changelog.d/18513.bugfix Normal file
View File

@@ -0,0 +1 @@
Support the import of the `RatelimitOverride` type from `synapse.module_api` in modules and rename `messages_per_second` to `per_second`.

1
changelog.d/18516.doc Normal file
View File

@@ -0,0 +1 @@
Surface hidden Admin API documentation regarding fetching of scheduled tasks.

View File

@@ -0,0 +1 @@
Successful requests to `/_matrix/app/v1/ping` will now force Synapse to reattempt delivering transactions to appservices.

View File

@@ -220,29 +220,24 @@
"yBucketBound": "auto"
},
{
"aliasColors": {},
"bars": false,
"dashLength": 10,
"dashes": false,
"datasource": {
"uid": "${DS_PROMETHEUS}"
"uid": "${DS_PROMETHEUS}",
"type": "prometheus"
},
"description": "",
"aliasColors": {},
"dashLength": 10,
"fieldConfig": {
"defaults": {
"links": []
},
"overrides": []
},
"fill": 0,
"fillGradient": 0,
"gridPos": {
"h": 9,
"w": 12,
"x": 12,
"y": 1
},
"hiddenSeries": false,
"id": 152,
"legend": {
"avg": false,
@@ -255,71 +250,81 @@
"values": false
},
"lines": true,
"linewidth": 0,
"links": [],
"nullPointMode": "connected",
"options": {
"alertThreshold": true
},
"paceLength": 10,
"percentage": false,
"pluginVersion": "9.2.2",
"pluginVersion": "10.4.3",
"pointradius": 5,
"points": false,
"renderer": "flot",
"seriesOverrides": [
{
"alias": "Avg",
"fill": 0,
"linewidth": 3
"linewidth": 3,
"$$hashKey": "object:48"
},
{
"alias": "99%",
"color": "#C4162A",
"fillBelowTo": "90%"
"fillBelowTo": "90%",
"$$hashKey": "object:49"
},
{
"alias": "90%",
"color": "#FF7383",
"fillBelowTo": "75%"
"fillBelowTo": "75%",
"$$hashKey": "object:50"
},
{
"alias": "75%",
"color": "#FFEE52",
"fillBelowTo": "50%"
"fillBelowTo": "50%",
"$$hashKey": "object:51"
},
{
"alias": "50%",
"color": "#73BF69",
"fillBelowTo": "25%"
"fillBelowTo": "25%",
"$$hashKey": "object:52"
},
{
"alias": "25%",
"color": "#1F60C4",
"fillBelowTo": "5%"
"fillBelowTo": "5%",
"$$hashKey": "object:53"
},
{
"alias": "5%",
"lines": false
"lines": false,
"$$hashKey": "object:54"
},
{
"alias": "Average",
"color": "rgb(255, 255, 255)",
"lines": true,
"linewidth": 3
"linewidth": 3,
"$$hashKey": "object:55"
},
{
"alias": "Events",
"alias": "Local events being persisted",
"color": "#96d98D",
"points": true,
"yaxis": 2,
"zindex": -3,
"$$hashKey": "object:56"
},
{
"$$hashKey": "object:329",
"color": "#B877D9",
"hideTooltip": true,
"alias": "All events being persisted",
"points": true,
"yaxis": 2,
"zindex": -3
}
],
"spaceLength": 10,
"stack": false,
"steppedLine": false,
"targets": [
{
"datasource": {
@@ -384,7 +389,20 @@
},
"expr": "sum(rate(synapse_http_server_response_time_seconds_sum{servlet='RoomSendEventRestServlet',index=~\"$index\",instance=\"$instance\",code=~\"2..\"}[$bucket_size])) / sum(rate(synapse_http_server_response_time_seconds_count{servlet='RoomSendEventRestServlet',index=~\"$index\",instance=\"$instance\",code=~\"2..\"}[$bucket_size]))",
"legendFormat": "Average",
"refId": "H"
"refId": "H",
"editorMode": "code",
"range": true
},
{
"datasource": {
"uid": "${DS_PROMETHEUS}"
},
"expr": "sum(rate(synapse_http_server_response_time_seconds_count{servlet='RoomSendEventRestServlet',index=~\"$index\",instance=\"$instance\",code=~\"2..\"}[$bucket_size]))",
"hide": false,
"instant": false,
"legendFormat": "Local events being persisted",
"refId": "E",
"editorMode": "code"
},
{
"datasource": {
@@ -393,8 +411,9 @@
"expr": "sum(rate(synapse_storage_events_persisted_events_total{instance=\"$instance\"}[$bucket_size]))",
"hide": false,
"instant": false,
"legendFormat": "Events",
"refId": "E"
"legendFormat": "All events being persisted",
"refId": "I",
"editorMode": "code"
}
],
"thresholds": [
@@ -428,7 +447,9 @@
"xaxis": {
"mode": "time",
"show": true,
"values": []
"values": [],
"name": null,
"buckets": null
},
"yaxes": [
{
@@ -450,7 +471,20 @@
],
"yaxis": {
"align": false
}
},
"bars": false,
"dashes": false,
"description": "",
"fill": 0,
"fillGradient": 0,
"hiddenSeries": false,
"linewidth": 0,
"percentage": false,
"points": false,
"stack": false,
"steppedLine": false,
"timeFrom": null,
"timeShift": null
},
{
"aliasColors": {},

12
debian/changelog vendored
View File

@@ -1,3 +1,15 @@
matrix-synapse-py3 (1.131.0) stable; urgency=medium
* New Synapse release 1.131.0.
-- Synapse Packaging team <packages@matrix.org> Tue, 03 Jun 2025 14:36:55 +0100
matrix-synapse-py3 (1.131.0~rc1) stable; urgency=medium
* New synapse release 1.131.0rc1.
-- Synapse Packaging team <packages@matrix.org> Wed, 28 May 2025 10:25:44 +0000
matrix-synapse-py3 (1.130.0) stable; urgency=medium
* New Synapse release 1.130.0.

View File

@@ -127,6 +127,8 @@ experimental_features:
msc3983_appservice_otk_claims: true
# Proxy key queries to exclusive ASes
msc3984_appservice_key_query: true
# Invite filtering
msc4155_enabled: true
server_notices:
system_mxid_localpart: _server

View File

@@ -49,6 +49,8 @@
- [Background update controller callbacks](modules/background_update_controller_callbacks.md)
- [Account data callbacks](modules/account_data_callbacks.md)
- [Add extra fields to client events unsigned section callbacks](modules/add_extra_fields_to_client_events_unsigned.md)
- [Media repository callbacks](modules/media_repository_callbacks.md)
- [Ratelimit callbacks](modules/ratelimit_callbacks.md)
- [Porting a legacy module to the new interface](modules/porting_legacy_module.md)
- [Workers](workers.md)
- [Using `synctl` with Workers](synctl_workers.md)
@@ -66,6 +68,7 @@
- [Registration Tokens](usage/administration/admin_api/registration_tokens.md)
- [Manipulate Room Membership](admin_api/room_membership.md)
- [Rooms](admin_api/rooms.md)
- [Scheduled tasks](admin_api/scheduled_tasks.md)
- [Server Notices](admin_api/server_notices.md)
- [Statistics](admin_api/statistics.md)
- [Users](admin_api/user_admin_api.md)

View File

@@ -163,7 +163,8 @@ Body parameters:
- `locked` - **bool**, optional. If unspecified, locked state will be left unchanged.
- `user_type` - **string** or null, optional. If not provided, the user type will be
not be changed. If `null` is given, the user type will be cleared.
Other allowed options are: `bot` and `support`.
Other allowed options are: `bot` and `support` and any extra values defined in the homserver
[configuration](../usage/configuration/config_documentation.md#user_types).
## List Accounts
### List Accounts (V2)

View File

@@ -0,0 +1,56 @@
# Media repository callbacks
Media repository callbacks allow module developers to customise the behaviour of the
media repository on a per user basis. Media repository callbacks can be registered
using the module API's `register_media_repository_callbacks` method.
The available media repository callbacks are:
### `get_media_config_for_user`
_First introduced in Synapse v1.132.0_
```python
async def get_media_config_for_user(user_id: str) -> Optional[JsonDict]
```
Called when processing a request from a client for the
[media config endpoint](https://spec.matrix.org/latest/client-server-api/#get_matrixclientv1mediaconfig).
The arguments passed to this callback are:
* `user_id`: The Matrix user ID of the user (e.g. `@alice:example.com`) making the request.
If the callback returns a dictionary then it will be used as the body of the response to the
client.
If multiple modules implement this callback, they will be considered in order. If a
callback returns `None`, Synapse falls through to the next one. The value of the first
callback that does not return `None` will be used. If this happens, Synapse will not call
any of the subsequent implementations of this callback.
If no module returns a non-`None` value then the default media config will be returned.
### `is_user_allowed_to_upload_media_of_size`
_First introduced in Synapse v1.132.0_
```python
async def is_user_allowed_to_upload_media_of_size(user_id: str, size: int) -> bool
```
Called before media is accepted for upload from a user, in case the module needs to
enforce a different limit for the particular user.
The arguments passed to this callback are:
* `user_id`: The Matrix user ID of the user (e.g. `@alice:example.com`) making the request.
* `size`: The size in bytes of media that is being requested to upload.
If the module returns `False`, the current request will be denied with the error code
`M_TOO_LARGE` and the HTTP status code 413.
If multiple modules implement this callback, they will be considered in order. If a callback
returns `True`, Synapse falls through to the next one. The value of the first callback that
returns `False` will be used. If this happens, Synapse will not call any of the subsequent
implementations of this callback.

View File

@@ -0,0 +1,38 @@
# Ratelimit callbacks
Ratelimit callbacks allow module developers to override ratelimit settings dynamically whilst
Synapse is running. Ratelimit callbacks can be registered using the module API's
`register_ratelimit_callbacks` method.
The available ratelimit callbacks are:
### `get_ratelimit_override_for_user`
_First introduced in Synapse v1.132.0_
```python
async def get_ratelimit_override_for_user(user: str, limiter_name: str) -> Optional[synapse.module_api.RatelimitOverride]
```
Called when constructing a ratelimiter of a particular type for a user. The module can
return a `messages_per_second` and `burst_count` to be used, or `None` if
the default settings are adequate. The user is represented by their Matrix user ID
(e.g. `@alice:example.com`). The limiter name is usually taken from the `RatelimitSettings` key
value.
The limiters that are currently supported are:
- `rc_invites.per_room`
- `rc_invites.per_user`
- `rc_invites.per_issuer`
The `RatelimitOverride` return type has the following fields:
- `per_second: float`. The number of actions that can be performed in a second. `0.0` means that ratelimiting is disabled.
- `burst_count: int`. The number of actions that can be performed before being limited.
If multiple modules implement this callback, they will be considered in order. If a
callback returns `None`, Synapse falls through to the next one. The value of the first
callback that does not return `None` will be used. If this happens, Synapse will not call
any of the subsequent implementations of this callback. If no module returns a non-`None` value
then the default settings will be used.

View File

@@ -159,12 +159,19 @@ _First introduced in Synapse v1.37.0_
_Changed in Synapse v1.62.0: `synapse.module_api.NOT_SPAM` and `synapse.module_api.errors.Codes` can be returned by this callback. Returning a boolean is now deprecated._
_Changed in Synapse v1.132.0: Added the `room_config` argument. Callbacks that only expect a single `user_id` argument are still supported._
```python
async def user_may_create_room(user_id: str) -> Union["synapse.module_api.NOT_SPAM", "synapse.module_api.errors.Codes", bool]
async def user_may_create_room(user_id: str, room_config: synapse.module_api.JsonDict) -> Union["synapse.module_api.NOT_SPAM", "synapse.module_api.errors.Codes", bool]
```
Called when processing a room creation request.
The arguments passed to this callback are:
* `user_id`: The Matrix user ID of the user (e.g. `@alice:example.com`).
* `room_config`: The contents of the body of a [/createRoom request](https://spec.matrix.org/latest/client-server-api/#post_matrixclientv3createroom) as a dictionary.
The callback must return one of:
- `synapse.module_api.NOT_SPAM`, to allow the operation. Other callbacks may still
decide to reject it.
@@ -239,6 +246,36 @@ be used. If this happens, Synapse will not call any of the subsequent implementa
this callback.
### `user_may_send_state_event`
_First introduced in Synapse v1.132.0_
```python
async def user_may_send_state_event(user_id: str, room_id: str, event_type: str, state_key: str, content: JsonDict) -> Union["synapse.module_api.NOT_SPAM", "synapse.module_api.errors.Codes"]
```
Called when processing a request to [send state events](https://spec.matrix.org/latest/client-server-api/#put_matrixclientv3roomsroomidstateeventtypestatekey) to a room.
The arguments passed to this callback are:
* `user_id`: The Matrix user ID of the user (e.g. `@alice:example.com`) sending the state event.
* `room_id`: The ID of the room that the requested state event is being sent to.
* `event_type`: The requested type of event.
* `state_key`: The requested state key.
* `content`: The requested event contents.
The callback must return one of:
- `synapse.module_api.NOT_SPAM`, to allow the operation. Other callbacks may still
decide to reject it.
- `synapse.module_api.errors.Codes` to reject the operation with an error code. In case
of doubt, `synapse.module_api.errors.Codes.FORBIDDEN` is a good error code.
If multiple modules implement this callback, they will be considered in order. If a
callback returns `synapse.module_api.NOT_SPAM`, Synapse falls through to the next one.
The value of the first callback that does not return `synapse.module_api.NOT_SPAM` will
be used. If this happens, Synapse will not call any of the subsequent implementations of
this callback.
### `check_username_for_spam`

View File

@@ -5,10 +5,10 @@ It is recommended to put a reverse proxy such as
[Apache](https://httpd.apache.org/docs/current/mod/mod_proxy_http.html),
[Caddy](https://caddyserver.com/docs/quick-starts/reverse-proxy),
[HAProxy](https://www.haproxy.org/) or
[relayd](https://man.openbsd.org/relayd.8) in front of Synapse. One advantage
of doing so is that it means that you can expose the default https port
(443) to Matrix clients without needing to run Synapse with root
privileges.
[relayd](https://man.openbsd.org/relayd.8) in front of Synapse.
This has the advantage of being able to expose the default HTTPS port (443) to Matrix
clients without requiring Synapse to bind to a privileged port (port numbers less than
1024), avoiding the need for `CAP_NET_BIND_SERVICE` or running as root.
You should configure your reverse proxy to forward requests to `/_matrix` or
`/_synapse/client` to Synapse, and have it set the `X-Forwarded-For` and

View File

@@ -63,7 +63,7 @@ class ExampleSpamChecker:
async def user_may_invite(self, inviter_userid, invitee_userid, room_id):
return True # allow all invites
async def user_may_create_room(self, userid):
async def user_may_create_room(self, userid, room_config):
return True # allow all room creations
async def user_may_create_room_alias(self, userid, room_alias):

View File

@@ -762,6 +762,24 @@ Example configuration:
max_event_delay_duration: 24h
```
---
### `user_types`
Configuration settings related to the user types feature.
This setting has the following sub-options:
* `default_user_type`: The default user type to use for registering new users when no value has been specified.
Defaults to none.
* `extra_user_types`: Array of additional user types to allow. These are treated as real users. Defaults to [].
Example configuration:
```yaml
user_types:
default_user_type: "custom"
extra_user_types:
- "custom"
- "custom2"
```
## Homeserver blocking
Useful options for Synapse admins.

View File

@@ -97,7 +97,7 @@ module-name = "synapse.synapse_rust"
[tool.poetry]
name = "matrix-synapse"
version = "1.130.0"
version = "1.131.0"
description = "Homeserver for the Matrix decentralised comms protocol"
authors = ["Matrix.org Team and Contributors <packages@matrix.org>"]
license = "AGPL-3.0-or-later"

View File

@@ -7,7 +7,7 @@ name = "synapse"
version = "0.1.0"
edition = "2021"
rust-version = "1.66.0"
rust-version = "1.81.0"
[lib]
name = "synapse"
@@ -36,13 +36,21 @@ pyo3 = { version = "0.24.2", features = [
"abi3",
"abi3-py39",
] }
pyo3-log = "0.12.0"
pyo3-log = "0.12.3"
pythonize = "0.24.0"
regex = "1.6.0"
sha2 = "0.10.8"
serde = { version = "1.0.144", features = ["derive"] }
serde_json = "1.0.85"
ulid = "1.1.2"
reqwest = { version = "0.12.15", default-features = false, features = [
"http2",
"stream",
"rustls-tls-native-roots",
] }
http-body-util = "0.1.3"
futures = "0.3.31"
tokio = { version = "1.44.2", features = ["rt", "rt-multi-thread"] }
[features]
extension-module = ["pyo3/extension-module"]

View File

@@ -58,3 +58,15 @@ impl NotFoundError {
NotFoundError::new_err(())
}
}
import_exception!(synapse.api.errors, HttpResponseException);
impl HttpResponseException {
pub fn new(status: StatusCode, bytes: Vec<u8>) -> pyo3::PyErr {
HttpResponseException::new_err((
status.as_u16(),
status.canonical_reason().unwrap_or_default(),
bytes,
))
}
}

218
rust/src/http_client.rs Normal file
View File

@@ -0,0 +1,218 @@
/*
* This file is licensed under the Affero General Public License (AGPL) version 3.
*
* Copyright (C) 2025 New Vector, Ltd
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU Affero General Public License as
* published by the Free Software Foundation, either version 3 of the
* License, or (at your option) any later version.
*
* See the GNU Affero General Public License for more details:
* <https://www.gnu.org/licenses/agpl-3.0.html>.
*/
use std::{collections::HashMap, future::Future, panic::AssertUnwindSafe, sync::LazyLock};
use anyhow::Context;
use futures::{FutureExt, TryStreamExt};
use pyo3::{exceptions::PyException, prelude::*, types::PyString};
use reqwest::RequestBuilder;
use tokio::runtime::Runtime;
use crate::errors::HttpResponseException;
/// The tokio runtime that we're using to run async Rust libs.
static RUNTIME: LazyLock<Runtime> = LazyLock::new(|| {
tokio::runtime::Builder::new_multi_thread()
.worker_threads(4)
.enable_all()
.build()
.unwrap()
});
/// A reference to the `Deferred` python class.
static DEFERRED_CLASS: LazyLock<PyObject> = LazyLock::new(|| {
Python::with_gil(|py| {
py.import("twisted.internet.defer")
.expect("module 'twisted.internet.defer' should be importable")
.getattr("Deferred")
.expect("module 'twisted.internet.defer' should have a 'Deferred' class")
.unbind()
})
});
/// A reference to the twisted `reactor`.
static TWISTED_REACTOR: LazyLock<Py<PyModule>> = LazyLock::new(|| {
Python::with_gil(|py| {
py.import("twisted.internet.reactor")
.expect("module 'twisted.internet.reactor' should be importable")
.unbind()
})
});
/// Called when registering modules with python.
pub fn register_module(py: Python<'_>, m: &Bound<'_, PyModule>) -> PyResult<()> {
let child_module: Bound<'_, PyModule> = PyModule::new(py, "http_client")?;
child_module.add_class::<HttpClient>()?;
// Make sure we fail early if we can't build the lazy statics.
LazyLock::force(&RUNTIME);
LazyLock::force(&DEFERRED_CLASS);
m.add_submodule(&child_module)?;
// We need to manually add the module to sys.modules to make `from
// synapse.synapse_rust import acl` work.
py.import("sys")?
.getattr("modules")?
.set_item("synapse.synapse_rust.http_client", child_module)?;
Ok(())
}
#[pyclass]
#[derive(Clone)]
struct HttpClient {
client: reqwest::Client,
}
#[pymethods]
impl HttpClient {
#[new]
pub fn py_new(user_agent: &str) -> PyResult<HttpClient> {
// The twisted reactor can only be imported after Synapse has been
// imported, to allow Synapse to change the twisted reactor. If we try
// and import the reactor too early twisted installs a default reactor,
// which can't be replaced.
LazyLock::force(&TWISTED_REACTOR);
Ok(HttpClient {
client: reqwest::Client::builder()
.user_agent(user_agent)
.build()
.context("building reqwest client")?,
})
}
pub fn get<'a>(
&self,
py: Python<'a>,
url: String,
response_limit: usize,
) -> PyResult<Bound<'a, PyAny>> {
self.send_request(py, self.client.get(url), response_limit)
}
pub fn post<'a>(
&self,
py: Python<'a>,
url: String,
response_limit: usize,
headers: HashMap<String, String>,
request_body: String,
) -> PyResult<Bound<'a, PyAny>> {
let mut builder = self.client.post(url);
for (name, value) in headers {
builder = builder.header(name, value);
}
builder = builder.body(request_body);
self.send_request(py, builder, response_limit)
}
}
impl HttpClient {
fn send_request<'a>(
&self,
py: Python<'a>,
builder: RequestBuilder,
response_limit: usize,
) -> PyResult<Bound<'a, PyAny>> {
create_deferred(py, async move {
let response = builder.send().await.context("sending request")?;
let status = response.status();
let mut stream = response.bytes_stream();
let mut buffer = Vec::new();
while let Some(chunk) = stream.try_next().await.context("reading body")? {
if buffer.len() + chunk.len() > response_limit {
Err(anyhow::anyhow!("Response size too large"))?;
}
buffer.extend_from_slice(&chunk);
}
if !status.is_success() {
return Err(HttpResponseException::new(status, buffer));
}
let r = Python::with_gil(|py| buffer.into_pyobject(py).map(|o| o.unbind()))?;
Ok(r)
})
}
}
/// Creates a twisted deferred from the given future, spawning the task on the
/// tokio runtime.
///
/// Does not handle deferred cancellation or contextvars.
fn create_deferred<F, O>(py: Python, fut: F) -> PyResult<Bound<'_, PyAny>>
where
F: Future<Output = PyResult<O>> + Send + 'static,
for<'a> O: IntoPyObject<'a>,
{
let deferred = DEFERRED_CLASS.bind(py).call0()?;
let deferred_callback = deferred.getattr("callback")?.unbind();
let deferred_errback = deferred.getattr("errback")?.unbind();
RUNTIME.spawn(async move {
// TODO: Is it safe to assert unwind safety here? I think so, as we
// don't use anything that could be tainted by the panic afterwards.
// Note that `.spawn(..)` asserts unwind safety on the future too.
let res = AssertUnwindSafe(fut).catch_unwind().await;
Python::with_gil(move |py| {
// Flatten the panic into standard python error
let res = match res {
Ok(r) => r,
Err(panic_err) => {
let panic_message = get_panic_message(&panic_err);
Err(PyException::new_err(
PyString::new(py, panic_message).unbind(),
))
}
};
// Send the result to the deferred, via `.callback(..)` or `.errback(..)`
match res {
Ok(obj) => {
TWISTED_REACTOR
.call_method(py, "callFromThread", (deferred_callback, obj), None)
.expect("callFromThread should not fail"); // There's nothing we can really do with errors here
}
Err(err) => {
TWISTED_REACTOR
.call_method(py, "callFromThread", (deferred_errback, err), None)
.expect("callFromThread should not fail"); // There's nothing we can really do with errors here
}
}
});
});
Ok(deferred)
}
/// Try and get the panic message out of the panic
fn get_panic_message<'a>(panic_err: &'a (dyn std::any::Any + Send + 'static)) -> &'a str {
// Apparently this is how you extract the panic message from a panic
if let Some(str_slice) = panic_err.downcast_ref::<&str>() {
str_slice
} else if let Some(string) = panic_err.downcast_ref::<String>() {
string
} else {
"unknown error"
}
}

View File

@@ -8,6 +8,7 @@ pub mod acl;
pub mod errors;
pub mod events;
pub mod http;
pub mod http_client;
pub mod identifier;
pub mod matrix_const;
pub mod push;
@@ -50,6 +51,7 @@ fn synapse_rust(py: Python<'_>, m: &Bound<'_, PyModule>) -> PyResult<()> {
acl::register_module(py, m)?;
push::register_module(py, m)?;
events::register_module(py, m)?;
http_client::register_module(py, m)?;
rendezvous::register_module(py, m)?;
Ok(())

View File

@@ -229,6 +229,7 @@ test_packages=(
./tests/msc3902
./tests/msc3967
./tests/msc4140
./tests/msc4155
)
# Enable dirty runs, so tests will reuse the same container where possible.

View File

@@ -30,9 +30,6 @@ from authlib.oauth2.rfc7662 import IntrospectionToken
from authlib.oidc.discovery import OpenIDProviderMetadata, get_well_known_url
from prometheus_client import Histogram
from twisted.web.client import readBody
from twisted.web.http_headers import Headers
from synapse.api.auth.base import BaseAuth
from synapse.api.errors import (
AuthError,
@@ -43,8 +40,14 @@ from synapse.api.errors import (
UnrecognizedRequestError,
)
from synapse.http.site import SynapseRequest
from synapse.logging.context import make_deferred_yieldable
from synapse.logging.opentracing import active_span, force_tracing, start_active_span
from synapse.logging.context import PreserveLoggingContext
from synapse.logging.opentracing import (
active_span,
force_tracing,
inject_request_headers,
start_active_span,
)
from synapse.synapse_rust.http_client import HttpClient
from synapse.types import Requester, UserID, create_requester
from synapse.util import json_decoder
from synapse.util.caches.cached_call import RetryOnExceptionCachedCall
@@ -179,6 +182,10 @@ class MSC3861DelegatedAuth(BaseAuth):
self._admin_token: Callable[[], Optional[str]] = self._config.admin_token
self._force_tracing_for_users = hs.config.tracing.force_tracing_for_users
self._rust_http_client = HttpClient(
user_agent=self._http_client.user_agent.decode("utf8")
)
# # Token Introspection Cache
# This remembers what users/devices are represented by which access tokens,
# in order to reduce overall system load:
@@ -301,7 +308,6 @@ class MSC3861DelegatedAuth(BaseAuth):
introspection_endpoint = await self._introspection_endpoint()
raw_headers: Dict[str, str] = {
"Content-Type": "application/x-www-form-urlencoded",
"User-Agent": str(self._http_client.user_agent, "utf-8"),
"Accept": "application/json",
# Tell MAS that we support reading the device ID as an explicit
# value, not encoded in the scope. This is supported by MAS 0.15+
@@ -315,38 +321,34 @@ class MSC3861DelegatedAuth(BaseAuth):
uri, raw_headers, body = self._client_auth.prepare(
method="POST", uri=introspection_endpoint, headers=raw_headers, body=body
)
headers = Headers({k: [v] for (k, v) in raw_headers.items()})
# Do the actual request
# We're not using the SimpleHttpClient util methods as we don't want to
# check the HTTP status code, and we do the body encoding ourselves.
logger.debug("Fetching token from MAS")
start_time = self._clock.time()
try:
response = await self._http_client.request(
method="POST",
uri=uri,
data=body.encode("utf-8"),
headers=headers,
)
resp_body = await make_deferred_yieldable(readBody(response))
with start_active_span("mas-introspect-token"):
inject_request_headers(raw_headers)
with PreserveLoggingContext():
resp_body = await self._rust_http_client.post(
url=uri,
response_limit=1 * 1024 * 1024,
headers=raw_headers,
request_body=body,
)
except HttpResponseException as e:
end_time = self._clock.time()
introspection_response_timer.labels(e.code).observe(end_time - start_time)
raise
except Exception:
end_time = self._clock.time()
introspection_response_timer.labels("ERR").observe(end_time - start_time)
raise
end_time = self._clock.time()
introspection_response_timer.labels(response.code).observe(
end_time - start_time
)
logger.debug("Fetched token from MAS")
if response.code < 200 or response.code >= 300:
raise HttpResponseException(
response.code,
response.phrase.decode("ascii", errors="replace"),
resp_body,
)
end_time = self._clock.time()
introspection_response_timer.labels(200).observe(end_time - start_time)
resp = json_decoder.decode(resp_body.decode("utf-8"))

View File

@@ -185,12 +185,18 @@ ServerNoticeLimitReached: Final = "m.server_notice.usage_limit_reached"
class UserTypes:
"""Allows for user type specific behaviour. With the benefit of hindsight
'admin' and 'guest' users should also be UserTypes. Normal users are type None
'admin' and 'guest' users should also be UserTypes. Extra user types can be
added in the configuration. Normal users are type None or one of the extra
user types (if configured).
"""
SUPPORT: Final = "support"
BOT: Final = "bot"
ALL_USER_TYPES: Final = (SUPPORT, BOT)
ALL_BUILTIN_USER_TYPES: Final = (SUPPORT, BOT)
"""
The user types that are built-in to Synapse. Extra user types can be
added in the configuration.
"""
class RelationTypes:
@@ -280,6 +286,10 @@ class AccountDataTypes:
IGNORED_USER_LIST: Final = "m.ignored_user_list"
TAG: Final = "m.tag"
PUSH_RULES: Final = "m.push_rules"
# MSC4155: Invite filtering
MSC4155_INVITE_PERMISSION_CONFIG: Final = (
"org.matrix.msc4155.invite_permission_config"
)
class HistoryVisibility:

View File

@@ -137,6 +137,9 @@ class Codes(str, Enum):
PROFILE_TOO_LARGE = "M_PROFILE_TOO_LARGE"
KEY_TOO_LARGE = "M_KEY_TOO_LARGE"
# Part of MSC4155
INVITE_BLOCKED = "ORG.MATRIX.MSC4155.M_INVITE_BLOCKED"
class CodeMessageException(RuntimeError):
"""An exception with integer code, a message string attributes and optional headers.

View File

@@ -20,7 +20,7 @@
#
#
from typing import Dict, Hashable, Optional, Tuple
from typing import TYPE_CHECKING, Dict, Hashable, Optional, Tuple
from synapse.api.errors import LimitExceededError
from synapse.config.ratelimiting import RatelimitSettings
@@ -28,6 +28,12 @@ from synapse.storage.databases.main import DataStore
from synapse.types import Requester
from synapse.util import Clock
if TYPE_CHECKING:
# To avoid circular imports:
from synapse.module_api.callbacks.ratelimit_callbacks import (
RatelimitModuleApiCallbacks,
)
class Ratelimiter:
"""
@@ -72,12 +78,14 @@ class Ratelimiter:
store: DataStore,
clock: Clock,
cfg: RatelimitSettings,
ratelimit_callbacks: Optional["RatelimitModuleApiCallbacks"] = None,
):
self.clock = clock
self.rate_hz = cfg.per_second
self.burst_count = cfg.burst_count
self.store = store
self._limiter_name = cfg.key
self._ratelimit_callbacks = ratelimit_callbacks
# A dictionary representing the token buckets tracked by this rate
# limiter. Each entry maps a key of arbitrary type to a tuple representing:
@@ -165,6 +173,20 @@ class Ratelimiter:
if override and not override.messages_per_second:
return True, -1.0
if requester and self._ratelimit_callbacks:
# Check if the user has a custom rate limit for this specific limiter
# as returned by the module API.
module_override = (
await self._ratelimit_callbacks.get_ratelimit_override_for_user(
requester.user.to_string(),
self._limiter_name,
)
)
if module_override:
rate_hz = module_override.per_second
burst_count = module_override.burst_count
# Override default values if set
time_now_s = _time_now_s if _time_now_s is not None else self.clock.time()
rate_hz = rate_hz if rate_hz is not None else self.rate_hz

View File

@@ -2,7 +2,7 @@
# This file is licensed under the Affero General Public License (AGPL) version 3.
#
# Copyright 2015, 2016 OpenMarket Ltd
# Copyright (C) 2023 New Vector, Ltd
# Copyright (C) 2023, 2025 New Vector, Ltd
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as
@@ -70,6 +70,8 @@ from typing import (
Tuple,
)
from twisted.internet.interfaces import IDelayedCall
from synapse.appservice import (
ApplicationService,
ApplicationServiceState,
@@ -450,6 +452,20 @@ class _TransactionController:
recoverer.recover()
logger.info("Now %i active recoverers", len(self.recoverers))
def force_retry(self, service: ApplicationService) -> None:
"""Forces a Recoverer to attempt delivery of transations immediately.
Args:
service:
"""
recoverer = self.recoverers.get(service.id)
if not recoverer:
# No need to force a retry on a happy AS.
logger.info(f"{service.id} is not in recovery, not forcing retry")
return
recoverer.force_retry()
async def _is_service_up(self, service: ApplicationService) -> bool:
state = await self.store.get_appservice_state(service)
return state == ApplicationServiceState.UP or state is None
@@ -482,11 +498,12 @@ class _Recoverer:
self.service = service
self.callback = callback
self.backoff_counter = 1
self.scheduled_recovery: Optional[IDelayedCall] = None
def recover(self) -> None:
delay = 2**self.backoff_counter
logger.info("Scheduling retries on %s in %fs", self.service.id, delay)
self.clock.call_later(
self.scheduled_recovery = self.clock.call_later(
delay, run_as_background_process, "as-recoverer", self.retry
)
@@ -496,6 +513,21 @@ class _Recoverer:
self.backoff_counter += 1
self.recover()
def force_retry(self) -> None:
"""Cancels the existing timer and forces an immediate retry in the background.
Args:
service:
"""
# Prevent the existing backoff from occuring
if self.scheduled_recovery:
self.clock.cancel_call_later(self.scheduled_recovery)
# Run a retry, which will resechedule a recovery if it fails.
run_as_background_process(
"retry",
self.retry,
)
async def retry(self) -> None:
logger.info("Starting retries on %s", self.service.id)
try:

View File

@@ -59,6 +59,7 @@ from synapse.config import ( # noqa: F401
tls,
tracer,
user_directory,
user_types,
voip,
workers,
)
@@ -122,6 +123,7 @@ class RootConfig:
retention: retention.RetentionConfig
background_updates: background_updates.BackgroundUpdateConfig
auto_accept_invites: auto_accept_invites.AutoAcceptInvitesConfig
user_types: user_types.UserTypesConfig
config_classes: List[Type["Config"]] = ...
config_files: List[str]

View File

@@ -566,3 +566,6 @@ class ExperimentalConfig(Config):
"msc4263_limit_key_queries_to_users_who_share_rooms",
False,
)
# MSC4155: Invite filtering
self.msc4155_enabled: bool = experimental.get("msc4155_enabled", False)

View File

@@ -94,5 +94,21 @@ class FederationConfig(Config):
2**62,
)
def is_domain_allowed_according_to_federation_whitelist(self, domain: str) -> bool:
"""
Returns whether a domain is allowed according to the federation whitelist. If a
federation whitelist is not set, all domains are allowed.
Args:
domain: The domain to test.
Returns:
True if the domain is allowed or if a whitelist is not set, False otherwise.
"""
if self.federation_domain_whitelist is None:
return True
return domain in self.federation_domain_whitelist
_METRICS_FOR_DOMAINS_SCHEMA = {"type": "array", "items": {"type": "string"}}

View File

@@ -59,6 +59,7 @@ from .third_party_event_rules import ThirdPartyRulesConfig
from .tls import TlsConfig
from .tracer import TracerConfig
from .user_directory import UserDirectoryConfig
from .user_types import UserTypesConfig
from .voip import VoipConfig
from .workers import WorkerConfig
@@ -107,4 +108,5 @@ class HomeServerConfig(RootConfig):
ExperimentalConfig,
BackgroundUpdateConfig,
AutoAcceptInvitesConfig,
UserTypesConfig,
]

View File

@@ -0,0 +1,44 @@
#
# This file is licensed under the Affero General Public License (AGPL) version 3.
#
# Copyright (C) 2025 New Vector, Ltd
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as
# published by the Free Software Foundation, either version 3 of the
# License, or (at your option) any later version.
#
# See the GNU Affero General Public License for more details:
# <https://www.gnu.org/licenses/agpl-3.0.html>.
#
from typing import Any, List, Optional
from synapse.api.constants import UserTypes
from synapse.types import JsonDict
from ._base import Config, ConfigError
class UserTypesConfig(Config):
section = "user_types"
def read_config(self, config: JsonDict, **kwargs: Any) -> None:
user_types: JsonDict = config.get("user_types", {})
self.default_user_type: Optional[str] = user_types.get(
"default_user_type", None
)
self.extra_user_types: List[str] = user_types.get("extra_user_types", [])
all_user_types: List[str] = []
all_user_types.extend(UserTypes.ALL_BUILTIN_USER_TYPES)
all_user_types.extend(self.extra_user_types)
self.all_user_types = all_user_types
if self.default_user_type is not None:
if self.default_user_type not in all_user_types:
raise ConfigError(
f"Default user type {self.default_user_type} is not in the list of all user types: {all_user_types}"
)

View File

@@ -342,6 +342,8 @@ class _DestinationWakeupQueue:
destination, _ = self.queue.popitem(last=False)
queue = self.sender._get_per_destination_queue(destination)
if queue is None:
continue
if not queue._new_data_to_send:
# The per destination queue has already been woken up.
@@ -436,12 +438,23 @@ class FederationSender(AbstractFederationSender):
self._wake_destinations_needing_catchup,
)
def _get_per_destination_queue(self, destination: str) -> PerDestinationQueue:
def _get_per_destination_queue(
self, destination: str
) -> Optional[PerDestinationQueue]:
"""Get or create a PerDestinationQueue for the given destination
Args:
destination: server_name of remote server
Returns:
None if the destination is not allowed by the federation whitelist.
Otherwise a PerDestinationQueue for this destination.
"""
if not self.hs.config.federation.is_domain_allowed_according_to_federation_whitelist(
destination
):
return None
queue = self._per_destination_queues.get(destination)
if not queue:
queue = PerDestinationQueue(self.hs, self._transaction_manager, destination)
@@ -718,6 +731,16 @@ class FederationSender(AbstractFederationSender):
# track the fact that we have a PDU for these destinations,
# to allow us to perform catch-up later on if the remote is unreachable
# for a while.
# Filter out any destinations not present in the federation_domain_whitelist, if
# the whitelist exists. These destinations should not be sent to so let's not
# waste time or space keeping track of events destined for them.
destinations = [
d
for d in destinations
if self.hs.config.federation.is_domain_allowed_according_to_federation_whitelist(
d
)
]
await self.store.store_destination_rooms_entries(
destinations,
pdu.room_id,
@@ -732,7 +755,12 @@ class FederationSender(AbstractFederationSender):
)
for destination in destinations:
self._get_per_destination_queue(destination).send_pdu(pdu)
queue = self._get_per_destination_queue(destination)
# We expect `queue` to not be None as we already filtered out
# non-whitelisted destinations above.
assert queue is not None
queue.send_pdu(pdu)
async def send_read_receipt(self, receipt: ReadReceipt) -> None:
"""Send a RR to any other servers in the room
@@ -841,12 +869,16 @@ class FederationSender(AbstractFederationSender):
for domain in immediate_domains:
# Add to destination queue and wake the destination up
queue = self._get_per_destination_queue(domain)
if queue is None:
continue
queue.queue_read_receipt(receipt)
queue.attempt_new_transaction()
for domain in delay_domains:
# Add to destination queue...
queue = self._get_per_destination_queue(domain)
if queue is None:
continue
queue.queue_read_receipt(receipt)
# ... and schedule the destination to be woken up.
@@ -882,9 +914,10 @@ class FederationSender(AbstractFederationSender):
if self.is_mine_server_name(destination):
continue
self._get_per_destination_queue(destination).send_presence(
states, start_loop=False
)
queue = self._get_per_destination_queue(destination)
if queue is None:
continue
queue.send_presence(states, start_loop=False)
self._destination_wakeup_queue.add_to_queue(destination)
@@ -934,6 +967,8 @@ class FederationSender(AbstractFederationSender):
return
queue = self._get_per_destination_queue(edu.destination)
if queue is None:
return
if key:
queue.send_keyed_edu(edu, key)
else:
@@ -958,9 +993,15 @@ class FederationSender(AbstractFederationSender):
for destination in destinations:
if immediate:
self._get_per_destination_queue(destination).attempt_new_transaction()
queue = self._get_per_destination_queue(destination)
if queue is None:
continue
queue.attempt_new_transaction()
else:
self._get_per_destination_queue(destination).mark_new_data()
queue = self._get_per_destination_queue(destination)
if queue is None:
continue
queue.mark_new_data()
self._destination_wakeup_queue.add_to_queue(destination)
def wake_destination(self, destination: str) -> None:
@@ -979,7 +1020,9 @@ class FederationSender(AbstractFederationSender):
):
return
self._get_per_destination_queue(destination).attempt_new_transaction()
queue = self._get_per_destination_queue(destination)
if queue is not None:
queue.attempt_new_transaction()
@staticmethod
def get_current_token() -> int:
@@ -1024,6 +1067,9 @@ class FederationSender(AbstractFederationSender):
d
for d in destinations_to_wake
if self._federation_shard_config.should_handle(self._instance_name, d)
and self.hs.config.federation.is_domain_allowed_according_to_federation_whitelist(
d
)
]
for destination in destinations_to_wake:

View File

@@ -78,6 +78,7 @@ from synapse.replication.http.federation import (
ReplicationStoreRoomOnOutlierMembershipRestServlet,
)
from synapse.storage.databases.main.events_worker import EventRedactBehaviour
from synapse.storage.invite_rule import InviteRule
from synapse.types import JsonDict, StrCollection, get_domain_from_id
from synapse.types.state import StateFilter
from synapse.util.async_helpers import Linearizer
@@ -1089,6 +1090,20 @@ class FederationHandler:
if event.state_key == self._server_notices_mxid:
raise SynapseError(HTTPStatus.FORBIDDEN, "Cannot invite this user")
# check the invitee's configuration and apply rules
invite_config = await self.store.get_invite_config_for_user(event.state_key)
rule = invite_config.get_invite_rule(event.sender)
if rule == InviteRule.BLOCK:
logger.info(
f"Automatically rejecting invite from {event.sender} due to the invite filtering rules of {event.state_key}"
)
raise SynapseError(
403,
"You are not permitted to invite this user.",
errcode=Codes.INVITE_BLOCKED,
)
# InviteRule.IGNORE is handled at the sync layer
# We retrieve the room member handler here as to not cause a cyclic dependency
member_handler = self.hs.get_room_member_handler()
# We don't rate limit based on room ID, as that should be done by

View File

@@ -115,6 +115,7 @@ class RegistrationHandler:
self._user_consent_version = self.hs.config.consent.user_consent_version
self._server_notices_mxid = hs.config.servernotices.server_notices_mxid
self._server_name = hs.hostname
self._user_types_config = hs.config.user_types
self._spam_checker_module_callbacks = hs.get_module_api_callbacks().spam_checker
@@ -306,6 +307,9 @@ class RegistrationHandler:
elif default_display_name is None:
default_display_name = localpart
if user_type is None:
user_type = self._user_types_config.default_user_type
await self.register_with_store(
user_id=user_id,
password_hash=password_hash,

View File

@@ -468,17 +468,6 @@ class RoomCreationHandler:
"""
user_id = requester.user.to_string()
spam_check = await self._spam_checker_module_callbacks.user_may_create_room(
user_id
)
if spam_check != self._spam_checker_module_callbacks.NOT_SPAM:
raise SynapseError(
403,
"You are not permitted to create rooms",
errcode=spam_check[0],
additional_fields=spam_check[1],
)
creation_content: JsonDict = {
"room_version": new_room_version.identifier,
"predecessor": {"room_id": old_room_id, "event_id": tombstone_event_id},
@@ -585,6 +574,24 @@ class RoomCreationHandler:
if current_power_level_int < needed_power_level:
user_power_levels[user_id] = needed_power_level
# We construct what the body of a call to /createRoom would look like for passing
# to the spam checker. We don't include a preset here, as we expect the
# initial state to contain everything we need.
spam_check = await self._spam_checker_module_callbacks.user_may_create_room(
user_id,
{
"creation_content": creation_content,
"initial_state": list(initial_state.items()),
},
)
if spam_check != self._spam_checker_module_callbacks.NOT_SPAM:
raise SynapseError(
403,
"You are not permitted to create rooms",
errcode=spam_check[0],
additional_fields=spam_check[1],
)
await self._send_events_for_new_room(
requester,
new_room_id,
@@ -786,7 +793,7 @@ class RoomCreationHandler:
if not is_requester_admin:
spam_check = await self._spam_checker_module_callbacks.user_may_create_room(
user_id
user_id, config
)
if spam_check != self._spam_checker_module_callbacks.NOT_SPAM:
raise SynapseError(

View File

@@ -53,6 +53,7 @@ from synapse.metrics import event_processing_positions
from synapse.metrics.background_process_metrics import run_as_background_process
from synapse.replication.http.push import ReplicationCopyPusherRestServlet
from synapse.storage.databases.main.state_deltas import StateDelta
from synapse.storage.invite_rule import InviteRule
from synapse.types import (
JsonDict,
Requester,
@@ -158,6 +159,7 @@ class RoomMemberHandler(metaclass=abc.ABCMeta):
store=self.store,
clock=self.clock,
cfg=hs.config.ratelimiting.rc_invites_per_room,
ratelimit_callbacks=hs.get_module_api_callbacks().ratelimit,
)
# Ratelimiter for invites, keyed by recipient (across all rooms, all
@@ -166,6 +168,7 @@ class RoomMemberHandler(metaclass=abc.ABCMeta):
store=self.store,
clock=self.clock,
cfg=hs.config.ratelimiting.rc_invites_per_user,
ratelimit_callbacks=hs.get_module_api_callbacks().ratelimit,
)
# Ratelimiter for invites, keyed by issuer (across all rooms, all
@@ -174,6 +177,7 @@ class RoomMemberHandler(metaclass=abc.ABCMeta):
store=self.store,
clock=self.clock,
cfg=hs.config.ratelimiting.rc_invites_per_issuer,
ratelimit_callbacks=hs.get_module_api_callbacks().ratelimit,
)
self._third_party_invite_limiter = Ratelimiter(
@@ -912,6 +916,21 @@ class RoomMemberHandler(metaclass=abc.ABCMeta):
additional_fields=block_invite_result[1],
)
# check the invitee's configuration and apply rules. Admins on the server can bypass.
if not is_requester_admin:
invite_config = await self.store.get_invite_config_for_user(target_id)
rule = invite_config.get_invite_rule(requester.user.to_string())
if rule == InviteRule.BLOCK:
logger.info(
f"Automatically rejecting invite from {target_id} due to the the invite filtering rules of {requester.user}"
)
raise SynapseError(
403,
"You are not permitted to invite this user.",
errcode=Codes.INVITE_BLOCKED,
)
# InviteRule.IGNORE is handled at the sync layer.
# An empty prev_events list is allowed as long as the auth_event_ids are present
if prev_event_ids is not None:
return await self._local_membership_update(

View File

@@ -49,6 +49,7 @@ from synapse.storage.databases.main.state import (
Sentinel as StateSentinel,
)
from synapse.storage.databases.main.stream import CurrentStateDeltaMembership
from synapse.storage.invite_rule import InviteRule
from synapse.storage.roommember import (
RoomsForUser,
RoomsForUserSlidingSync,
@@ -278,6 +279,7 @@ class SlidingSyncRoomLists:
# Remove invites from ignored users
ignored_users = await self.store.ignored_users(user_id)
invite_config = await self.store.get_invite_config_for_user(user_id)
if ignored_users:
# FIXME: It would be nice to avoid this copy but since
# `get_sliding_sync_rooms_for_user_from_membership_snapshots` is cached, it
@@ -292,7 +294,14 @@ class SlidingSyncRoomLists:
room_for_user_sliding_sync = room_membership_for_user_map[room_id]
if (
room_for_user_sliding_sync.membership == Membership.INVITE
and room_for_user_sliding_sync.sender in ignored_users
and room_for_user_sliding_sync.sender
and (
room_for_user_sliding_sync.sender in ignored_users
or invite_config.get_invite_rule(
room_for_user_sliding_sync.sender
)
== InviteRule.IGNORE
)
):
room_membership_for_user_map.pop(room_id, None)

View File

@@ -66,6 +66,7 @@ from synapse.logging.opentracing import (
from synapse.storage.databases.main.event_push_actions import RoomNotifCounts
from synapse.storage.databases.main.roommember import extract_heroes_from_room_summary
from synapse.storage.databases.main.stream import PaginateFunction
from synapse.storage.invite_rule import InviteRule
from synapse.storage.roommember import MemberSummary
from synapse.types import (
DeviceListUpdates,
@@ -2549,6 +2550,7 @@ class SyncHandler:
room_entries: List[RoomSyncResultBuilder] = []
invited: List[InvitedSyncResult] = []
knocked: List[KnockedSyncResult] = []
invite_config = await self.store.get_invite_config_for_user(user_id)
for room_id, events in mem_change_events_by_room_id.items():
# The body of this loop will add this room to at least one of the five lists
# above. Things get messy if you've e.g. joined, left, joined then left the
@@ -2631,7 +2633,11 @@ class SyncHandler:
# Only bother if we're still currently invited
should_invite = last_non_join.membership == Membership.INVITE
if should_invite:
if last_non_join.sender not in ignored_users:
if (
last_non_join.sender not in ignored_users
and invite_config.get_invite_rule(last_non_join.sender)
!= InviteRule.IGNORE
):
invite_room_sync = InvitedSyncResult(room_id, invite=last_non_join)
if invite_room_sync:
invited.append(invite_room_sync)
@@ -2786,6 +2792,7 @@ class SyncHandler:
membership_list=Membership.LIST,
excluded_rooms=sync_result_builder.excluded_room_ids,
)
invite_config = await self.store.get_invite_config_for_user(user_id)
room_entries = []
invited = []
@@ -2811,6 +2818,8 @@ class SyncHandler:
elif event.membership == Membership.INVITE:
if event.sender in ignored_users:
continue
if invite_config.get_invite_rule(event.sender) == InviteRule.IGNORE:
continue
invite = await self.store.get_event(event.event_id)
invited.append(InvitedSyncResult(room_id=event.room_id, invite=invite))
elif event.membership == Membership.KNOCK:

View File

@@ -796,6 +796,13 @@ def inject_response_headers(response_headers: Headers) -> None:
response_headers.addRawHeader("Synapse-Trace-Id", f"{trace_id:x}")
@ensure_active_span("inject the span into a header dict")
def inject_request_headers(headers: Dict[str, str]) -> None:
span = opentracing.tracer.active_span
assert span is not None
opentracing.tracer.inject(span.context, opentracing.Format.HTTP_HEADERS, headers)
@ensure_active_span(
"get the active span context as a dict", ret=cast(Dict[str, str], {})
)

View File

@@ -90,6 +90,14 @@ from synapse.module_api.callbacks.account_validity_callbacks import (
ON_USER_LOGIN_CALLBACK,
ON_USER_REGISTRATION_CALLBACK,
)
from synapse.module_api.callbacks.media_repository_callbacks import (
GET_MEDIA_CONFIG_FOR_USER_CALLBACK,
IS_USER_ALLOWED_TO_UPLOAD_MEDIA_OF_SIZE_CALLBACK,
)
from synapse.module_api.callbacks.ratelimit_callbacks import (
GET_RATELIMIT_OVERRIDE_FOR_USER_CALLBACK,
RatelimitOverride,
)
from synapse.module_api.callbacks.spamchecker_callbacks import (
CHECK_EVENT_FOR_SPAM_CALLBACK,
CHECK_LOGIN_FOR_SPAM_CALLBACK,
@@ -103,6 +111,7 @@ from synapse.module_api.callbacks.spamchecker_callbacks import (
USER_MAY_JOIN_ROOM_CALLBACK,
USER_MAY_PUBLISH_ROOM_CALLBACK,
USER_MAY_SEND_3PID_INVITE_CALLBACK,
USER_MAY_SEND_STATE_EVENT_CALLBACK,
SpamCheckerModuleApiCallbacks,
)
from synapse.module_api.callbacks.third_party_event_rules_callbacks import (
@@ -189,6 +198,7 @@ __all__ = [
"ProfileInfo",
"RoomAlias",
"UserProfile",
"RatelimitOverride",
]
logger = logging.getLogger(__name__)
@@ -311,6 +321,7 @@ class ModuleApi:
USER_MAY_CREATE_ROOM_ALIAS_CALLBACK
] = None,
user_may_publish_room: Optional[USER_MAY_PUBLISH_ROOM_CALLBACK] = None,
user_may_send_state_event: Optional[USER_MAY_SEND_STATE_EVENT_CALLBACK] = None,
check_username_for_spam: Optional[CHECK_USERNAME_FOR_SPAM_CALLBACK] = None,
check_registration_for_spam: Optional[
CHECK_REGISTRATION_FOR_SPAM_CALLBACK
@@ -335,6 +346,7 @@ class ModuleApi:
check_registration_for_spam=check_registration_for_spam,
check_media_file_for_spam=check_media_file_for_spam,
check_login_for_spam=check_login_for_spam,
user_may_send_state_event=user_may_send_state_event,
)
def register_account_validity_callbacks(
@@ -360,6 +372,36 @@ class ModuleApi:
on_legacy_admin_request=on_legacy_admin_request,
)
def register_ratelimit_callbacks(
self,
*,
get_ratelimit_override_for_user: Optional[
GET_RATELIMIT_OVERRIDE_FOR_USER_CALLBACK
] = None,
) -> None:
"""Registers callbacks for ratelimit capabilities.
Added in Synapse v1.132.0.
"""
return self._callbacks.ratelimit.register_callbacks(
get_ratelimit_override_for_user=get_ratelimit_override_for_user,
)
def register_media_repository_callbacks(
self,
*,
get_media_config_for_user: Optional[GET_MEDIA_CONFIG_FOR_USER_CALLBACK] = None,
is_user_allowed_to_upload_media_of_size: Optional[
IS_USER_ALLOWED_TO_UPLOAD_MEDIA_OF_SIZE_CALLBACK
] = None,
) -> None:
"""Registers callbacks for media repository capabilities.
Added in Synapse v1.132.0.
"""
return self._callbacks.media_repository.register_callbacks(
get_media_config_for_user=get_media_config_for_user,
is_user_allowed_to_upload_media_of_size=is_user_allowed_to_upload_media_of_size,
)
def register_third_party_rules_callbacks(
self,
*,

View File

@@ -27,6 +27,12 @@ if TYPE_CHECKING:
from synapse.module_api.callbacks.account_validity_callbacks import (
AccountValidityModuleApiCallbacks,
)
from synapse.module_api.callbacks.media_repository_callbacks import (
MediaRepositoryModuleApiCallbacks,
)
from synapse.module_api.callbacks.ratelimit_callbacks import (
RatelimitModuleApiCallbacks,
)
from synapse.module_api.callbacks.spamchecker_callbacks import (
SpamCheckerModuleApiCallbacks,
)
@@ -38,5 +44,7 @@ from synapse.module_api.callbacks.third_party_event_rules_callbacks import (
class ModuleApiCallbacks:
def __init__(self, hs: "HomeServer") -> None:
self.account_validity = AccountValidityModuleApiCallbacks()
self.media_repository = MediaRepositoryModuleApiCallbacks(hs)
self.ratelimit = RatelimitModuleApiCallbacks(hs)
self.spam_checker = SpamCheckerModuleApiCallbacks(hs)
self.third_party_event_rules = ThirdPartyEventRulesModuleApiCallbacks(hs)

View File

@@ -0,0 +1,76 @@
#
# This file is licensed under the Affero General Public License (AGPL) version 3.
#
# Copyright (C) 2025 New Vector, Ltd
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as
# published by the Free Software Foundation, either version 3 of the
# License, or (at your option) any later version.
#
# See the GNU Affero General Public License for more details:
# <https://www.gnu.org/licenses/agpl-3.0.html>.
#
import logging
from typing import TYPE_CHECKING, Awaitable, Callable, List, Optional
from synapse.types import JsonDict
from synapse.util.async_helpers import delay_cancellation
from synapse.util.metrics import Measure
if TYPE_CHECKING:
from synapse.server import HomeServer
logger = logging.getLogger(__name__)
GET_MEDIA_CONFIG_FOR_USER_CALLBACK = Callable[[str], Awaitable[Optional[JsonDict]]]
IS_USER_ALLOWED_TO_UPLOAD_MEDIA_OF_SIZE_CALLBACK = Callable[[str, int], Awaitable[bool]]
class MediaRepositoryModuleApiCallbacks:
def __init__(self, hs: "HomeServer") -> None:
self.clock = hs.get_clock()
self._get_media_config_for_user_callbacks: List[
GET_MEDIA_CONFIG_FOR_USER_CALLBACK
] = []
self._is_user_allowed_to_upload_media_of_size_callbacks: List[
IS_USER_ALLOWED_TO_UPLOAD_MEDIA_OF_SIZE_CALLBACK
] = []
def register_callbacks(
self,
get_media_config_for_user: Optional[GET_MEDIA_CONFIG_FOR_USER_CALLBACK] = None,
is_user_allowed_to_upload_media_of_size: Optional[
IS_USER_ALLOWED_TO_UPLOAD_MEDIA_OF_SIZE_CALLBACK
] = None,
) -> None:
"""Register callbacks from module for each hook."""
if get_media_config_for_user is not None:
self._get_media_config_for_user_callbacks.append(get_media_config_for_user)
if is_user_allowed_to_upload_media_of_size is not None:
self._is_user_allowed_to_upload_media_of_size_callbacks.append(
is_user_allowed_to_upload_media_of_size
)
async def get_media_config_for_user(self, user_id: str) -> Optional[JsonDict]:
for callback in self._get_media_config_for_user_callbacks:
with Measure(self.clock, f"{callback.__module__}.{callback.__qualname__}"):
res: Optional[JsonDict] = await delay_cancellation(callback(user_id))
if res:
return res
return None
async def is_user_allowed_to_upload_media_of_size(
self, user_id: str, size: int
) -> bool:
for callback in self._is_user_allowed_to_upload_media_of_size_callbacks:
with Measure(self.clock, f"{callback.__module__}.{callback.__qualname__}"):
res: bool = await delay_cancellation(callback(user_id, size))
if not res:
return res
return True

View File

@@ -0,0 +1,74 @@
#
# This file is licensed under the Affero General Public License (AGPL) version 3.
#
# Copyright (C) 2025 New Vector, Ltd
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as
# published by the Free Software Foundation, either version 3 of the
# License, or (at your option) any later version.
#
# See the GNU Affero General Public License for more details:
# <https://www.gnu.org/licenses/agpl-3.0.html>.
#
import logging
from typing import TYPE_CHECKING, Awaitable, Callable, List, Optional
import attr
from synapse.util.async_helpers import delay_cancellation
from synapse.util.metrics import Measure
if TYPE_CHECKING:
from synapse.server import HomeServer
logger = logging.getLogger(__name__)
@attr.s(auto_attribs=True)
class RatelimitOverride:
"""Represents a ratelimit being overridden."""
per_second: float
"""The number of actions that can be performed in a second. `0.0` means that ratelimiting is disabled."""
burst_count: int
"""How many actions that can be performed before being limited."""
GET_RATELIMIT_OVERRIDE_FOR_USER_CALLBACK = Callable[
[str, str], Awaitable[Optional[RatelimitOverride]]
]
class RatelimitModuleApiCallbacks:
def __init__(self, hs: "HomeServer") -> None:
self.clock = hs.get_clock()
self._get_ratelimit_override_for_user_callbacks: List[
GET_RATELIMIT_OVERRIDE_FOR_USER_CALLBACK
] = []
def register_callbacks(
self,
get_ratelimit_override_for_user: Optional[
GET_RATELIMIT_OVERRIDE_FOR_USER_CALLBACK
] = None,
) -> None:
"""Register callbacks from module for each hook."""
if get_ratelimit_override_for_user is not None:
self._get_ratelimit_override_for_user_callbacks.append(
get_ratelimit_override_for_user
)
async def get_ratelimit_override_for_user(
self, user_id: str, limiter_name: str
) -> Optional[RatelimitOverride]:
for callback in self._get_ratelimit_override_for_user_callbacks:
with Measure(self.clock, f"{callback.__module__}.{callback.__qualname__}"):
res: Optional[RatelimitOverride] = await delay_cancellation(
callback(user_id, limiter_name)
)
if res:
return res
return None

View File

@@ -22,6 +22,7 @@
import functools
import inspect
import logging
from copy import deepcopy
from typing import (
TYPE_CHECKING,
Any,
@@ -120,20 +121,24 @@ USER_MAY_SEND_3PID_INVITE_CALLBACK = Callable[
]
],
]
USER_MAY_CREATE_ROOM_CALLBACK = Callable[
[str],
Awaitable[
Union[
Literal["NOT_SPAM"],
Codes,
# Highly experimental, not officially part of the spamchecker API, may
# disappear without warning depending on the results of ongoing
# experiments.
# Use this to return additional information as part of an error.
Tuple[Codes, JsonDict],
# Deprecated
bool,
]
USER_MAY_CREATE_ROOM_CALLBACK_RETURN_VALUE = Union[
Literal["NOT_SPAM"],
Codes,
# Highly experimental, not officially part of the spamchecker API, may
# disappear without warning depending on the results of ongoing
# experiments.
# Use this to return additional information as part of an error.
Tuple[Codes, JsonDict],
# Deprecated
bool,
]
USER_MAY_CREATE_ROOM_CALLBACK = Union[
Callable[
[str, JsonDict],
Awaitable[USER_MAY_CREATE_ROOM_CALLBACK_RETURN_VALUE],
],
Callable[ # Single argument variant for backwards compatibility
[str], Awaitable[USER_MAY_CREATE_ROOM_CALLBACK_RETURN_VALUE]
],
]
USER_MAY_CREATE_ROOM_ALIAS_CALLBACK = Callable[
@@ -168,6 +173,20 @@ USER_MAY_PUBLISH_ROOM_CALLBACK = Callable[
]
],
]
USER_MAY_SEND_STATE_EVENT_CALLBACK = Callable[
[str, str, str, str, JsonDict],
Awaitable[
Union[
Literal["NOT_SPAM"],
Codes,
# Highly experimental, not officially part of the spamchecker API, may
# disappear without warning depending on the results of ongoing
# experiments.
# Use this to return additional information as part of an error.
Tuple[Codes, JsonDict],
]
],
]
CHECK_USERNAME_FOR_SPAM_CALLBACK = Union[
Callable[[UserProfile], Awaitable[bool]],
Callable[[UserProfile, str], Awaitable[bool]],
@@ -332,6 +351,9 @@ class SpamCheckerModuleApiCallbacks:
USER_MAY_SEND_3PID_INVITE_CALLBACK
] = []
self._user_may_create_room_callbacks: List[USER_MAY_CREATE_ROOM_CALLBACK] = []
self._user_may_send_state_event_callbacks: List[
USER_MAY_SEND_STATE_EVENT_CALLBACK
] = []
self._user_may_create_room_alias_callbacks: List[
USER_MAY_CREATE_ROOM_ALIAS_CALLBACK
] = []
@@ -367,6 +389,7 @@ class SpamCheckerModuleApiCallbacks:
] = None,
check_media_file_for_spam: Optional[CHECK_MEDIA_FILE_FOR_SPAM_CALLBACK] = None,
check_login_for_spam: Optional[CHECK_LOGIN_FOR_SPAM_CALLBACK] = None,
user_may_send_state_event: Optional[USER_MAY_SEND_STATE_EVENT_CALLBACK] = None,
) -> None:
"""Register callbacks from module for each hook."""
if check_event_for_spam is not None:
@@ -391,6 +414,11 @@ class SpamCheckerModuleApiCallbacks:
if user_may_create_room is not None:
self._user_may_create_room_callbacks.append(user_may_create_room)
if user_may_send_state_event is not None:
self._user_may_send_state_event_callbacks.append(
user_may_send_state_event,
)
if user_may_create_room_alias is not None:
self._user_may_create_room_alias_callbacks.append(
user_may_create_room_alias,
@@ -622,16 +650,41 @@ class SpamCheckerModuleApiCallbacks:
return self.NOT_SPAM
async def user_may_create_room(
self, userid: str
self, userid: str, room_config: JsonDict
) -> Union[Tuple[Codes, dict], Literal["NOT_SPAM"]]:
"""Checks if a given user may create a room
Args:
userid: The ID of the user attempting to create a room
room_config: The room creation configuration which is the body of the /createRoom request
"""
for callback in self._user_may_create_room_callbacks:
with Measure(self.clock, f"{callback.__module__}.{callback.__qualname__}"):
res = await delay_cancellation(callback(userid))
checker_args = inspect.signature(callback)
# Also ensure backwards compatibility with spam checker callbacks
# that don't expect the room_config argument.
if len(checker_args.parameters) == 2:
callback_with_requester_id = cast(
Callable[
[str, JsonDict],
Awaitable[USER_MAY_CREATE_ROOM_CALLBACK_RETURN_VALUE],
],
callback,
)
# We make a copy of the config to ensure the spam checker cannot modify it.
res = await delay_cancellation(
callback_with_requester_id(userid, deepcopy(room_config))
)
else:
callback_without_requester_id = cast(
Callable[
[str], Awaitable[USER_MAY_CREATE_ROOM_CALLBACK_RETURN_VALUE]
],
callback,
)
res = await delay_cancellation(
callback_without_requester_id(userid)
)
if res is True or res is self.NOT_SPAM:
continue
elif res is False:
@@ -653,6 +706,40 @@ class SpamCheckerModuleApiCallbacks:
return self.NOT_SPAM
async def user_may_send_state_event(
self,
user_id: str,
room_id: str,
event_type: str,
state_key: str,
content: JsonDict,
) -> Union[Tuple[Codes, dict], Literal["NOT_SPAM"]]:
"""Checks if a given user may create a room with a given visibility
Args:
user_id: The ID of the user attempting to create a room
room_id: The ID of the room that the event will be sent to
event_type: The type of the state event
state_key: The state key of the state event
content: The content of the state event
"""
for callback in self._user_may_send_state_event_callbacks:
with Measure(self.clock, f"{callback.__module__}.{callback.__qualname__}"):
# We make a copy of the content to ensure that the spam checker cannot modify it.
res = await delay_cancellation(
callback(user_id, room_id, event_type, state_key, deepcopy(content))
)
if res is self.NOT_SPAM:
continue
elif isinstance(res, synapse.api.errors.Codes):
return res, {}
else:
logger.warning(
"Module returned invalid value, rejecting room creation as spam"
)
return synapse.api.errors.Codes.FORBIDDEN, {}
return self.NOT_SPAM
async def user_may_create_room_alias(
self, userid: str, room_alias: RoomAlias
) -> Union[Tuple[Codes, dict], Literal["NOT_SPAM"]]:

View File

@@ -52,6 +52,7 @@ from synapse.events.snapshot import EventContext
from synapse.logging.context import make_deferred_yieldable, run_in_background
from synapse.state import POWER_KEY
from synapse.storage.databases.main.roommember import EventIdMembership
from synapse.storage.invite_rule import InviteRule
from synapse.storage.roommember import ProfileInfo
from synapse.synapse_rust.push import FilteredPushRules, PushRuleEvaluator
from synapse.types import JsonValue
@@ -191,9 +192,17 @@ class BulkPushRuleEvaluator:
# if this event is an invite event, we may need to run rules for the user
# who's been invited, otherwise they won't get told they've been invited
if event.type == EventTypes.Member and event.membership == Membership.INVITE:
if (
event.is_state()
and event.type == EventTypes.Member
and event.membership == Membership.INVITE
):
invited = event.state_key
if invited and self.hs.is_mine_id(invited) and invited not in local_users:
invite_config = await self.store.get_invite_config_for_user(invited)
if invite_config.get_invite_rule(event.sender) != InviteRule.ALLOW:
# Invite was blocked or ignored, never notify.
return {}
if self.hs.is_mine_id(invited) and invited not in local_users:
local_users.append(invited)
if not local_users:

View File

@@ -28,7 +28,7 @@ from typing import TYPE_CHECKING, Dict, List, Optional, Tuple, Union
import attr
from synapse._pydantic_compat import StrictBool, StrictInt, StrictStr
from synapse.api.constants import Direction, UserTypes
from synapse.api.constants import Direction
from synapse.api.errors import Codes, NotFoundError, SynapseError
from synapse.http.servlet import (
RestServlet,
@@ -230,6 +230,7 @@ class UserRestServletV2(RestServlet):
self.registration_handler = hs.get_registration_handler()
self.pusher_pool = hs.get_pusherpool()
self._msc3866_enabled = hs.config.experimental.msc3866.enabled
self._all_user_types = hs.config.user_types.all_user_types
async def on_GET(
self, request: SynapseRequest, user_id: str
@@ -277,7 +278,7 @@ class UserRestServletV2(RestServlet):
assert_params_in_dict(external_id, ["auth_provider", "external_id"])
user_type = body.get("user_type", None)
if user_type is not None and user_type not in UserTypes.ALL_USER_TYPES:
if user_type is not None and user_type not in self._all_user_types:
raise SynapseError(HTTPStatus.BAD_REQUEST, "Invalid user type")
set_admin_to = body.get("admin", False)
@@ -524,6 +525,7 @@ class UserRegisterServlet(RestServlet):
self.reactor = hs.get_reactor()
self.nonces: Dict[str, int] = {}
self.hs = hs
self._all_user_types = hs.config.user_types.all_user_types
def _clear_old_nonces(self) -> None:
"""
@@ -605,7 +607,7 @@ class UserRegisterServlet(RestServlet):
user_type = body.get("user_type", None)
displayname = body.get("displayname", None)
if user_type is not None and user_type not in UserTypes.ALL_USER_TYPES:
if user_type is not None and user_type not in self._all_user_types:
raise SynapseError(HTTPStatus.BAD_REQUEST, "Invalid user type")
if "mac" not in body:

View File

@@ -2,7 +2,7 @@
# This file is licensed under the Affero General Public License (AGPL) version 3.
#
# Copyright 2023 Tulir Asokan
# Copyright (C) 2023 New Vector, Ltd
# Copyright (C) 2023, 2025 New Vector, Ltd
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as
@@ -53,6 +53,7 @@ class AppservicePingRestServlet(RestServlet):
def __init__(self, hs: "HomeServer"):
super().__init__()
self.as_api = hs.get_application_service_api()
self.scheduler = hs.get_application_service_scheduler()
self.auth = hs.get_auth()
async def on_POST(
@@ -85,6 +86,10 @@ class AppservicePingRestServlet(RestServlet):
start = time.monotonic()
try:
await self.as_api.ping(requester.app_service, txn_id)
# We got a OK response, so if the AS needs to be recovered then lets recover it now.
# This sets off a task in the background and so is safe to execute and forget.
self.scheduler.txn_ctrl.force_retry(requester.app_service)
except RequestTimedOutError as e:
raise SynapseError(
HTTPStatus.GATEWAY_TIMEOUT,

View File

@@ -102,10 +102,17 @@ class MediaConfigResource(RestServlet):
self.clock = hs.get_clock()
self.auth = hs.get_auth()
self.limits_dict = {"m.upload.size": config.media.max_upload_size}
self.media_repository_callbacks = hs.get_module_api_callbacks().media_repository
async def on_GET(self, request: SynapseRequest) -> None:
await self.auth.get_user_by_req(request)
respond_with_json(request, 200, self.limits_dict, send_cors=True)
requester = await self.auth.get_user_by_req(request)
user_specific_config = (
await self.media_repository_callbacks.get_media_config_for_user(
requester.user.to_string(),
)
)
response = user_specific_config if user_specific_config else self.limits_dict
respond_with_json(request, 200, response, send_cors=True)
class ThumbnailResource(RestServlet):

View File

@@ -198,6 +198,7 @@ class RoomStateEventRestServlet(RestServlet):
self.delayed_events_handler = hs.get_delayed_events_handler()
self.auth = hs.get_auth()
self._max_event_delay_ms = hs.config.server.max_event_delay_ms
self._spam_checker_module_callbacks = hs.get_module_api_callbacks().spam_checker
def register(self, http_server: HttpServer) -> None:
# /rooms/$roomid/state/$eventtype
@@ -289,6 +290,25 @@ class RoomStateEventRestServlet(RestServlet):
content = parse_json_object_from_request(request)
is_requester_admin = await self.auth.is_server_admin(requester)
if not is_requester_admin:
spam_check = (
await self._spam_checker_module_callbacks.user_may_send_state_event(
user_id=requester.user.to_string(),
room_id=room_id,
event_type=event_type,
state_key=state_key,
content=content,
)
)
if spam_check != self._spam_checker_module_callbacks.NOT_SPAM:
raise SynapseError(
403,
"You are not permitted to send the state event",
errcode=spam_check[0],
additional_fields=spam_check[1],
)
origin_server_ts = None
if requester.app_service:
origin_server_ts = parse_integer(request, "ts")

View File

@@ -174,6 +174,8 @@ class VersionsRestServlet(RestServlet):
"org.matrix.simplified_msc3575": msc3575_enabled,
# Arbitrary key-value profile fields.
"uk.tcpip.msc4133": self.config.experimental.msc4133_enabled,
# MSC4155: Invite filtering
"org.matrix.msc4155": self.config.experimental.msc4155_enabled,
},
},
)

View File

@@ -40,7 +40,14 @@ class MediaConfigResource(RestServlet):
self.clock = hs.get_clock()
self.auth = hs.get_auth()
self.limits_dict = {"m.upload.size": config.media.max_upload_size}
self.media_repository_callbacks = hs.get_module_api_callbacks().media_repository
async def on_GET(self, request: SynapseRequest) -> None:
await self.auth.get_user_by_req(request)
respond_with_json(request, 200, self.limits_dict, send_cors=True)
requester = await self.auth.get_user_by_req(request)
user_specific_config = (
await self.media_repository_callbacks.get_media_config_for_user(
requester.user.to_string()
)
)
response = user_specific_config if user_specific_config else self.limits_dict
respond_with_json(request, 200, response, send_cors=True)

View File

@@ -50,9 +50,12 @@ class BaseUploadServlet(RestServlet):
self.server_name = hs.hostname
self.auth = hs.get_auth()
self.max_upload_size = hs.config.media.max_upload_size
self._media_repository_callbacks = (
hs.get_module_api_callbacks().media_repository
)
def _get_file_metadata(
self, request: SynapseRequest
async def _get_file_metadata(
self, request: SynapseRequest, user_id: str
) -> Tuple[int, Optional[str], str]:
raw_content_length = request.getHeader("Content-Length")
if raw_content_length is None:
@@ -67,7 +70,14 @@ class BaseUploadServlet(RestServlet):
code=413,
errcode=Codes.TOO_LARGE,
)
if not await self._media_repository_callbacks.is_user_allowed_to_upload_media_of_size(
user_id, content_length
):
raise SynapseError(
msg="Upload request body is too large",
code=413,
errcode=Codes.TOO_LARGE,
)
args: Dict[bytes, List[bytes]] = request.args # type: ignore
upload_name_bytes = parse_bytes_from_args(args, "filename")
if upload_name_bytes:
@@ -104,7 +114,9 @@ class UploadServlet(BaseUploadServlet):
async def on_POST(self, request: SynapseRequest) -> None:
requester = await self.auth.get_user_by_req(request)
content_length, upload_name, media_type = self._get_file_metadata(request)
content_length, upload_name, media_type = await self._get_file_metadata(
request, requester.user.to_string()
)
try:
content: IO = request.content # type: ignore
@@ -152,7 +164,9 @@ class AsyncUploadServlet(BaseUploadServlet):
async with lock:
await self.media_repo.verify_can_upload(media_id, requester.user)
content_length, upload_name, media_type = self._get_file_metadata(request)
content_length, upload_name, media_type = await self._get_file_metadata(
request, requester.user.to_string()
)
try:
content: IO = request.content # type: ignore

View File

@@ -34,6 +34,7 @@ from typing import (
)
from synapse.api.constants import AccountDataTypes
from synapse.api.errors import Codes, SynapseError
from synapse.replication.tcp.streams import AccountDataStream
from synapse.storage._base import db_to_json
from synapse.storage.database import (
@@ -43,6 +44,7 @@ from synapse.storage.database import (
)
from synapse.storage.databases.main.cache import CacheInvalidationWorkerStore
from synapse.storage.databases.main.push_rule import PushRulesWorkerStore
from synapse.storage.invite_rule import InviteRulesConfig
from synapse.storage.util.id_generators import MultiWriterIdGenerator
from synapse.types import JsonDict, JsonMapping
from synapse.util import json_encoder
@@ -102,6 +104,8 @@ class AccountDataWorkerStore(PushRulesWorkerStore, CacheInvalidationWorkerStore)
self._delete_account_data_for_deactivated_users,
)
self._msc4155_enabled = hs.config.experimental.msc4155_enabled
def get_max_account_data_stream_id(self) -> int:
"""Get the current max stream ID for account data stream
@@ -557,6 +561,23 @@ class AccountDataWorkerStore(PushRulesWorkerStore, CacheInvalidationWorkerStore)
)
)
async def get_invite_config_for_user(self, user_id: str) -> InviteRulesConfig:
"""
Get the invite configuration for the current user.
Args:
user_id:
"""
if not self._msc4155_enabled:
# This equates to allowing all invites, as if the setting was off.
return InviteRulesConfig(None)
data = await self.get_global_account_data_by_type_for_user(
user_id, AccountDataTypes.MSC4155_INVITE_PERMISSION_CONFIG
)
return InviteRulesConfig(data)
def process_replication_rows(
self,
stream_name: str,
@@ -760,6 +781,9 @@ class AccountDataWorkerStore(PushRulesWorkerStore, CacheInvalidationWorkerStore)
else:
currently_ignored_users = set()
if user_id in currently_ignored_users:
raise SynapseError(400, "You cannot ignore yourself", Codes.INVALID_PARAM)
# If the data has not changed, nothing to do.
if previously_ignored_users == currently_ignored_users:
return

View File

@@ -583,7 +583,9 @@ class RegistrationWorkerStore(CacheInvalidationWorkerStore):
await self.db_pool.runInteraction("set_shadow_banned", set_shadow_banned_txn)
async def set_user_type(self, user: UserID, user_type: Optional[UserTypes]) -> None:
async def set_user_type(
self, user: UserID, user_type: Optional[Union[UserTypes, str]]
) -> None:
"""Sets the user type.
Args:
@@ -683,7 +685,7 @@ class RegistrationWorkerStore(CacheInvalidationWorkerStore):
retcol="user_type",
allow_none=True,
)
return res is None
return res is None or res not in [UserTypes.BOT, UserTypes.SUPPORT]
def is_support_user_txn(self, txn: LoggingTransaction, user_id: str) -> bool:
res = self.db_pool.simple_select_one_onecol_txn(
@@ -959,10 +961,12 @@ class RegistrationWorkerStore(CacheInvalidationWorkerStore):
return await self.db_pool.runInteraction("count_users", _count_users)
async def count_real_users(self) -> int:
"""Counts all users without a special user_type registered on the homeserver."""
"""Counts all users without the bot or support user_types registered on the homeserver."""
def _count_users(txn: LoggingTransaction) -> int:
txn.execute("SELECT COUNT(*) FROM users where user_type is null")
txn.execute(
f"SELECT COUNT(*) FROM users WHERE user_type IS NULL OR user_type NOT IN ('{UserTypes.BOT}', '{UserTypes.SUPPORT}')"
)
row = txn.fetchone()
assert row is not None
return row[0]
@@ -2545,7 +2549,8 @@ class RegistrationStore(StatsStore, RegistrationBackgroundUpdateStore):
the user, setting their displayname to the given value
admin: is an admin user?
user_type: type of user. One of the values from api.constants.UserTypes,
or None for a normal user.
a custom value set in the configuration file, or None for a normal
user.
shadow_banned: Whether the user is shadow-banned, i.e. they may be
told their requests succeeded but we ignore them.
approved: Whether to consider the user has already been approved by an

View File

@@ -77,6 +77,8 @@ logger = logging.getLogger(__name__)
@attr.s(slots=True, frozen=True, auto_attribs=True)
class RatelimitOverride:
# n.b. elsewhere in Synapse messages_per_second is represented as a float, but it is
# an integer in the database
messages_per_second: int
burst_count: int

View File

@@ -86,10 +86,10 @@ class TransactionWorkerStore(CacheInvalidationWorkerStore):
@wrap_as_background_process("cleanup_transactions")
async def _cleanup_transactions(self) -> None:
now = self._clock.time_msec()
month_ago = now - 30 * 24 * 60 * 60 * 1000
day_ago = now - 24 * 60 * 60 * 1000
def _cleanup_transactions_txn(txn: LoggingTransaction) -> None:
txn.execute("DELETE FROM received_transactions WHERE ts < ?", (month_ago,))
txn.execute("DELETE FROM received_transactions WHERE ts < ?", (day_ago,))
await self.db_pool.runInteraction(
"_cleanup_transactions", _cleanup_transactions_txn

View File

@@ -43,6 +43,7 @@ try:
USE_ICU = True
except ModuleNotFoundError:
# except ModuleNotFoundError:
USE_ICU = False
from synapse.api.errors import StoreError

View File

@@ -0,0 +1,110 @@
import logging
from enum import Enum
from typing import Optional, Pattern
from matrix_common.regex import glob_to_regex
from synapse.types import JsonMapping, UserID
logger = logging.getLogger(__name__)
class InviteRule(Enum):
"""Enum to define the action taken when an invite matches a rule."""
ALLOW = "allow"
BLOCK = "block"
IGNORE = "ignore"
class InviteRulesConfig:
"""Class to determine if a given user permits an invite from another user, and the action to take."""
def __init__(self, account_data: Optional[JsonMapping]):
self.allowed_users: list[Pattern[str]] = []
self.ignored_users: list[Pattern[str]] = []
self.blocked_users: list[Pattern[str]] = []
self.allowed_servers: list[Pattern[str]] = []
self.ignored_servers: list[Pattern[str]] = []
self.blocked_servers: list[Pattern[str]] = []
def process_field(
values: Optional[list[str]],
ruleset: list[Pattern[str]],
rule: InviteRule,
) -> None:
if isinstance(values, list):
for value in values:
if isinstance(value, str) and len(value) > 0:
# User IDs cannot exceed 255 bytes. Don't process large, potentially
# expensive glob patterns.
if len(value) > 255:
logger.debug(
"Ignoring invite config glob pattern that is >255 bytes: {value}"
)
continue
try:
ruleset.append(glob_to_regex(value))
except Exception as e:
# If for whatever reason we can't process this, just ignore it.
logger.debug(
f"Could not process '{value}' field of invite rule config, ignoring: {e}"
)
if account_data:
process_field(
account_data.get("allowed_users"), self.allowed_users, InviteRule.ALLOW
)
process_field(
account_data.get("ignored_users"), self.ignored_users, InviteRule.IGNORE
)
process_field(
account_data.get("blocked_users"), self.blocked_users, InviteRule.BLOCK
)
process_field(
account_data.get("allowed_servers"),
self.allowed_servers,
InviteRule.ALLOW,
)
process_field(
account_data.get("ignored_servers"),
self.ignored_servers,
InviteRule.IGNORE,
)
process_field(
account_data.get("blocked_servers"),
self.blocked_servers,
InviteRule.BLOCK,
)
def get_invite_rule(self, user_id: str) -> InviteRule:
"""Get the invite rule that matches this user. Will return InviteRule.ALLOW if no rules match
Args:
user_id: The user ID of the inviting user.
"""
user = UserID.from_string(user_id)
# The order here is important. We always process user rules before server rules
# and we always process in the order of Allow, Ignore, Block.
for patterns, rule in [
(self.allowed_users, InviteRule.ALLOW),
(self.ignored_users, InviteRule.IGNORE),
(self.blocked_users, InviteRule.BLOCK),
]:
for regex in patterns:
if regex.match(user_id):
return rule
for patterns, rule in [
(self.allowed_servers, InviteRule.ALLOW),
(self.ignored_servers, InviteRule.IGNORE),
(self.blocked_servers, InviteRule.BLOCK),
]:
for regex in patterns:
if regex.match(user.domain):
return rule
return InviteRule.ALLOW

View File

@@ -0,0 +1,24 @@
# This file is licensed under the Affero General Public License (AGPL) version 3.
#
# Copyright (C) 2025 New Vector, Ltd
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as
# published by the Free Software Foundation, either version 3 of the
# License, or (at your option) any later version.
#
# See the GNU Affero General Public License for more details:
# <https://www.gnu.org/licenses/agpl-3.0.html>.
from typing import Awaitable, Mapping
class HttpClient:
def __init__(self, user_agent: str) -> None: ...
def get(self, url: str, response_limit: int) -> Awaitable[bytes]: ...
def post(
self,
url: str,
response_limit: int,
headers: Mapping[str, str],
request_body: str,
) -> Awaitable[bytes]: ...

View File

@@ -1,6 +1,10 @@
from typing import Optional
from synapse.api.ratelimiting import LimitExceededError, Ratelimiter
from synapse.appservice import ApplicationService
from synapse.config.ratelimiting import RatelimitSettings
from synapse.module_api import RatelimitOverride
from synapse.module_api.callbacks.ratelimit_callbacks import RatelimitModuleApiCallbacks
from synapse.types import create_requester
from tests import unittest
@@ -440,3 +444,49 @@ class TestRatelimiter(unittest.HomeserverTestCase):
limiter.can_do_action(requester=None, key="a", _time_now_s=20.0)
)
self.assertTrue(success)
def test_get_ratelimit_override_for_user_callback(self) -> None:
test_user_id = "@user:test"
test_limiter_name = "name"
callbacks = RatelimitModuleApiCallbacks(self.hs)
requester = create_requester(test_user_id)
limiter = Ratelimiter(
store=self.hs.get_datastores().main,
clock=self.clock,
cfg=RatelimitSettings(
test_limiter_name,
per_second=0.1,
burst_count=3,
),
ratelimit_callbacks=callbacks,
)
# Observe four actions, exceeding the burst_count.
limiter.record_action(requester=requester, n_actions=4, _time_now_s=0.0)
# We should be prevented from taking a new action now.
success, _ = self.get_success_or_raise(
limiter.can_do_action(requester=requester, _time_now_s=0.0)
)
self.assertFalse(success)
# Now register a callback that overrides the ratelimit for this user
# and limiter name.
async def get_ratelimit_override_for_user(
user_id: str, limiter_name: str
) -> Optional[RatelimitOverride]:
if user_id == test_user_id:
return RatelimitOverride(
per_second=0.1,
burst_count=10,
)
return None
callbacks.register_callbacks(
get_ratelimit_override_for_user=get_ratelimit_override_for_user
)
success, _ = self.get_success_or_raise(
limiter.can_do_action(requester=requester, _time_now_s=0.0)
)
self.assertTrue(success)

View File

@@ -2,7 +2,7 @@
# This file is licensed under the Affero General Public License (AGPL) version 3.
#
# Copyright 2015, 2016 OpenMarket Ltd
# Copyright (C) 2023 New Vector, Ltd
# Copyright (C) 2023, 2025 New Vector, Ltd
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as
@@ -234,6 +234,41 @@ class ApplicationServiceSchedulerRecovererTestCase(unittest.TestCase):
self.assertEqual(1, txn.complete.call_count)
self.callback.assert_called_once_with(self.recoverer)
def test_recover_force_retry(self) -> None:
txn = Mock()
txns = [txn, None]
pop_txn = False
def take_txn(
*args: object, **kwargs: object
) -> "defer.Deferred[Optional[Mock]]":
if pop_txn:
return defer.succeed(txns.pop(0))
else:
return defer.succeed(txn)
self.store.get_oldest_unsent_txn = Mock(side_effect=take_txn)
# Start the recovery, and then fail the first attempt.
self.recoverer.recover()
self.assertEqual(0, self.store.get_oldest_unsent_txn.call_count)
txn.send = AsyncMock(return_value=False)
txn.complete = AsyncMock(return_value=None)
self.clock.advance_time(2)
self.assertEqual(1, txn.send.call_count)
self.assertEqual(0, txn.complete.call_count)
self.assertEqual(0, self.callback.call_count)
# Now allow the send to succeed, and force a retry.
pop_txn = True # returns the txn the first time, then no more.
txn.send = AsyncMock(return_value=True) # successfully send the txn
self.recoverer.force_retry()
self.assertEqual(1, txn.send.call_count) # new mock reset call count
self.assertEqual(1, txn.complete.call_count)
# Ensure we call the callback to say we're done!
self.callback.assert_called_once_with(self.recoverer)
# Corresponds to synapse.appservice.scheduler._TransactionController.send
TxnCtrlArgs: TypeAlias = """

View File

@@ -19,9 +19,10 @@
#
#
import json
from http import HTTPStatus
from io import BytesIO
from typing import Any, Dict, Optional, Union
from typing import Any, Dict, Union
from unittest.mock import ANY, AsyncMock, Mock
from urllib.parse import parse_qs
@@ -33,12 +34,11 @@ from signedjson.key import (
from signedjson.sign import sign_json
from twisted.test.proto_helpers import MemoryReactor
from twisted.web.http_headers import Headers
from twisted.web.iweb import IResponse
from synapse.api.errors import (
AuthError,
Codes,
HttpResponseException,
InvalidClientTokenError,
OAuthInsufficientScopeError,
SynapseError,
@@ -52,7 +52,7 @@ from synapse.types import JsonDict, UserID
from synapse.util import Clock
from tests.server import FakeChannel
from tests.test_utils import FakeResponse, get_awaitable_result
from tests.test_utils import get_awaitable_result
from tests.unittest import HomeserverTestCase, override_config, skip_unless
from tests.utils import HAS_AUTHLIB, checked_cast, mock_getRawHeaders
@@ -145,6 +145,9 @@ class MSC3861OAuthDelegation(HomeserverTestCase):
self.auth = checked_cast(MSC3861DelegatedAuth, hs.get_auth())
self._rust_client = Mock(spec=["post"])
self.auth._rust_http_client = self._rust_client
return hs
def prepare(
@@ -157,9 +160,15 @@ class MSC3861OAuthDelegation(HomeserverTestCase):
store.store_device(USER_ID, DEVICE, initial_device_display_name=None)
)
def _set_introspection_returnvalue(self, response_value: Any) -> AsyncMock:
self._rust_client.post = mock = AsyncMock(
return_value=json.dumps(response_value).encode("utf-8")
)
return mock
def _assertParams(self) -> None:
"""Assert that the request parameters are correct."""
params = parse_qs(self.http_client.request.call_args[1]["data"].decode("utf-8"))
params = parse_qs(self._rust_client.post.call_args[1]["request_body"])
self.assertEqual(params["token"], ["mockAccessToken"])
self.assertEqual(params["client_id"], [CLIENT_ID])
self.assertEqual(params["client_secret"], [CLIENT_SECRET])
@@ -167,128 +176,125 @@ class MSC3861OAuthDelegation(HomeserverTestCase):
def test_inactive_token(self) -> None:
"""The handler should return a 403 where the token is inactive."""
self.http_client.request = AsyncMock(
return_value=FakeResponse.json(
code=200,
payload={"active": False},
)
)
self._set_introspection_returnvalue({"active": False})
request = Mock(args={})
request.args[b"access_token"] = [b"mockAccessToken"]
request.requestHeaders.getRawHeaders = mock_getRawHeaders()
self.get_failure(self.auth.get_user_by_req(request), InvalidClientTokenError)
self.http_client.get_json.assert_called_once_with(WELL_KNOWN)
self.http_client.request.assert_called_once_with(
method="POST", uri=INTROSPECTION_ENDPOINT, data=ANY, headers=ANY
self._rust_client.post.assert_called_once_with(
url=INTROSPECTION_ENDPOINT,
response_limit=ANY,
request_body=ANY,
headers=ANY,
)
self._assertParams()
def test_active_no_scope(self) -> None:
"""The handler should return a 403 where no scope is given."""
self.http_client.request = AsyncMock(
return_value=FakeResponse.json(
code=200,
payload={"active": True},
)
)
self._set_introspection_returnvalue({"active": True})
request = Mock(args={})
request.args[b"access_token"] = [b"mockAccessToken"]
request.requestHeaders.getRawHeaders = mock_getRawHeaders()
self.get_failure(self.auth.get_user_by_req(request), InvalidClientTokenError)
self.http_client.get_json.assert_called_once_with(WELL_KNOWN)
self.http_client.request.assert_called_once_with(
method="POST", uri=INTROSPECTION_ENDPOINT, data=ANY, headers=ANY
self._rust_client.post.assert_called_once_with(
url=INTROSPECTION_ENDPOINT,
response_limit=ANY,
request_body=ANY,
headers=ANY,
)
self._assertParams()
def test_active_user_no_subject(self) -> None:
"""The handler should return a 500 when no subject is present."""
self.http_client.request = AsyncMock(
return_value=FakeResponse.json(
code=200,
payload={"active": True, "scope": " ".join([MATRIX_USER_SCOPE])},
)
self._set_introspection_returnvalue(
{"active": True, "scope": " ".join([MATRIX_USER_SCOPE])}
)
request = Mock(args={})
request.args[b"access_token"] = [b"mockAccessToken"]
request.requestHeaders.getRawHeaders = mock_getRawHeaders()
self.get_failure(self.auth.get_user_by_req(request), InvalidClientTokenError)
self.http_client.get_json.assert_called_once_with(WELL_KNOWN)
self.http_client.request.assert_called_once_with(
method="POST", uri=INTROSPECTION_ENDPOINT, data=ANY, headers=ANY
self._rust_client.post.assert_called_once_with(
url=INTROSPECTION_ENDPOINT,
response_limit=ANY,
request_body=ANY,
headers=ANY,
)
self._assertParams()
def test_active_no_user_scope(self) -> None:
"""The handler should return a 500 when no subject is present."""
self.http_client.request = AsyncMock(
return_value=FakeResponse.json(
code=200,
payload={
"active": True,
"sub": SUBJECT,
"scope": " ".join([MATRIX_DEVICE_SCOPE]),
},
)
self._set_introspection_returnvalue(
{
"active": True,
"sub": SUBJECT,
"scope": " ".join([MATRIX_DEVICE_SCOPE]),
}
)
request = Mock(args={})
request.args[b"access_token"] = [b"mockAccessToken"]
request.requestHeaders.getRawHeaders = mock_getRawHeaders()
self.get_failure(self.auth.get_user_by_req(request), InvalidClientTokenError)
self.http_client.get_json.assert_called_once_with(WELL_KNOWN)
self.http_client.request.assert_called_once_with(
method="POST", uri=INTROSPECTION_ENDPOINT, data=ANY, headers=ANY
self._rust_client.post.assert_called_once_with(
url=INTROSPECTION_ENDPOINT,
response_limit=ANY,
request_body=ANY,
headers=ANY,
)
self._assertParams()
def test_active_admin_not_user(self) -> None:
"""The handler should raise when the scope has admin right but not user."""
self.http_client.request = AsyncMock(
return_value=FakeResponse.json(
code=200,
payload={
"active": True,
"sub": SUBJECT,
"scope": " ".join([SYNAPSE_ADMIN_SCOPE]),
"username": USERNAME,
},
)
self._set_introspection_returnvalue(
{
"active": True,
"sub": SUBJECT,
"scope": " ".join([SYNAPSE_ADMIN_SCOPE]),
"username": USERNAME,
}
)
request = Mock(args={})
request.args[b"access_token"] = [b"mockAccessToken"]
request.requestHeaders.getRawHeaders = mock_getRawHeaders()
self.get_failure(self.auth.get_user_by_req(request), InvalidClientTokenError)
self.http_client.get_json.assert_called_once_with(WELL_KNOWN)
self.http_client.request.assert_called_once_with(
method="POST", uri=INTROSPECTION_ENDPOINT, data=ANY, headers=ANY
self._rust_client.post.assert_called_once_with(
url=INTROSPECTION_ENDPOINT,
response_limit=ANY,
request_body=ANY,
headers=ANY,
)
self._assertParams()
def test_active_admin(self) -> None:
"""The handler should return a requester with admin rights."""
self.http_client.request = AsyncMock(
return_value=FakeResponse.json(
code=200,
payload={
"active": True,
"sub": SUBJECT,
"scope": " ".join([SYNAPSE_ADMIN_SCOPE, MATRIX_USER_SCOPE]),
"username": USERNAME,
},
)
self._set_introspection_returnvalue(
{
"active": True,
"sub": SUBJECT,
"scope": " ".join([SYNAPSE_ADMIN_SCOPE, MATRIX_USER_SCOPE]),
"username": USERNAME,
}
)
request = Mock(args={})
request.args[b"access_token"] = [b"mockAccessToken"]
request.requestHeaders.getRawHeaders = mock_getRawHeaders()
requester = self.get_success(self.auth.get_user_by_req(request))
self.http_client.get_json.assert_called_once_with(WELL_KNOWN)
self.http_client.request.assert_called_once_with(
method="POST", uri=INTROSPECTION_ENDPOINT, data=ANY, headers=ANY
self._rust_client.post.assert_called_once_with(
url=INTROSPECTION_ENDPOINT,
response_limit=ANY,
request_body=ANY,
headers=ANY,
)
self._assertParams()
self.assertEqual(requester.user.to_string(), "@%s:%s" % (USERNAME, SERVER_NAME))
@@ -301,26 +307,26 @@ class MSC3861OAuthDelegation(HomeserverTestCase):
def test_active_admin_highest_privilege(self) -> None:
"""The handler should resolve to the most permissive scope."""
self.http_client.request = AsyncMock(
return_value=FakeResponse.json(
code=200,
payload={
"active": True,
"sub": SUBJECT,
"scope": " ".join(
[SYNAPSE_ADMIN_SCOPE, MATRIX_USER_SCOPE, MATRIX_GUEST_SCOPE]
),
"username": USERNAME,
},
)
self._set_introspection_returnvalue(
{
"active": True,
"sub": SUBJECT,
"scope": " ".join(
[SYNAPSE_ADMIN_SCOPE, MATRIX_USER_SCOPE, MATRIX_GUEST_SCOPE]
),
"username": USERNAME,
}
)
request = Mock(args={})
request.args[b"access_token"] = [b"mockAccessToken"]
request.requestHeaders.getRawHeaders = mock_getRawHeaders()
requester = self.get_success(self.auth.get_user_by_req(request))
self.http_client.get_json.assert_called_once_with(WELL_KNOWN)
self.http_client.request.assert_called_once_with(
method="POST", uri=INTROSPECTION_ENDPOINT, data=ANY, headers=ANY
self._rust_client.post.assert_called_once_with(
url=INTROSPECTION_ENDPOINT,
response_limit=ANY,
request_body=ANY,
headers=ANY,
)
self._assertParams()
self.assertEqual(requester.user.to_string(), "@%s:%s" % (USERNAME, SERVER_NAME))
@@ -333,24 +339,24 @@ class MSC3861OAuthDelegation(HomeserverTestCase):
def test_active_user(self) -> None:
"""The handler should return a requester with normal user rights."""
self.http_client.request = AsyncMock(
return_value=FakeResponse.json(
code=200,
payload={
"active": True,
"sub": SUBJECT,
"scope": " ".join([MATRIX_USER_SCOPE]),
"username": USERNAME,
},
)
self._set_introspection_returnvalue(
{
"active": True,
"sub": SUBJECT,
"scope": " ".join([MATRIX_USER_SCOPE]),
"username": USERNAME,
}
)
request = Mock(args={})
request.args[b"access_token"] = [b"mockAccessToken"]
request.requestHeaders.getRawHeaders = mock_getRawHeaders()
requester = self.get_success(self.auth.get_user_by_req(request))
self.http_client.get_json.assert_called_once_with(WELL_KNOWN)
self.http_client.request.assert_called_once_with(
method="POST", uri=INTROSPECTION_ENDPOINT, data=ANY, headers=ANY
self._rust_client.post.assert_called_once_with(
url=INTROSPECTION_ENDPOINT,
response_limit=ANY,
request_body=ANY,
headers=ANY,
)
self._assertParams()
self.assertEqual(requester.user.to_string(), "@%s:%s" % (USERNAME, SERVER_NAME))
@@ -363,24 +369,24 @@ class MSC3861OAuthDelegation(HomeserverTestCase):
def test_active_user_with_device(self) -> None:
"""The handler should return a requester with normal user rights and a device ID."""
self.http_client.request = AsyncMock(
return_value=FakeResponse.json(
code=200,
payload={
"active": True,
"sub": SUBJECT,
"scope": " ".join([MATRIX_USER_SCOPE, MATRIX_DEVICE_SCOPE]),
"username": USERNAME,
},
)
self._set_introspection_returnvalue(
{
"active": True,
"sub": SUBJECT,
"scope": " ".join([MATRIX_USER_SCOPE, MATRIX_DEVICE_SCOPE]),
"username": USERNAME,
}
)
request = Mock(args={})
request.args[b"access_token"] = [b"mockAccessToken"]
request.requestHeaders.getRawHeaders = mock_getRawHeaders()
requester = self.get_success(self.auth.get_user_by_req(request))
self.http_client.get_json.assert_called_once_with(WELL_KNOWN)
self.http_client.request.assert_called_once_with(
method="POST", uri=INTROSPECTION_ENDPOINT, data=ANY, headers=ANY
self._rust_client.post.assert_called_once_with(
url=INTROSPECTION_ENDPOINT,
response_limit=ANY,
request_body=ANY,
headers=ANY,
)
self._assertParams()
self.assertEqual(requester.user.to_string(), "@%s:%s" % (USERNAME, SERVER_NAME))
@@ -393,32 +399,32 @@ class MSC3861OAuthDelegation(HomeserverTestCase):
def test_active_user_with_device_explicit_device_id(self) -> None:
"""The handler should return a requester with normal user rights and a device ID, given explicitly, as supported by MAS 0.15+"""
self.http_client.request = AsyncMock(
return_value=FakeResponse.json(
code=200,
payload={
"active": True,
"sub": SUBJECT,
"scope": " ".join([MATRIX_USER_SCOPE]),
"device_id": DEVICE,
"username": USERNAME,
},
)
self._set_introspection_returnvalue(
{
"active": True,
"sub": SUBJECT,
"scope": " ".join([MATRIX_USER_SCOPE]),
"device_id": DEVICE,
"username": USERNAME,
}
)
request = Mock(args={})
request.args[b"access_token"] = [b"mockAccessToken"]
request.requestHeaders.getRawHeaders = mock_getRawHeaders()
requester = self.get_success(self.auth.get_user_by_req(request))
self.http_client.get_json.assert_called_once_with(WELL_KNOWN)
self.http_client.request.assert_called_once_with(
method="POST", uri=INTROSPECTION_ENDPOINT, data=ANY, headers=ANY
self._rust_client.post.assert_called_once_with(
url=INTROSPECTION_ENDPOINT,
response_limit=ANY,
request_body=ANY,
headers=ANY,
)
# It should have called with the 'X-MAS-Supports-Device-Id: 1' header
self.assertEqual(
self.http_client.request.call_args[1]["headers"].getRawHeaders(
b"X-MAS-Supports-Device-Id",
self._rust_client.post.call_args[1]["headers"].get(
"X-MAS-Supports-Device-Id",
),
[b"1"],
"1",
)
self._assertParams()
self.assertEqual(requester.user.to_string(), "@%s:%s" % (USERNAME, SERVER_NAME))
@@ -431,22 +437,19 @@ class MSC3861OAuthDelegation(HomeserverTestCase):
def test_multiple_devices(self) -> None:
"""The handler should raise an error if multiple devices are found in the scope."""
self.http_client.request = AsyncMock(
return_value=FakeResponse.json(
code=200,
payload={
"active": True,
"sub": SUBJECT,
"scope": " ".join(
[
MATRIX_USER_SCOPE,
f"{MATRIX_DEVICE_SCOPE_PREFIX}AABBCC",
f"{MATRIX_DEVICE_SCOPE_PREFIX}DDEEFF",
]
),
"username": USERNAME,
},
)
self._set_introspection_returnvalue(
{
"active": True,
"sub": SUBJECT,
"scope": " ".join(
[
MATRIX_USER_SCOPE,
f"{MATRIX_DEVICE_SCOPE_PREFIX}AABBCC",
f"{MATRIX_DEVICE_SCOPE_PREFIX}DDEEFF",
]
),
"username": USERNAME,
}
)
request = Mock(args={})
request.args[b"access_token"] = [b"mockAccessToken"]
@@ -456,16 +459,13 @@ class MSC3861OAuthDelegation(HomeserverTestCase):
def test_active_guest_not_allowed(self) -> None:
"""The handler should return an insufficient scope error."""
self.http_client.request = AsyncMock(
return_value=FakeResponse.json(
code=200,
payload={
"active": True,
"sub": SUBJECT,
"scope": " ".join([MATRIX_GUEST_SCOPE, MATRIX_DEVICE_SCOPE]),
"username": USERNAME,
},
)
self._set_introspection_returnvalue(
{
"active": True,
"sub": SUBJECT,
"scope": " ".join([MATRIX_GUEST_SCOPE, MATRIX_DEVICE_SCOPE]),
"username": USERNAME,
}
)
request = Mock(args={})
request.args[b"access_token"] = [b"mockAccessToken"]
@@ -474,8 +474,11 @@ class MSC3861OAuthDelegation(HomeserverTestCase):
self.auth.get_user_by_req(request), OAuthInsufficientScopeError
)
self.http_client.get_json.assert_called_once_with(WELL_KNOWN)
self.http_client.request.assert_called_once_with(
method="POST", uri=INTROSPECTION_ENDPOINT, data=ANY, headers=ANY
self._rust_client.post.assert_called_once_with(
url=INTROSPECTION_ENDPOINT,
response_limit=ANY,
request_body=ANY,
headers=ANY,
)
self._assertParams()
self.assertEqual(
@@ -486,16 +489,13 @@ class MSC3861OAuthDelegation(HomeserverTestCase):
def test_active_guest_allowed(self) -> None:
"""The handler should return a requester with guest user rights and a device ID."""
self.http_client.request = AsyncMock(
return_value=FakeResponse.json(
code=200,
payload={
"active": True,
"sub": SUBJECT,
"scope": " ".join([MATRIX_GUEST_SCOPE, MATRIX_DEVICE_SCOPE]),
"username": USERNAME,
},
)
self._set_introspection_returnvalue(
{
"active": True,
"sub": SUBJECT,
"scope": " ".join([MATRIX_GUEST_SCOPE, MATRIX_DEVICE_SCOPE]),
"username": USERNAME,
}
)
request = Mock(args={})
request.args[b"access_token"] = [b"mockAccessToken"]
@@ -504,8 +504,11 @@ class MSC3861OAuthDelegation(HomeserverTestCase):
self.auth.get_user_by_req(request, allow_guest=True)
)
self.http_client.get_json.assert_called_once_with(WELL_KNOWN)
self.http_client.request.assert_called_once_with(
method="POST", uri=INTROSPECTION_ENDPOINT, data=ANY, headers=ANY
self._rust_client.post.assert_called_once_with(
url=INTROSPECTION_ENDPOINT,
response_limit=ANY,
request_body=ANY,
headers=ANY,
)
self._assertParams()
self.assertEqual(requester.user.to_string(), "@%s:%s" % (USERNAME, SERVER_NAME))
@@ -522,30 +525,28 @@ class MSC3861OAuthDelegation(HomeserverTestCase):
request.requestHeaders.getRawHeaders = mock_getRawHeaders()
# The introspection endpoint is returning an error.
self.http_client.request = AsyncMock(
return_value=FakeResponse(code=500, body=b"Internal Server Error")
)
error = self.get_failure(self.auth.get_user_by_req(request), SynapseError)
self.assertEqual(error.value.code, 503)
# The introspection endpoint request fails.
self.http_client.request = AsyncMock(side_effect=Exception())
error = self.get_failure(self.auth.get_user_by_req(request), SynapseError)
self.assertEqual(error.value.code, 503)
# The introspection endpoint does not return a JSON object.
self.http_client.request = AsyncMock(
return_value=FakeResponse.json(
code=200, payload=["this is an array", "not an object"]
self._rust_client.post = AsyncMock(
side_effect=HttpResponseException(
code=500, msg="Internal Server Error", response=b"{}"
)
)
error = self.get_failure(self.auth.get_user_by_req(request), SynapseError)
self.assertEqual(error.value.code, 503)
# The introspection endpoint request fails.
self._rust_client.post = AsyncMock(side_effect=Exception())
error = self.get_failure(self.auth.get_user_by_req(request), SynapseError)
self.assertEqual(error.value.code, 503)
# The introspection endpoint does not return a JSON object.
self._set_introspection_returnvalue(["this is an array", "not an object"])
error = self.get_failure(self.auth.get_user_by_req(request), SynapseError)
self.assertEqual(error.value.code, 503)
# The introspection endpoint does not return valid JSON.
self.http_client.request = AsyncMock(
return_value=FakeResponse(code=200, body=b"this is not valid JSON")
)
self._set_introspection_returnvalue("this is not valid JSON")
error = self.get_failure(self.auth.get_user_by_req(request), SynapseError)
self.assertEqual(error.value.code, 503)
@@ -554,23 +555,21 @@ class MSC3861OAuthDelegation(HomeserverTestCase):
an expiry time, the introspection response is cached and then the entry is
re-requested after it has expired."""
self.http_client.request = introspection_mock = AsyncMock(
return_value=FakeResponse.json(
code=200,
payload={
"active": True,
"sub": SUBJECT,
"scope": " ".join(
[
MATRIX_USER_SCOPE,
f"{MATRIX_DEVICE_SCOPE_PREFIX}AABBCC",
]
),
"username": USERNAME,
"expires_in": 60,
},
)
introspection_mock = self._set_introspection_returnvalue(
{
"active": True,
"sub": SUBJECT,
"scope": " ".join(
[
MATRIX_USER_SCOPE,
f"{MATRIX_DEVICE_SCOPE_PREFIX}AABBCC",
]
),
"username": USERNAME,
"expires_in": 60,
}
)
request = Mock(args={})
request.args[b"access_token"] = [b"mockAccessToken"]
request.requestHeaders.getRawHeaders = mock_getRawHeaders()
@@ -607,16 +606,13 @@ class MSC3861OAuthDelegation(HomeserverTestCase):
def test_cross_signing(self) -> None:
"""Try uploading device keys with OAuth delegation enabled."""
self.http_client.request = AsyncMock(
return_value=FakeResponse.json(
code=200,
payload={
"active": True,
"sub": SUBJECT,
"scope": " ".join([MATRIX_USER_SCOPE, MATRIX_DEVICE_SCOPE]),
"username": USERNAME,
},
)
self._set_introspection_returnvalue(
{
"active": True,
"sub": SUBJECT,
"scope": " ".join([MATRIX_USER_SCOPE, MATRIX_DEVICE_SCOPE]),
"username": USERNAME,
}
)
keys_upload_body = self.make_device_keys(USER_ID, DEVICE)
channel = self.make_request(
@@ -778,16 +774,13 @@ class MSC3861OAuthDelegation(HomeserverTestCase):
# Because we still support those endpoints with ASes, it checks the
# access token before returning 404
self.http_client.request = AsyncMock(
return_value=FakeResponse.json(
code=200,
payload={
"active": True,
"sub": SUBJECT,
"scope": " ".join([MATRIX_USER_SCOPE, MATRIX_DEVICE_SCOPE]),
"username": USERNAME,
},
)
self._set_introspection_returnvalue(
{
"active": True,
"sub": SUBJECT,
"scope": " ".join([MATRIX_USER_SCOPE, MATRIX_DEVICE_SCOPE]),
"username": USERNAME,
},
)
self.expect_unrecognized("POST", "/_matrix/client/v3/delete_devices", auth=True)
@@ -820,9 +813,7 @@ class MSC3861OAuthDelegation(HomeserverTestCase):
def test_admin_token(self) -> None:
"""The handler should return a requester with admin rights when admin_token is used."""
self.http_client.request = AsyncMock(
return_value=FakeResponse.json(code=200, payload={"active": False}),
)
self._set_introspection_returnvalue({"active": False})
request = Mock(args={})
request.args[b"access_token"] = [b"admin_token_value"]
@@ -839,7 +830,7 @@ class MSC3861OAuthDelegation(HomeserverTestCase):
)
# There should be no call to the introspection endpoint
self.http_client.request.assert_not_called()
self._rust_client.post.assert_not_called()
@override_config({"mau_stats_only": True})
def test_request_tracking(self) -> None:
@@ -852,28 +843,23 @@ class MSC3861OAuthDelegation(HomeserverTestCase):
known_token = "token-token-GOOD-:)"
async def mock_http_client_request(
method: str,
uri: str,
data: Optional[bytes] = None,
headers: Optional[Headers] = None,
) -> IResponse:
url: str, request_body: str, **kwargs: Any
) -> bytes:
"""Mocked auth provider response."""
assert method == "POST"
token = parse_qs(data)[b"token"][0].decode("utf-8")
token = parse_qs(request_body)["token"][0]
if token == known_token:
return FakeResponse.json(
code=200,
payload={
return json.dumps(
{
"active": True,
"scope": MATRIX_USER_SCOPE,
"sub": SUBJECT,
"username": USERNAME,
},
)
).encode("utf-8")
return FakeResponse.json(code=200, payload={"active": False})
return json.dumps({"active": False}).encode("utf-8")
self.http_client.request = mock_http_client_request
self._rust_client.post = mock_http_client_request
EXAMPLE_IPV4_ADDR = "123.123.123.123"
EXAMPLE_USER_AGENT = "httprettygood"

View File

@@ -738,6 +738,41 @@ class RegistrationTestCase(unittest.HomeserverTestCase):
self.handler.register_user(localpart="bobflimflob", auth_provider_id="saml")
)
def test_register_default_user_type(self) -> None:
"""Test that the default user type is none when registering a user."""
user_id = self.get_success(self.handler.register_user(localpart="user"))
user_info = self.get_success(self.store.get_user_by_id(user_id))
assert user_info is not None
self.assertEqual(user_info.user_type, None)
def test_register_extra_user_types_valid(self) -> None:
"""
Test that the specified user type is set correctly when registering a user.
n.b. No validation is done on the user type, so this test
is only to ensure that the user type can be set to any value.
"""
user_id = self.get_success(
self.handler.register_user(localpart="user", user_type="anyvalue")
)
user_info = self.get_success(self.store.get_user_by_id(user_id))
assert user_info is not None
self.assertEqual(user_info.user_type, "anyvalue")
@override_config(
{
"user_types": {
"extra_user_types": ["extra1", "extra2"],
"default_user_type": "extra1",
}
}
)
def test_register_extra_user_types_with_default(self) -> None:
"""Test that the default_user_type in config is set correctly when registering a user."""
user_id = self.get_success(self.handler.register_user(localpart="user"))
user_info = self.get_success(self.store.get_user_by_id(user_id))
assert user_info is not None
self.assertEqual(user_info.user_type, "extra1")
async def get_or_create_user(
self,
requester: Requester,

View File

@@ -5,10 +5,13 @@ from twisted.test.proto_helpers import MemoryReactor
import synapse.rest.admin
import synapse.rest.client.login
import synapse.rest.client.room
from synapse.api.constants import EventTypes, Membership
from synapse.api.constants import AccountDataTypes, EventTypes, Membership
from synapse.api.errors import Codes, LimitExceededError, SynapseError
from synapse.crypto.event_signing import add_hashes_and_signatures
from synapse.events import FrozenEventV3
from synapse.federation.federation_base import (
event_from_pdu_json,
)
from synapse.federation.federation_client import SendJoinResult
from synapse.server import HomeServer
from synapse.types import UserID, create_requester
@@ -453,3 +456,165 @@ class RoomMemberMasterHandlerTestCase(HomeserverTestCase):
new_count = rows[0][0]
self.assertEqual(initial_count, new_count)
class TestInviteFiltering(FederatingHomeserverTestCase):
servlets = [
synapse.rest.admin.register_servlets,
synapse.rest.client.login.register_servlets,
synapse.rest.client.room.register_servlets,
]
def prepare(self, reactor: MemoryReactor, clock: Clock, hs: HomeServer) -> None:
self.handler = hs.get_room_member_handler()
self.fed_handler = hs.get_federation_handler()
self.store = hs.get_datastores().main
# Create three users.
self.alice = self.register_user("alice", "pass")
self.alice_token = self.login("alice", "pass")
self.bob = self.register_user("bob", "pass")
self.bob_token = self.login("bob", "pass")
@override_config({"experimental_features": {"msc4155_enabled": True}})
def test_misc4155_block_invite_local(self) -> None:
"""Test that MSC4155 will block a user from being invited to a room"""
room_id = self.helper.create_room_as(self.alice, tok=self.alice_token)
self.get_success(
self.store.add_account_data_for_user(
self.bob,
AccountDataTypes.MSC4155_INVITE_PERMISSION_CONFIG,
{
"blocked_users": [self.alice],
},
)
)
f = self.get_failure(
self.handler.update_membership(
requester=create_requester(self.alice),
target=UserID.from_string(self.bob),
room_id=room_id,
action=Membership.INVITE,
),
SynapseError,
).value
self.assertEqual(f.code, 403)
self.assertEqual(f.errcode, "ORG.MATRIX.MSC4155.M_INVITE_BLOCKED")
@override_config({"experimental_features": {"msc4155_enabled": False}})
def test_msc4155_disabled_allow_invite_local(self) -> None:
"""Test that MSC4155 will block a user from being invited to a room"""
room_id = self.helper.create_room_as(self.alice, tok=self.alice_token)
self.get_success(
self.store.add_account_data_for_user(
self.bob,
AccountDataTypes.MSC4155_INVITE_PERMISSION_CONFIG,
{
"blocked_users": [self.alice],
},
)
)
self.get_success(
self.handler.update_membership(
requester=create_requester(self.alice),
target=UserID.from_string(self.bob),
room_id=room_id,
action=Membership.INVITE,
),
)
@override_config({"experimental_features": {"msc4155_enabled": True}})
def test_msc4155_block_invite_remote(self) -> None:
"""Test that MSC4155 will block a remote user from being invited to a room"""
# A remote user who sends the invite
remote_server = "otherserver"
remote_user = "@otheruser:" + remote_server
self.get_success(
self.store.add_account_data_for_user(
self.bob,
AccountDataTypes.MSC4155_INVITE_PERMISSION_CONFIG,
{"blocked_users": [remote_user]},
)
)
room_id = self.helper.create_room_as(
room_creator=self.alice, tok=self.alice_token
)
room_version = self.get_success(self.store.get_room_version(room_id))
invite_event = event_from_pdu_json(
{
"type": EventTypes.Member,
"content": {"membership": "invite"},
"room_id": room_id,
"sender": remote_user,
"state_key": self.bob,
"depth": 32,
"prev_events": [],
"auth_events": [],
"origin_server_ts": self.clock.time_msec(),
},
room_version,
)
f = self.get_failure(
self.fed_handler.on_invite_request(
remote_server,
invite_event,
invite_event.room_version,
),
SynapseError,
).value
self.assertEqual(f.code, 403)
self.assertEqual(f.errcode, "ORG.MATRIX.MSC4155.M_INVITE_BLOCKED")
@override_config({"experimental_features": {"msc4155_enabled": True}})
def test_msc4155_block_invite_remote_server(self) -> None:
"""Test that MSC4155 will block a remote server's user from being invited to a room"""
# A remote user who sends the invite
remote_server = "otherserver"
remote_user = "@otheruser:" + remote_server
self.get_success(
self.store.add_account_data_for_user(
self.bob,
AccountDataTypes.MSC4155_INVITE_PERMISSION_CONFIG,
{"blocked_servers": [remote_server]},
)
)
room_id = self.helper.create_room_as(
room_creator=self.alice, tok=self.alice_token
)
room_version = self.get_success(self.store.get_room_version(room_id))
invite_event = event_from_pdu_json(
{
"type": EventTypes.Member,
"content": {"membership": "invite"},
"room_id": room_id,
"sender": remote_user,
"state_key": self.bob,
"depth": 32,
"prev_events": [],
"auth_events": [],
"origin_server_ts": self.clock.time_msec(),
},
room_version,
)
f = self.get_failure(
self.fed_handler.on_invite_request(
remote_server,
invite_event,
invite_event.room_version,
),
SynapseError,
).value
self.assertEqual(f.code, 403)
self.assertEqual(f.errcode, "ORG.MATRIX.MSC4155.M_INVITE_BLOCKED")

View File

@@ -1360,3 +1360,42 @@ class MediaHashesTestCase(unittest.HomeserverTestCase):
store_media.sha256,
SMALL_PNG_SHA256,
)
class MediaRepoSizeModuleCallbackTestCase(unittest.HomeserverTestCase):
servlets = [
login.register_servlets,
admin.register_servlets,
]
def prepare(self, reactor: MemoryReactor, clock: Clock, hs: HomeServer) -> None:
self.user = self.register_user("user", "pass")
self.tok = self.login("user", "pass")
self.mock_result = True # Allow all uploads by default
hs.get_module_api().register_media_repository_callbacks(
is_user_allowed_to_upload_media_of_size=self.is_user_allowed_to_upload_media_of_size,
)
def create_resource_dict(self) -> Dict[str, Resource]:
resources = super().create_resource_dict()
resources["/_matrix/media"] = self.hs.get_media_repository_resource()
return resources
async def is_user_allowed_to_upload_media_of_size(
self, user_id: str, size: int
) -> bool:
self.last_user_id = user_id
self.last_size = size
return self.mock_result
def test_upload_allowed(self) -> None:
self.helper.upload_media(SMALL_PNG, tok=self.tok, expect_code=200)
assert self.last_user_id == self.user
assert self.last_size == len(SMALL_PNG)
def test_upload_not_allowed(self) -> None:
self.mock_result = False
self.helper.upload_media(SMALL_PNG, tok=self.tok, expect_code=413)
assert self.last_user_id == self.user
assert self.last_size == len(SMALL_PNG)

Some files were not shown because too many files have changed in this diff Show More