Merge remote-tracking branch 'origin/develop' into erikj/bg_update_co…

…ntroller
matrix-org · Nov 16, 2021 · dddfdca · dddfdca
2 parents c77bad8 + dfa5364
commit dddfdca
Show file tree

Hide file tree

Showing 179 changed files with 2,581 additions and 861 deletions.
diff --git a/CHANGES.md b/CHANGES.md
@@ -1,3 +1,97 @@
+Synapse 1.47.0rc2 (2021-11-10)
+==============================
+
+This fixes an issue with publishing the Debian packages for 1.47.0rc1.
+It is otherwise identical to 1.47.0rc1.
+
+
+Synapse 1.47.0rc1 (2021-11-09)
+==============================
+
+Deprecations and Removals
+-------------------------
+
+- The `user_may_create_room_with_invites` module callback is now deprecated. Please refer to the [upgrade notes](https://matrix-org.github.io/synapse/develop/upgrade#upgrading-to-v1470) for more information. ([\#11206](https://github.com/matrix-org/synapse/issues/11206))
+- Remove deprecated admin API to delete rooms (`POST /_synapse/admin/v1/rooms/<room_id>/delete`). ([\#11213](https://github.com/matrix-org/synapse/issues/11213))
+
+
+Features
+--------
+
+- Advertise support for Client-Server API r0.6.1. ([\#11097](https://github.com/matrix-org/synapse/issues/11097))
+- Add search by room ID and room alias to the List Room admin API. ([\#11099](https://github.com/matrix-org/synapse/issues/11099))
+- Add an `on_new_event` third-party rules callback to allow Synapse modules to act after an event has been sent into a room. ([\#11126](https://github.com/matrix-org/synapse/issues/11126))
+- Add a module API method to update a user's membership in a room. ([\#11147](https://github.com/matrix-org/synapse/issues/11147))
+- Add metrics for thread pool usage. ([\#11178](https://github.com/matrix-org/synapse/issues/11178))
+- Support the stable room type field for [MSC3288](https://github.com/matrix-org/matrix-doc/pull/3288). ([\#11187](https://github.com/matrix-org/synapse/issues/11187))
+- Add a module API method to retrieve the current state of a room. ([\#11204](https://github.com/matrix-org/synapse/issues/11204))
+- Calculate a default value for `public_baseurl` based on `server_name`. ([\#11210](https://github.com/matrix-org/synapse/issues/11210))
+- Add support for serving `/.well-known/matrix/server` files, to redirect federation traffic to port 443. ([\#11211](https://github.com/matrix-org/synapse/issues/11211))
+- Add admin APIs to pause, start and check the status of background updates. ([\#11263](https://github.com/matrix-org/synapse/issues/11263))
+
+
+Bugfixes
+--------
+
+- Fix a long-standing bug which allowed hidden devices to receive to-device messages, resulting in unnecessary database bloat. ([\#10097](https://github.com/matrix-org/synapse/issues/10097))
+- Fix a long-standing bug where messages in the `device_inbox` table for deleted devices would persist indefinitely. Contributed by @dklimpel and @JohannesKleine. ([\#10969](https://github.com/matrix-org/synapse/issues/10969), [\#11212](https://github.com/matrix-org/synapse/issues/11212))
+- Do not accept events if a third-party rule `check_event_allowed` callback raises an exception. ([\#11033](https://github.com/matrix-org/synapse/issues/11033))
+- Fix long-standing bug where verification requests could fail in certain cases if a federation whitelist was in place but did not include your own homeserver. ([\#11129](https://github.com/matrix-org/synapse/issues/11129))
+- Allow an empty list of `state_events_at_start` to be sent when using the [MSC2716](https://github.com/matrix-org/matrix-doc/pull/2716) `/batch_send` endpoint and the author of the historical messages is already part of the current room state at the given `?prev_event_id`. ([\#11188](https://github.com/matrix-org/synapse/issues/11188))
+- Fix a bug introduced in Synapse 1.45.0 which prevented the `synapse_review_recent_signups` script from running. Contributed by @samuel-p. ([\#11191](https://github.com/matrix-org/synapse/issues/11191))
+- Delete `to_device` messages for hidden devices that will never be read, reducing database size. ([\#11199](https://github.com/matrix-org/synapse/issues/11199))
+- Fix a long-standing bug wherein a missing `Content-Type` header when downloading remote media would cause Synapse to throw an error. ([\#11200](https://github.com/matrix-org/synapse/issues/11200))
+- Fix a long-standing bug which could result in serialization errors and potentially duplicate transaction data when sending ephemeral events to application services. Contributed by @Fizzadar at Beeper. ([\#11207](https://github.com/matrix-org/synapse/issues/11207))
+- Fix a bug introduced in Synapse 1.35.0 which made it impossible to join rooms that return a `send_join` response containing floats. ([\#11217](https://github.com/matrix-org/synapse/issues/11217))
+- Fix long-standing bug where cross signing keys were not included in the response to `/r0/keys/query` the first time a remote user was queried. ([\#11234](https://github.com/matrix-org/synapse/issues/11234))
+- Fix a long-standing bug where all requests that read events from the database could get stuck as a result of losing the database connection. ([\#11240](https://github.com/matrix-org/synapse/issues/11240))
+- Fix a bug preventing Synapse from being rolled back to an earlier version when using workers. ([\#11255](https://github.com/matrix-org/synapse/issues/11255), [\#11276](https://github.com/matrix-org/synapse/issues/11276))
+- Fix a bug introduced in Synapse 1.37.1 which caused a remote event being processed by a worker to not get processed on restart if the worker was killed. ([\#11262](https://github.com/matrix-org/synapse/issues/11262))
+- Only allow old Element/Riot Android clients to send read receipts without a request body. All other clients must include a request body as required by the specification. Contributed by @rogersheu. ([\#11157](https://github.com/matrix-org/synapse/issues/11157))
+
+
+Updates to the Docker image
+---------------------------
+
+- Avoid changing user ID when started as a non-root user, and no explicit `UID` is set. ([\#11209](https://github.com/matrix-org/synapse/issues/11209))
+
+
+Improved Documentation
+----------------------
+
+- Improve example HAProxy config in the docs to properly handle HTTP `Host` headers with port information. This is required for federation over port 443 to work correctly. ([\#11128](https://github.com/matrix-org/synapse/issues/11128))
+- Add documentation for using Authentik as an OpenID Connect Identity Provider. Contributed by @samip5. ([\#11151](https://github.com/matrix-org/synapse/issues/11151))
+- Clarify lack of support for Windows. ([\#11198](https://github.com/matrix-org/synapse/issues/11198))
+- Improve code formatting and fix a few typos in docs. Contributed by @sumnerevans at Beeper. ([\#11221](https://github.com/matrix-org/synapse/issues/11221))
+- Add documentation for using LemonLDAP as an OpenID Connect Identity Provider. Contributed by @l00ptr. ([\#11257](https://github.com/matrix-org/synapse/issues/11257))
+
+
+Internal Changes
+----------------
+
+- Add type annotations for the `log_function` decorator. ([\#10943](https://github.com/matrix-org/synapse/issues/10943))
+- Add type hints to `synapse.events`. ([\#11098](https://github.com/matrix-org/synapse/issues/11098))
+- Remove and document unnecessary `RoomStreamToken` checks in application service ephemeral event code. ([\#11137](https://github.com/matrix-org/synapse/issues/11137))
+- Add type hints so that `synapse.http` passes `mypy` checks. ([\#11164](https://github.com/matrix-org/synapse/issues/11164))
+- Update scripts to pass Shellcheck lints. ([\#11166](https://github.com/matrix-org/synapse/issues/11166))
+- Add knock information in admin export. Contributed by Rafael Gonçalves. ([\#11171](https://github.com/matrix-org/synapse/issues/11171))
+- Add tests to check that `ClientIpStore.get_last_client_ip_by_device` and `get_user_ip_and_agents` combine database and in-memory data correctly. ([\#11179](https://github.com/matrix-org/synapse/issues/11179))
+- Refactor `Filter` to check different fields depending on the data type. ([\#11194](https://github.com/matrix-org/synapse/issues/11194))
+- Improve type hints for the relations datastore. ([\#11205](https://github.com/matrix-org/synapse/issues/11205))
+- Replace outdated links in the pull request checklist with links to the rendered documentation. ([\#11225](https://github.com/matrix-org/synapse/issues/11225))
+- Fix a bug in unit test `test_block_room_and_not_purge`. ([\#11226](https://github.com/matrix-org/synapse/issues/11226))
+- In `ObservableDeferred`, run observers in the order they were registered. ([\#11229](https://github.com/matrix-org/synapse/issues/11229))
+- Minor speed up to start up times and getting updates for groups by adding missing index to `local_group_updates.stream_id`. ([\#11231](https://github.com/matrix-org/synapse/issues/11231))
+- Add `twine` and `towncrier` as dev dependencies, as they're used by the release script. ([\#11233](https://github.com/matrix-org/synapse/issues/11233))
+- Allow `stream_writers.typing` config to be a list of one worker. ([\#11237](https://github.com/matrix-org/synapse/issues/11237))
+- Remove debugging statement in tests. ([\#11239](https://github.com/matrix-org/synapse/issues/11239))
+- Fix [MSC2716](https://github.com/matrix-org/matrix-doc/pull/2716) historical messages backfilling in random order on remote homeservers. ([\#11244](https://github.com/matrix-org/synapse/issues/11244))
+- Add an additional test for the `cachedList` method decorator. ([\#11246](https://github.com/matrix-org/synapse/issues/11246))
+- Make minor correction to the type of `auth_checkers` callbacks. ([\#11253](https://github.com/matrix-org/synapse/issues/11253))
+- Clean up trivial aspects of the Debian package build tooling. ([\#11269](https://github.com/matrix-org/synapse/issues/11269), [\#11273](https://github.com/matrix-org/synapse/issues/11273))
+- Blacklist new SyTest that checks that key uploads are valid pending the validation being implemented in Synapse. ([\#11270](https://github.com/matrix-org/synapse/issues/11270))
+
+
 Synapse 1.46.0 (2021-11-02)
 ===========================
 
@@ -8584,14 +8678,14 @@ General:
 
 Federation:
 
--   Add key distribution mechanisms for fetching public keys of unavailable remote home servers. See [Retrieving Server Keys](https://github.com/matrix-org/matrix-doc/blob/6f2698/specification/30_server_server_api.rst#retrieving-server-keys) in the spec.
+-   Add key distribution mechanisms for fetching public keys of unavailable remote homeservers. See [Retrieving Server Keys](https://github.com/matrix-org/matrix-doc/blob/6f2698/specification/30_server_server_api.rst#retrieving-server-keys) in the spec.
 
 Configuration:
 
 -   Add support for multiple config files.
 -   Add support for dictionaries in config files.
 -   Remove support for specifying config options on the command line, except for:
-    -   `--daemonize` - Daemonize the home server.
+    -   `--daemonize` - Daemonize the homeserver.
     -   `--manhole` - Turn on the twisted telnet manhole service on the given port.
     -   `--database-path` - The path to a sqlite database to use.
     -   `--verbose` - The verbosity level.
@@ -8796,7 +8890,7 @@ This version adds support for using a TURN server. See docs/turn-howto.rst on ho
 Homeserver:
 
 -   Add support for redaction of messages.
--   Fix bug where inviting a user on a remote home server could take up to 20-30s.
+-   Fix bug where inviting a user on a remote homeserver could take up to 20-30s.
 -   Implement a get current room state API.
 -   Add support specifying and retrieving turn server configuration.
 
@@ -8886,7 +8980,7 @@ Changes in synapse 0.2.3 (2014-09-12)
 
 Homeserver:
 
--   Fix bug where we stopped sending events to remote home servers if a user from that home server left, even if there were some still in the room.
+-   Fix bug where we stopped sending events to remote homeservers if a user from that homeserver left, even if there were some still in the room.
 -   Fix bugs in the state conflict resolution where it was incorrectly rejecting events.
 
 Webclient:

diff --git a/changelog.d/10097.bugfix b/changelog.d/10097.bugfix
diff --git a/changelog.d/10943.misc b/changelog.d/10943.misc
diff --git a/changelog.d/10969.bugfix b/changelog.d/10969.bugfix
diff --git a/changelog.d/11033.bugfix b/changelog.d/11033.bugfix
diff --git a/changelog.d/11097.feature b/changelog.d/11097.feature
diff --git a/changelog.d/11098.misc b/changelog.d/11098.misc
diff --git a/changelog.d/11099.feature b/changelog.d/11099.feature
diff --git a/changelog.d/11126.feature b/changelog.d/11126.feature
diff --git a/changelog.d/11128.doc b/changelog.d/11128.doc
diff --git a/changelog.d/11129.bugfix b/changelog.d/11129.bugfix
diff --git a/changelog.d/11137.misc b/changelog.d/11137.misc
diff --git a/changelog.d/11147.feature b/changelog.d/11147.feature
diff --git a/changelog.d/11151.doc b/changelog.d/11151.doc
diff --git a/changelog.d/11157.misc b/changelog.d/11157.misc
diff --git a/changelog.d/11164.misc b/changelog.d/11164.misc
diff --git a/changelog.d/11166.misc b/changelog.d/11166.misc
diff --git a/changelog.d/11171.misc b/changelog.d/11171.misc
diff --git a/changelog.d/11178.feature b/changelog.d/11178.feature
diff --git a/changelog.d/11179.misc b/changelog.d/11179.misc
diff --git a/changelog.d/11187.feature b/changelog.d/11187.feature
diff --git a/changelog.d/11188.bugfix b/changelog.d/11188.bugfix
diff --git a/changelog.d/11191.bugfix b/changelog.d/11191.bugfix
diff --git a/changelog.d/11194.misc b/changelog.d/11194.misc
diff --git a/changelog.d/11198.doc b/changelog.d/11198.doc
diff --git a/changelog.d/11199.bugfix b/changelog.d/11199.bugfix
diff --git a/changelog.d/11200.bugfix b/changelog.d/11200.bugfix
diff --git a/changelog.d/11204.feature b/changelog.d/11204.feature
diff --git a/changelog.d/11205.misc b/changelog.d/11205.misc
diff --git a/changelog.d/11206.removal b/changelog.d/11206.removal
diff --git a/changelog.d/11207.bugfix b/changelog.d/11207.bugfix
diff --git a/changelog.d/11209.docker b/changelog.d/11209.docker
diff --git a/changelog.d/11210.feature b/changelog.d/11210.feature
diff --git a/changelog.d/11211.feature b/changelog.d/11211.feature
diff --git a/changelog.d/11212.bugfix b/changelog.d/11212.bugfix
diff --git a/changelog.d/11213.removal b/changelog.d/11213.removal
diff --git a/changelog.d/11217.bugfix b/changelog.d/11217.bugfix
diff --git a/changelog.d/11221.doc b/changelog.d/11221.doc
diff --git a/changelog.d/11223.feature b/changelog.d/11223.feature
@@ -0,0 +1 @@
+Add a new version of delete room admin API `DELETE /_synapse/admin/v2/rooms/<room_id>` to run it in background. Contributed by @dklimpel.
diff --git a/changelog.d/11225.misc b/changelog.d/11225.misc
diff --git a/changelog.d/11226.misc b/changelog.d/11226.misc
diff --git a/changelog.d/11229.misc b/changelog.d/11229.misc
diff --git a/changelog.d/11230.bugfix b/changelog.d/11230.bugfix
@@ -0,0 +1,2 @@
+Fix a long-standing bug wherein display names or avatar URLs containing null bytes cause an internal server error
+when stored in the DB.
diff --git a/changelog.d/11231.misc b/changelog.d/11231.misc
diff --git a/changelog.d/11233.misc b/changelog.d/11233.misc
diff --git a/changelog.d/11234.bugfix b/changelog.d/11234.bugfix
diff --git a/changelog.d/11237.misc b/changelog.d/11237.misc
diff --git a/changelog.d/11239.misc b/changelog.d/11239.misc
diff --git a/changelog.d/11240.bugfix b/changelog.d/11240.bugfix
diff --git a/changelog.d/11244.misc b/changelog.d/11244.misc
diff --git a/changelog.d/11246.misc b/changelog.d/11246.misc
diff --git a/changelog.d/11253.misc b/changelog.d/11253.misc
diff --git a/changelog.d/11255.bugfix b/changelog.d/11255.bugfix
diff --git a/changelog.d/11257.doc b/changelog.d/11257.doc
diff --git a/changelog.d/11262.bugfix b/changelog.d/11262.bugfix
diff --git a/changelog.d/11263.feature b/changelog.d/11263.feature
diff --git a/changelog.d/11269.misc b/changelog.d/11269.misc
diff --git a/changelog.d/11270.misc b/changelog.d/11270.misc
diff --git a/changelog.d/11273.misc b/changelog.d/11273.misc
diff --git a/changelog.d/11276.bugfix b/changelog.d/11276.bugfix
diff --git a/changelog.d/11280.misc b/changelog.d/11280.misc
@@ -0,0 +1 @@
+Drop unused db tables `room_stats_historical` and `user_stats_historical`.
diff --git a/changelog.d/11281.doc b/changelog.d/11281.doc
@@ -0,0 +1 @@
+Suggest users of the Debian packages add configuration to `/etc/matrix-synapse/conf.d/` to prevent, upon upgrade, being asked to choose between their configuration and the maintainer's.
diff --git a/changelog.d/11287.misc b/changelog.d/11287.misc
@@ -0,0 +1 @@
+Add missing type hints to `synapse.app`.
diff --git a/changelog.d/11288.bugfix b/changelog.d/11288.bugfix
@@ -0,0 +1 @@
+Fix a long-standing bug where uploading extremely thin images (e.g. 1000x1) would fail. Contributed by @Neeeflix.
diff --git a/changelog.d/11292.misc b/changelog.d/11292.misc
@@ -0,0 +1 @@
+Remove unused parameters on `FederationEventHandler._check_event_auth`.
diff --git a/changelog.d/11297.misc b/changelog.d/11297.misc
@@ -0,0 +1 @@
+Add type hints to `synapse._scripts`.
diff --git a/changelog.d/11298.doc b/changelog.d/11298.doc
@@ -0,0 +1 @@
+Add Single Sign-On, SAML and CAS pages to the documentation.
diff --git a/changelog.d/11303.misc b/changelog.d/11303.misc
@@ -0,0 +1 @@
+Fix an issue which prevented the 'remove deleted devices from device_inbox column' background process from running when updating from a recent Synapse version.
diff --git a/changelog.d/11307.misc b/changelog.d/11307.misc
@@ -0,0 +1 @@
+Add type hints to storage classes.
diff --git a/changelog.d/11310.misc b/changelog.d/11310.misc
@@ -0,0 +1 @@
+Add type hints to storage classes.
diff --git a/changelog.d/11311.misc b/changelog.d/11311.misc
@@ -0,0 +1 @@
+Add type hints to storage classes.
diff --git a/changelog.d/11312.misc b/changelog.d/11312.misc
@@ -0,0 +1 @@
+Add type hints to storage classes.
diff --git a/changelog.d/11313.misc b/changelog.d/11313.misc
@@ -0,0 +1 @@
+Add type hints to storage classes.
diff --git a/changelog.d/11314.misc b/changelog.d/11314.misc
@@ -0,0 +1 @@
+Add type hints to storage classes.
diff --git a/changelog.d/11316.misc b/changelog.d/11316.misc
@@ -0,0 +1 @@
+Add type hints to storage classes.
diff --git a/changelog.d/11318.feature b/changelog.d/11318.feature
@@ -0,0 +1 @@
+Add support for the `/_matrix/client/v3` APIs from Matrix v1.1.
Original file line number	Diff line number	Diff line change
		@@ -0,0 +1 @@
		Add a new version of delete room admin API `DELETE /_synapse/admin/v2/rooms/<room_id>` to run it in background. Contributed by @dklimpel.
Original file line number	Diff line number	Diff line change
		@@ -0,0 +1,2 @@
		Fix a long-standing bug wherein display names or avatar URLs containing null bytes cause an internal server error
		when stored in the DB.
Original file line number	Diff line number	Diff line change
		@@ -0,0 +1 @@
		Drop unused db tables `room_stats_historical` and `user_stats_historical`.
Original file line number	Diff line number	Diff line change
		@@ -0,0 +1 @@
		Suggest users of the Debian packages add configuration to `/etc/matrix-synapse/conf.d/` to prevent, upon upgrade, being asked to choose between their configuration and the maintainer's.
Original file line number	Diff line number	Diff line change
		@@ -0,0 +1 @@
		Fix a long-standing bug where uploading extremely thin images (e.g. 1000x1) would fail. Contributed by @Neeeflix.
Original file line number	Diff line number	Diff line change
		@@ -0,0 +1 @@
		Remove unused parameters on `FederationEventHandler._check_event_auth`.
Original file line number	Diff line number	Diff line change
		@@ -0,0 +1 @@
		Add Single Sign-On, SAML and CAS pages to the documentation.
Original file line number	Diff line number	Diff line change
		@@ -0,0 +1 @@
		Fix an issue which prevented the 'remove deleted devices from device_inbox column' background process from running when updating from a recent Synapse version.
Original file line number	Diff line number	Diff line change
		@@ -0,0 +1 @@
		Add support for the `/_matrix/client/v3` APIs from Matrix v1.1.