Update tracer to give more information

Add a utility class that can be used to generate a twisted deferred aware call graph
2025-12-19 02:20:44 +00:00 · 2015-03-13 11:38:57 +00:00 · 2015-03-12 16:52:02 +00:00
1071 changed files with 26987 additions and 161073 deletions
--- a/.buildkite/.env
+++ b/.buildkite/.env
@@ -1,13 +0,0 @@
 CI
 BUILDKITE
 BUILDKITE_BUILD_NUMBER
 BUILDKITE_BRANCH
 BUILDKITE_BUILD_NUMBER
 BUILDKITE_JOB_ID
 BUILDKITE_BUILD_URL
 BUILDKITE_PROJECT_SLUG
 BUILDKITE_COMMIT
 BUILDKITE_PULL_REQUEST
 BUILDKITE_TAG
 CODECOV_TOKEN
 TRIAL_FLAGS
--- a/.buildkite/docker-compose.py35.pg95.yaml
+++ b/.buildkite/docker-compose.py35.pg95.yaml
@@ -1,22 +0,0 @@
 version: '3.1'
 services:
  postgres:
    image: postgres:9.5
    environment:
      POSTGRES_PASSWORD: postgres
    command: -c fsync=off
  testenv:
    image: python:3.5
    depends_on:
      - postgres
    env_file: .env
    environment:
      SYNAPSE_POSTGRES_HOST: postgres
      SYNAPSE_POSTGRES_USER: postgres
      SYNAPSE_POSTGRES_PASSWORD: postgres
    working_dir: /src
    volumes:
      - ..:/src
--- a/.buildkite/docker-compose.py37.pg11.yaml
+++ b/.buildkite/docker-compose.py37.pg11.yaml
@@ -1,22 +0,0 @@
 version: '3.1'
 services:
  postgres:
    image: postgres:11
    environment:
      POSTGRES_PASSWORD: postgres
    command: -c fsync=off
  testenv:
    image: python:3.7
    depends_on:
      - postgres
    env_file: .env
    environment:
      SYNAPSE_POSTGRES_HOST: postgres
      SYNAPSE_POSTGRES_USER: postgres
      SYNAPSE_POSTGRES_PASSWORD: postgres
    working_dir: /src
    volumes:
      - ..:/src
--- a/.buildkite/docker-compose.py37.pg95.yaml
+++ b/.buildkite/docker-compose.py37.pg95.yaml
@@ -1,22 +0,0 @@
 version: '3.1'
 services:
  postgres:
    image: postgres:9.5
    environment:
      POSTGRES_PASSWORD: postgres
    command: -c fsync=off
  testenv:
    image: python:3.7
    depends_on:
      - postgres
    env_file: .env
    environment:
      SYNAPSE_POSTGRES_HOST: postgres
      SYNAPSE_POSTGRES_USER: postgres
      SYNAPSE_POSTGRES_PASSWORD: postgres
    working_dir: /src
    volumes:
      - ..:/src
--- a/.buildkite/format_tap.py
+++ b/.buildkite/format_tap.py
@@ -1,48 +0,0 @@
 # -*- coding: utf-8 -*-
 # Copyright 2019 The Matrix.org Foundation C.I.C.
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
 # You may obtain a copy of the License at
 #
 #     http://www.apache.org/licenses/LICENSE-2.0
 #
 # Unless required by applicable law or agreed to in writing, software
 # distributed under the License is distributed on an "AS IS" BASIS,
 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 # See the License for the specific language governing permissions and
 # limitations under the License.
 import sys
 from tap.parser import Parser
 from tap.line import Result, Unknown, Diagnostic
 out = ["### TAP Output for " + sys.argv[2]]
 p = Parser()
 in_error = False
 for line in p.parse_file(sys.argv[1]):
    if isinstance(line, Result):
        if in_error:
            out.append("")
            out.append("</pre></code></details>")
            out.append("")
            out.append("----")
            out.append("")
        in_error = False
        if not line.ok and not line.todo:
            in_error = True
            out.append("FAILURE Test #%d: ``%s``" % (line.number, line.description))
            out.append("")
            out.append("<details><summary>Show log</summary><code><pre>")
    elif isinstance(line, Diagnostic) and in_error:
        out.append(line.text)
 if out:
    for line in out[:-3]:
        print(line)
--- a/.buildkite/merge_base_branch.sh
+++ b/.buildkite/merge_base_branch.sh
@@ -1,33 +0,0 @@
 #!/usr/bin/env bash
 set -ex
 if [[ "$BUILDKITE_BRANCH" =~ ^(develop|master|dinsic|shhs|release-.*)$ ]]; then
    echo "Not merging forward, as this is a release branch"
    exit 0
 fi
 if [[ -z $BUILDKITE_PULL_REQUEST_BASE_BRANCH ]]; then
    echo "Not a pull request, or hasn't had a PR opened yet..."
    # It probably hasn't had a PR opened yet. Since all PRs land on develop, we
    # can probably assume it's based on it and will be merged into it.
    GITBASE="develop"
 else
    # Get the reference, using the GitHub API
    GITBASE=$BUILDKITE_PULL_REQUEST_BASE_BRANCH
 fi
 # Show what we are before
 git --no-pager show -s
 # Set up username so it can do a merge
 git config --global user.email bot@matrix.org
 git config --global user.name "A robot"
 # Fetch and merge. If it doesn't work, it will raise due to set -e.
 git fetch -u origin $GITBASE
 git merge --no-edit --no-commit origin/$GITBASE
 # Show what we are after.
 git --no-pager show -s
--- a/.buildkite/worker-blacklist
+++ b/.buildkite/worker-blacklist
@@ -1,30 +0,0 @@
 # This file serves as a blacklist for SyTest tests that we expect will fail in
 # Synapse when run under worker mode. For more details, see sytest-blacklist.
 Message history can be paginated
 Can re-join room if re-invited
 /upgrade creates a new room
 The only membership state included in an initial sync is for all the senders in the timeline
 Local device key changes get to remote servers
 If remote user leaves room we no longer receive device updates
 Forgotten room messages cannot be paginated
 Inbound federation can get public room list
 Members from the gap are included in gappy incr LL sync
 Leaves are present in non-gapped incremental syncs
 Old leaves are present in gapped incremental syncs
 User sees updates to presence from other users in the incremental sync.
 Gapped incremental syncs include all state changes
 Old members are included in gappy incr LL sync if they start speaking
--- a/.circleci/config.yml
+++ b/.circleci/config.yml
@@ -1,33 +0,0 @@
 version: 2
 jobs:
  dockerhubuploadrelease:
    machine: true
    steps:
      - checkout
      - run: docker build -f docker/Dockerfile --label gitsha1=${CIRCLE_SHA1} -t matrixdotorg/synapse:${CIRCLE_TAG} -t matrixdotorg/synapse:${CIRCLE_TAG}-py3 .
      - run: docker login --username $DOCKER_HUB_USERNAME --password $DOCKER_HUB_PASSWORD
      - run: docker push matrixdotorg/synapse:${CIRCLE_TAG}
      - run: docker push matrixdotorg/synapse:${CIRCLE_TAG}-py3
  dockerhubuploadlatest:
    machine: true
    steps:
      - checkout
      - run: docker build -f docker/Dockerfile --label gitsha1=${CIRCLE_SHA1} -t matrixdotorg/synapse:latest -t matrixdotorg/synapse:latest-py3 .
      - run: docker login --username $DOCKER_HUB_USERNAME --password $DOCKER_HUB_PASSWORD
      - run: docker push matrixdotorg/synapse:latest
      - run: docker push matrixdotorg/synapse:latest-py3
 workflows:
  version: 2
  build:
    jobs:
      - dockerhubuploadrelease:
          filters:
            tags:
              only: /v[0-9].[0-9]+.[0-9]+.*/
            branches:
              ignore: /.*/
      - dockerhubuploadlatest:
          filters:
            branches:
              only: master
--- a/.codecov.yml
+++ b/.codecov.yml
@@ -1,14 +0,0 @@
 comment: off
 coverage:
  status:
    project:
      default:
        target: 0  # Target % coverage, can be auto. Turned off for now
        threshold: null
        base: auto
    patch:
      default:
        target: 0
        threshold: null
        base: auto
--- a/.coveragerc
+++ b/.coveragerc
@@ -1,8 +0,0 @@
 [run]
 branch = True
 parallel = True
 include=$TOP/synapse/*
 data_file = $TOP/.coverage
 [report]
 precision = 2
--- a/.dockerignore
+++ b/.dockerignore
@@ -1,13 +0,0 @@
 # ignore everything by default
 *
 # things to include
 !docker
 !scripts
 !synapse
 !MANIFEST.in
 !README.rst
 !setup.py
 !synctl
 **/__pycache__
--- a/.editorconfig
+++ b/.editorconfig
@@ -1,9 +0,0 @@
 # EditorConfig https://EditorConfig.org
 # top-most EditorConfig file
 root = true
 # 4 space indentation
 [*.py]
 indent_style = space
 indent_size = 4
--- a/.github/FUNDING.yml
+++ b/.github/FUNDING.yml
@@ -1,4 +0,0 @@
 # One username per supported platform and one custom link
 patreon: matrixdotorg
 liberapay: matrixdotorg
 custom: https://paypal.me/matrixdotorg
--- a/.github/ISSUE_TEMPLATE/BUG_REPORT.md
+++ b/.github/ISSUE_TEMPLATE/BUG_REPORT.md
@@ -1,66 +0,0 @@
 ---
 name: Bug report
 about: Create a report to help us improve
 ---
 <!--
 **IF YOU HAVE SUPPORT QUESTIONS ABOUT RUNNING OR CONFIGURING YOUR OWN HOME SERVER**:
 You will likely get better support more quickly if you ask in ** #matrix:matrix.org ** ;)
 This is a bug report template. By following the instructions below and
 filling out the sections with your information, you will help the us to get all
 the necessary data to fix your issue.
 You can also preview your report before submitting it. You may remove sections
 that aren't relevant to your particular case.
 Text between <!-- and --> marks will be invisible in the report.
 -->
 ### Description
 <!-- Describe here the problem that you are experiencing -->
 ### Steps to reproduce
 - list the steps
 - that reproduce the bug
 - using hyphens as bullet points
 <!--
 Describe how what happens differs from what you expected.
 If you can identify any relevant log snippets from _homeserver.log_, please include
 those (please be careful to remove any personal or private data). Please surround them with
 ``` (three backticks, on a line on their own), so that they are formatted legibly.
 -->
 ### Version information
 <!-- IMPORTANT: please answer the following questions, to help us narrow down the problem -->
 <!-- Was this issue identified on matrix.org or another homeserver? -->
 - **Homeserver**: 
 If not matrix.org:
 <!--
 What version of Synapse is running?
 You can find the Synapse version by inspecting the server headers (replace matrix.org with
 your own homeserver domain):
 $ curl -v https://matrix.org/_matrix/client/versions 2>&1 | grep "Server:"
 -->
 - **Version**: 
 - **Install method**: 
 <!-- examples: package manager/git clone/pip  -->
 - **Platform**: 
 <!--
 Tell us about the environment in which your homeserver is operating
 distro, hardware, if it's running in a vm/container, etc.
 -->
--- a/.github/ISSUE_TEMPLATE/FEATURE_REQUEST.md
+++ b/.github/ISSUE_TEMPLATE/FEATURE_REQUEST.md
@@ -1,9 +0,0 @@
 ---
 name: Feature request
 about: Suggest an idea for this project
 ---
 **Description:**
 <!-- Describe here the feature you are requesting. -->
--- a/.github/ISSUE_TEMPLATE/SUPPORT_REQUEST.md
+++ b/.github/ISSUE_TEMPLATE/SUPPORT_REQUEST.md
@@ -1,10 +0,0 @@
 ---
 name: Support request
 about: I need support for Synapse
 ---
 Please don't file github issues asking for support.
 Instead, please join [`#synapse:matrix.org`](https://matrix.to/#/#synapse:matrix.org)
 (from a matrix.org account if necessary), and ask there.
--- a/.github/PULL_REQUEST_TEMPLATE.md
+++ b/.github/PULL_REQUEST_TEMPLATE.md
@@ -1,7 +0,0 @@
 ### Pull Request Checklist
 <!-- Please read CONTRIBUTING.rst before submitting your pull request -->
 * [ ] Pull request is based on the develop branch
 * [ ] Pull request includes a [changelog file](https://github.com/matrix-org/synapse/blob/master/CONTRIBUTING.rst#changelog)
 * [ ] Pull request includes a [sign off](https://github.com/matrix-org/synapse/blob/master/CONTRIBUTING.rst#sign-off)
--- a/.github/SUPPORT.md
+++ b/.github/SUPPORT.md
@@ -1,3 +0,0 @@
 [**#synapse:matrix.org**](https://matrix.to/#/#synapse:matrix.org) is the official support room for
 Synapse, and can be accessed by any client from https://matrix.org/docs/projects/try-matrix-now.html.
 Please ask for support there, rather than filing github issues.
--- a/.gitignore
+++ b/.gitignore
@@ -1,42 +1,44 @@
 # filename patterns
 *~
 .*.swp
 .#*
 *.deb
 *.egg
 *.egg-info
 *.lock
 *.pyc
-*.tac
+.*.swp
 .DS_Store
 _trial_temp/
-_trial_temp*/
+logs/
 dbs/
 *.egg
 dist/
 docs/build/
 *.egg-info
-# stuff that is likely to exist when you run a server locally
+cmdclient_config.json
-/*.db
+homeserver*.db
-/*.log
+homeserver*.log
-/*.log.config
+homeserver*.pid
-/*.pid
+homeserver*.yaml
 /.python-version
 /*.signing.key
 /env/
 /homeserver*.yaml
 /logs
 /media_store/
 /uploads
-# IDEs
+*.signing.key
-/.idea/
+*.tls.crt
-/.ropeproject/
+*.tls.dh
-/.vscode/
+*.tls.key
-# build products
+.coverage
-!/.coveragerc
+htmlcov
-/.coverage*
+
-/.mypy_cache/
+demo/*.db
-/.tox
+demo/*.log
-/build/
+demo/*.log.*
-/coverage.*
+demo/*.pid
-/dist/
+demo/media_store.*
-/docs/build/
+demo/etc
-/htmlcov
+
-/pip-wheel-metadata/
+uploads
 .idea/
 media_store/
 *.tac
 build/
 localhost-800*/
 static/client/register/register_config.js
--- a/AUTHORS.rst
+++ b/AUTHORS.rst
@@ -1,77 +0,0 @@
 Erik Johnston <erik at matrix.org>
 * HS core
 * Federation API impl
 Mark Haines <mark at matrix.org>
 * HS core
 * Crypto
 * Content repository
 * CS v2 API impl
 Kegan Dougal <kegan at matrix.org>
 * HS core
 * CS v1 API impl
 * AS API impl
 Paul "LeoNerd" Evans <paul at matrix.org>
 * HS core
 * Presence
 * Typing Notifications
 * Performance metrics and caching layer
 Dave Baker <dave at matrix.org>
 * Push notifications
 * Auth CS v2 impl
 Matthew Hodgson <matthew at matrix.org>
 * General doc & housekeeping
 * Vertobot/vertobridge matrix<->verto PoC
 Emmanuel Rohee <manu at matrix.org>
 * Supporting iOS clients (testability and fallback registration)
 Turned to Dust <dwinslow86 at gmail.com>
 * ArchLinux installation instructions
 Brabo <brabo at riseup.net>
 * Installation instruction fixes
 Ivan Shapovalov <intelfx100 at gmail.com>
 * contrib/systemd: a sample systemd unit file and a logger configuration
 Eric Myhre <hash at exultant.us>
 * Fix bug where ``media_store_path`` config option was ignored by v0 content
   repository API.
 Muthu Subramanian <muthu.subramanian.karunanidhi at ericsson.com>
 * Add SAML2 support for registration and login.
 Steven Hammerton <steven.hammerton at openmarket.com>
 * Add CAS support for registration and login.
 Mads Robin Christensen <mads at v42 dot dk>
 * CentOS 7 installation instructions.
 Florent Violleau <floviolleau at gmail dot com>
 * Add Raspberry Pi installation instructions and general troubleshooting items
 Niklas Riekenbrauck <nikriek at gmail dot.com>
 * Add JWT support for registration and login
 Christoph Witzany <christoph at web.crofting.com>
 * Add LDAP support for authentication
 Pierre Jaury <pierre at jaury.eu>
 * Docker packaging
 Serban Constantin <serban.constantin at gmail dot com>
 * Small bug fix
 Jason Robinson <jasonr at matrix.org>
 * Minor fixes
 Joseph Weston <joseph at weston.cloud>
 + Add admin API for querying HS version
 Benjamin Saunders <ben.e.saunders at gmail dot com>
 * Documentation improvements
--- a/CHANGES.md
+++ b/CHANGES.md
--- a/CHANGES.rst
+++ b/CHANGES.rst
@@ -0,0 +1,449 @@
 Changes in synapse v0.8.0 (2015-03-06)
 ======================================
 General:
 * Add support for registration fallback. This is a page hosted on the server
  which allows a user to register for an account, regardless of what client
  they are using (e.g. mobile devices).
 * Added new default push rules and made them configurable by clients:
  * Suppress all notice messages.
  * Notify when invited to a new room.
  * Notify for messages that don't match any rule.
  * Notify on incoming call.
 Federation:
 * Added per host server side rate-limiting of incoming federation requests.
 * Added a ``/get_missing_events/`` API to federation to reduce number of
  ``/events/`` requests.
 Configuration:
 * Added configuration option to disable registration:
  ``disable_registration``.
 * Added configuration option to change soft limit of number of open file
  descriptors: ``soft_file_limit``.
 * Make ``tls_private_key_path`` optional when running with ``no_tls``.
 Application services:
 * Application services can now poll on the CS API ``/events`` for their events,
  by providing their application service ``access_token``.
 * Added exclusive namespace support to application services API.
 Changes in synapse v0.7.1 (2015-02-19)
 ======================================
 * Initial alpha implementation of parts of the Application Services API.
  Including:
  - AS Registration / Unregistration
  - User Query API
  - Room Alias Query API
  - Push transport for receiving events.
  - User/Alias namespace admin control
 * Add cache when fetching events from remote servers to stop repeatedly
  fetching events with bad signatures.
 * Respect the per remote server retry scheme when fetching both events and
  server keys to reduce the number of times we send requests to dead servers.
 * Inform remote servers when the local server fails to handle a received event.
 * Turn off python bytecode generation due to problems experienced when
  upgrading from previous versions.
 Changes in synapse v0.7.0 (2015-02-12)
 ======================================
 * Add initial implementation of the query auth federation API, allowing
  servers to agree on whether an event should be allowed or rejected.
 * Persist events we have rejected from federation, fixing the bug where
  servers would keep requesting the same events.
 * Various federation performance improvements, including:
  - Add in memory caches on queries such as:
     * Computing the state of a room at a point in time, used for
       authorization on federation requests.
     * Fetching events from the database.
     * User's room membership, used for authorizing presence updates.
  - Upgraded JSON library to improve parsing and serialisation speeds.
 * Add default avatars to new user accounts using pydenticon library.
 * Correctly time out federation requests.
 * Retry federation requests against different servers.
 * Add support for push and push rules.
 * Add alpha versions of proposed new CSv2 APIs, including ``/sync`` API.
 Changes in synapse 0.6.1 (2015-01-07)
 =====================================
 * Major optimizations to improve performance of initial sync and event sending
  in large rooms (by up to 10x)
 * Media repository now includes a Content-Length header on media downloads.
 * Improve quality of thumbnails by changing resizing algorithm.
 Changes in synapse 0.6.0 (2014-12-16)
 =====================================
 * Add new API for media upload and download that supports thumbnailing.
 * Replicate media uploads over multiple homeservers so media is always served
  to clients from their local homeserver.  This obsoletes the
  --content-addr parameter and confusion over accessing content directly
  from remote homeservers.
 * Implement exponential backoff when retrying federation requests when
  sending to remote homeservers which are offline.
 * Implement typing notifications.
 * Fix bugs where we sent events with invalid signatures due to bugs where
  we incorrectly persisted events.
 * Improve performance of database queries involving retrieving events.
 Changes in synapse 0.5.4a (2014-12-13)
 ======================================
 * Fix bug while generating the error message when a file path specified in
  the config doesn't exist.
 Changes in synapse 0.5.4 (2014-12-03)
 =====================================
 * Fix presence bug where some rooms did not display presence updates for
  remote users.
 * Do not log SQL timing log lines when started with "-v"
 * Fix potential memory leak.
 Changes in synapse 0.5.3c (2014-12-02)
 ======================================
 * Change the default value for the `content_addr` option to use the HTTP
  listener, as by default the HTTPS listener will be using a self-signed
  certificate.
 Changes in synapse 0.5.3 (2014-11-27)
 =====================================
 * Fix bug that caused joining a remote room to fail if a single event was not
  signed correctly.
 * Fix bug which caused servers to continuously try and fetch events from other
  servers.
 Changes in synapse 0.5.2 (2014-11-26)
 =====================================
 Fix major bug that caused rooms to disappear from peoples initial sync.
 Changes in synapse 0.5.1 (2014-11-26)
 =====================================
 See UPGRADES.rst for specific instructions on how to upgrade.
 * Fix bug where we served up an Event that did not match its signatures.
 * Fix regression where we no longer correctly handled the case where a
   homeserver receives an event for a room it doesn't recognise (but is in.)
 Changes in synapse 0.5.0 (2014-11-19)
 =====================================
 This release includes changes to the federation protocol and client-server API
 that is not backwards compatible.
 This release also changes the internal database schemas and so requires servers to
 drop their current history. See UPGRADES.rst for details.
 Homeserver:
 * Add authentication and authorization to the federation protocol. Events are
   now signed by their originating homeservers.
 * Implement the new authorization model for rooms.
 * Split out web client into a seperate repository: matrix-angular-sdk.
 * Change the structure of PDUs.
 * Fix bug where user could not join rooms via an alias containing 4-byte
   UTF-8 characters.
 * Merge concept of PDUs and Events internally.
 * Improve logging by adding request ids to log lines.
 * Implement a very basic room initial sync API.
 * Implement the new invite/join federation APIs.
 Webclient:
 * The webclient has been moved to a seperate repository.
 Changes in synapse 0.4.2 (2014-10-31)
 =====================================
 Homeserver:
 * Fix bugs where we did not notify users of correct presence updates.
 * Fix bug where we did not handle sub second event stream timeouts.
 Webclient:
 * Add ability to click on messages to see JSON.
 * Add ability to redact messages.
 * Add ability to view and edit all room state JSON.
 * Handle incoming redactions.
 * Improve feedback on errors.
 * Fix bugs in mobile CSS.
 * Fix bugs with desktop notifications.
 Changes in synapse 0.4.1 (2014-10-17)
 =====================================
 Webclient:
 * Fix bug with display of timestamps.
 Changes in synpase 0.4.0 (2014-10-17)
 =====================================
 This release includes changes to the federation protocol and client-server API
 that is not backwards compatible.
 The Matrix specification has been moved to a separate git repository:
 http://github.com/matrix-org/matrix-doc
 You will also need an updated syutil and config. See UPGRADES.rst.
 Homeserver:
 * Sign federation transactions to assert strong identity over federation.
 * Rename timestamp keys in PDUs and events from 'ts' and 'hsob_ts' to 'origin_server_ts'.
 Changes in synapse 0.3.4 (2014-09-25)
 =====================================
 This version adds support for using a TURN server. See docs/turn-howto.rst on
 how to set one up.
 Homeserver:
 * Add support for redaction of messages.
 * Fix bug where inviting a user on a remote home server could take up to
   20-30s.
 * Implement a get current room state API.
 * Add support specifying and retrieving turn server configuration.
 Webclient:
 * Add button to send messages to users from the home page.
 * Add support for using TURN for VoIP calls.
 * Show display name change messages.
 * Fix bug where the client didn't get the state of a newly joined room
   until after it has been refreshed.
 * Fix bugs with tab complete.
 * Fix bug where holding down the down arrow caused chrome to chew 100% CPU.
 * Fix bug where desktop notifications occasionally used "Undefined" as the
   display name.
 * Fix more places where we sometimes saw room IDs incorrectly.
 * Fix bug which caused lag when entering text in the text box.
 Changes in synapse 0.3.3 (2014-09-22)
 =====================================
 Homeserver:
 * Fix bug where you continued to get events for rooms you had left.
 Webclient:
 * Add support for video calls with basic UI.
 * Fix bug where one to one chats were named after your display name rather
   than the other person's.
 * Fix bug which caused lag when typing in the textarea.
 * Refuse to run on browsers we know won't work.
 * Trigger pagination when joining new rooms.
 * Fix bug where we sometimes didn't display invitations in recents.
 * Automatically join room when accepting a VoIP call.
 * Disable outgoing and reject incoming calls on browsers we don't support
   VoIP in.
 * Don't display desktop notifications for messages in the room you are
   non-idle and speaking in.
 Changes in synapse 0.3.2 (2014-09-18)
 =====================================
 Webclient:
 * Fix bug where an empty "bing words" list in old accounts didn't send
   notifications when it should have done.
 Changes in synapse 0.3.1 (2014-09-18)
 =====================================
 This is a release to hotfix v0.3.0 to fix two regressions.
 Webclient:
 * Fix a regression where we sometimes displayed duplicate events.
 * Fix a regression where we didn't immediately remove rooms you were
   banned in from the recents list.
 Changes in synapse 0.3.0 (2014-09-18)
 =====================================
 See UPGRADE for information about changes to the client server API, including
 breaking backwards compatibility with VoIP calls and registration API.
 Homeserver:
 * When a user changes their displayname or avatar the server will now update 
   all their join states to reflect this.
 * The server now adds "age" key to events to indicate how old they are. This
   is clock independent, so at no point does any server or webclient have to
   assume their clock is in sync with everyone else.
 * Fix bug where we didn't correctly pull in missing PDUs.
 * Fix bug where prev_content key wasn't always returned.
 * Add support for password resets.
 Webclient:
 * Improve page content loading.
 * Join/parts now trigger desktop notifications.
 * Always show room aliases in the UI if one is present.
 * No longer show user-count in the recents side panel.
 * Add up & down arrow support to the text box for message sending to step
   through your sent history.
 * Don't display notifications for our own messages.
 * Emotes are now formatted correctly in desktop notifications.
 * The recents list now differentiates between public & private rooms.
 * Fix bug where when switching between rooms the pagination flickered before
   the view jumped to the bottom of the screen.
 * Add bing word support.
 Registration API:
 * The registration API has been overhauled to function like the login API. In
   practice, this means registration requests must now include the following:
   'type':'m.login.password'. See UPGRADE for more information on this.
 * The 'user_id' key has been renamed to 'user' to better match the login API.
 * There is an additional login type: 'm.login.email.identity'.
 * The command client and web client have been updated to reflect these changes.
 Changes in synapse 0.2.3 (2014-09-12)
 =====================================
 Homeserver:
 * Fix bug where we stopped sending events to remote home servers if a
   user from that home server left, even if there were some still in the
   room.
 * Fix bugs in the state conflict resolution where it was incorrectly
   rejecting events.
 Webclient:
 * Display room names and topics.
 * Allow setting/editing of room names and topics.
 * Display information about rooms on the main page.
 * Handle ban and kick events in real time.
 * VoIP UI and reliability improvements.
 * Add glare support for VoIP.
 * Improvements to initial startup speed.
 * Don't display duplicate join events.
 * Local echo of messages.
 * Differentiate sending and sent of local echo.
 * Various minor bug fixes.
 Changes in synapse 0.2.2 (2014-09-06)
 =====================================
 Homeserver:
 * When the server returns state events it now also includes the previous 
   content.
 * Add support for inviting people when creating a new room.
 * Make the homeserver inform the room via `m.room.aliases` when a new alias
   is added for a room.
 * Validate `m.room.power_level` events.
 Webclient:
 * Add support for captchas on registration.
 * Handle `m.room.aliases` events.
 * Asynchronously send messages and show a local echo.
 * Inform the UI when a message failed to send.
 * Only autoscroll on receiving a new message if the user was already at the 
   bottom of the screen.
 * Add support for ban/kick reasons.
 Changes in synapse 0.2.1 (2014-09-03)
 =====================================
 Homeserver:
 * Added support for signing up with a third party id.
 * Add synctl scripts.
 * Added rate limiting.
 * Add option to change the external address the content repo uses.
 * Presence bug fixes.
 Webclient:
 * Added support for signing up with a third party id.
 * Added support for banning and kicking users.
 * Added support for displaying and setting ops.
 * Added support for room names.
 * Fix bugs with room membership event display.
 Changes in synapse 0.2.0 (2014-09-02)
 =====================================
 This update changes many configuration options, updates the
 database schema and mandates SSL for server-server connections.
 Homeserver:
 * Require SSL for server-server connections.
 * Add SSL listener for client-server connections.
 * Add ability to use config files.
 * Add support for kicking/banning and power levels.
 * Allow setting of room names and topics on creation.
 * Change presence to include last seen time of the user.
 * Change url path prefix to /_matrix/...
 * Bug fixes to presence.
 Webclient:
 * Reskin the CSS for registration and login.
 * Various improvements to rooms CSS.
 * Support changes in client-server API.
 * Bug fixes to VOIP UI.
 * Various bug fixes to handling of changes to room member list.
 Changes in synapse 0.1.2 (2014-08-29)
 =====================================
 Webclient:
 * Add basic call state UI for VoIP calls.
 Changes in synapse 0.1.1 (2014-08-29)
 =====================================
 Homeserver:
    * Fix bug that caused the event stream to not notify some clients about
      changes.
 Changes in synapse 0.1.0 (2014-08-29)
 =====================================
 Presence has been reenabled in this release.
 Homeserver:
 * Update client to server API, including:
    - Use a more consistent url scheme.
    - Provide more useful information in the initial sync api.
 * Change the presence handling to be much more efficient.
 * Change the presence server to server API to not require explicit polling of
   all users who share a room with a user.
 * Fix races in the event streaming logic.
 Webclient:
 * Update to use new client to server API.
 * Add basic VOIP support.
 * Add idle timers that change your status to away.
 * Add recent rooms column when viewing a room.
 * Various network efficiency improvements.
 * Add basic mobile browser support.
 * Add a settings page.
 Changes in synapse 0.0.1 (2014-08-22)
 =====================================
 Presence has been disabled in this release due to a bug that caused the
 homeserver to spam other remote homeservers.
 Homeserver:
 * Completely change the database schema to support generic event types.
 * Improve presence reliability.
 * Improve reliability of joining remote rooms.
 * Fix bug where room join events were duplicated.
 * Improve initial sync API to return more information to the client.
 * Stop generating fake messages for room membership events.
 Webclient:
 * Add tab completion of names.
 * Add ability to upload and send images.
 * Add profile pages.
 * Improve CSS layout of room.
 * Disambiguate identical display names.
 * Don't get remote users display names and avatars individually.
 * Use the new initial sync API to reduce number of round trips to the homeserver.
 * Change url scheme to use room aliases instead of room ids where known.
 * Increase longpoll timeout.
 Changes in synapse 0.0.0 (2014-08-13)
 =====================================
 * Initial alpha release
--- a/CONTRIBUTING.rst
+++ b/CONTRIBUTING.rst
@@ -1,198 +0,0 @@
 Contributing code to Matrix
 ===========================
 Everyone is welcome to contribute code to Matrix
 (https://github.com/matrix-org), provided that they are willing to license
 their contributions under the same license as the project itself. We follow a
 simple 'inbound=outbound' model for contributions: the act of submitting an
 'inbound' contribution means that the contributor agrees to license the code
 under the same terms as the project's overall 'outbound' license - in our
 case, this is almost always Apache Software License v2 (see LICENSE).
 How to contribute
 ~~~~~~~~~~~~~~~~~
 The preferred and easiest way to contribute changes to Matrix is to fork the
 relevant project on github, and then create a pull request to ask us to pull
 your changes into our repo
 (https://help.github.com/articles/using-pull-requests/)
 **The single biggest thing you need to know is: please base your changes on
 the develop branch - /not/ master.**
 We use the master branch to track the most recent release, so that folks who
 blindly clone the repo and automatically check out master get something that
 works. Develop is the unstable branch where all the development actually
 happens: the workflow is that contributors should fork the develop branch to
 make a 'feature' branch for a particular contribution, and then make a pull
 request to merge this back into the matrix.org 'official' develop branch. We
 use github's pull request workflow to review the contribution, and either ask
 you to make any refinements needed or merge it and make them ourselves. The
 changes will then land on master when we next do a release.
 We use `Buildkite <https://buildkite.com/matrix-dot-org/synapse>`_ for
 continuous integration.  Buildkite builds need to be authorised by a
 maintainer. If your change breaks the build, this will be shown in GitHub, so
 please keep an eye on the pull request for feedback.
 To run unit tests in a local development environment, you can use:
 - ``tox -e py35`` (requires tox to be installed by ``pip install tox``)
  for SQLite-backed Synapse on Python 3.5.
 - ``tox -e py36`` for SQLite-backed Synapse on Python 3.6.
 - ``tox -e py36-postgres`` for PostgreSQL-backed Synapse on Python 3.6
  (requires a running local PostgreSQL with access to create databases).
 - ``./test_postgresql.sh`` for PostgreSQL-backed Synapse on Python 3.5
  (requires Docker). Entirely self-contained, recommended if you don't want to
  set up PostgreSQL yourself.
 Docker images are available for running the integration tests (SyTest) locally,
 see the `documentation in the SyTest repo
 <https://github.com/matrix-org/sytest/blob/develop/docker/README.md>`_ for more
 information.
 Code style
 ~~~~~~~~~~
 All Matrix projects have a well-defined code-style - and sometimes we've even
 got as far as documenting it... For instance, synapse's code style doc lives
 at https://github.com/matrix-org/synapse/tree/master/docs/code_style.rst.
 Please ensure your changes match the cosmetic style of the existing project,
 and **never** mix cosmetic and functional changes in the same commit, as it
 makes it horribly hard to review otherwise.
 Changelog
 ~~~~~~~~~
 All changes, even minor ones, need a corresponding changelog / newsfragment
 entry. These are managed by Towncrier
 (https://github.com/hawkowl/towncrier).
 To create a changelog entry, make a new file in the ``changelog.d`` file named
 in the format of ``PRnumber.type``. The type can be one of the following:
 * ``feature``.
 * ``bugfix``.
 * ``docker`` (for updates to the Docker image).
 * ``doc`` (for updates to the documentation).
 * ``removal`` (also used for deprecations).
 * ``misc`` (for internal-only changes).
 The content of the file is your changelog entry, which should be a short
 description of your change in the same style as the rest of our `changelog
 <https://github.com/matrix-org/synapse/blob/master/CHANGES.md>`_. The file can
 contain Markdown formatting, and should end with a full stop ('.') for
 consistency.
 Adding credits to the changelog is encouraged, we value your
 contributions and would like to have you shouted out in the release notes!
 For example, a fix in PR #1234 would have its changelog entry in
 ``changelog.d/1234.bugfix``, and contain content like "The security levels of
 Florbs are now validated when recieved over federation. Contributed by Jane
 Matrix.".
 Debian changelog
 ----------------
 Changes which affect the debian packaging files (in ``debian``) are an
 exception.
 In this case, you will need to add an entry to the debian changelog for the
 next release. For this, run the following command::
  dch
 This will make up a new version number (if there isn't already an unreleased
 version in flight), and open an editor where you can add a new changelog entry.
 (Our release process will ensure that the version number and maintainer name is
 corrected for the release.)
 If your change affects both the debian packaging *and* files outside the debian
 directory, you will need both a regular newsfragment *and* an entry in the
 debian changelog. (Though typically such changes should be submitted as two
 separate pull requests.)
 Attribution
 ~~~~~~~~~~~
 Everyone who contributes anything to Matrix is welcome to be listed in the
 AUTHORS.rst file for the project in question. Please feel free to include a
 change to AUTHORS.rst in your pull request to list yourself and a short
 description of the area(s) you've worked on. Also, we sometimes have swag to
 give away to contributors - if you feel that Matrix-branded apparel is missing
 from your life, please mail us your shipping address to matrix at matrix.org and
 we'll try to fix it :)
 Sign off
 ~~~~~~~~
 In order to have a concrete record that your contribution is intentional
 and you agree to license it under the same terms as the project's license, we've adopted the
 same lightweight approach that the Linux Kernel
 `submitting patches process <https://www.kernel.org/doc/html/latest/process/submitting-patches.html#sign-your-work-the-developer-s-certificate-of-origin>`_, Docker
 (https://github.com/docker/docker/blob/master/CONTRIBUTING.md), and many other
 projects use: the DCO (Developer Certificate of Origin:
 http://developercertificate.org/). This is a simple declaration that you wrote
 the contribution or otherwise have the right to contribute it to Matrix::
    Developer Certificate of Origin
    Version 1.1
    Copyright (C) 2004, 2006 The Linux Foundation and its contributors.
    660 York Street, Suite 102,
    San Francisco, CA 94110 USA
    Everyone is permitted to copy and distribute verbatim copies of this
    license document, but changing it is not allowed.
    Developer's Certificate of Origin 1.1
    By making a contribution to this project, I certify that:
    (a) The contribution was created in whole or in part by me and I
        have the right to submit it under the open source license
        indicated in the file; or
    (b) The contribution is based upon previous work that, to the best
        of my knowledge, is covered under an appropriate open source
        license and I have the right under that license to submit that
        work with modifications, whether created in whole or in part
        by me, under the same open source license (unless I am
        permitted to submit under a different license), as indicated
        in the file; or
    (c) The contribution was provided directly to me by some other
        person who certified (a), (b) or (c) and I have not modified
        it.
    (d) I understand and agree that this project and the contribution
        are public and that a record of the contribution (including all
        personal information I submit with it, including my sign-off) is
        maintained indefinitely and may be redistributed consistent with
        this project or the open source license(s) involved.
 If you agree to this for your contribution, then all that's needed is to
 include the line in your commit or pull request comment::
    Signed-off-by: Your Name <your@email.example.org>
 We accept contributions under a legally identifiable name, such as
 your name on government documentation or common-law names (names
 claimed by legitimate usage or repute). Unfortunately, we cannot
 accept anonymous contributions at this time.
 Git allows you to add this signoff automatically when using the ``-s``
 flag to ``git commit``, which uses the name and email set in your
 ``user.name`` and ``user.email`` git configs.
 Conclusion
 ~~~~~~~~~~
 That's it!  Matrix is a very open and collaborative project as you might expect
 given our obsession with open communication.  If we're going to successfully
 matrix together all the fragmented communication technologies out there we are
 reliant on contributions and collaboration from the community to do so.  So
 please get involved - and we hope you have as much fun hacking on Matrix as we
 do!
--- a/INSTALL.md
+++ b/INSTALL.md
@@ -1,464 +0,0 @@
 - [Choosing your server name](#choosing-your-server-name)
 - [Installing Synapse](#installing-synapse)
  - [Installing from source](#installing-from-source)
    - [Platform-Specific Instructions](#platform-specific-instructions)
    - [Troubleshooting Installation](#troubleshooting-installation)
  - [Prebuilt packages](#prebuilt-packages)
 - [Setting up Synapse](#setting-up-synapse)
  - [TLS certificates](#tls-certificates)
  - [Email](#email)
  - [Registering a user](#registering-a-user)
  - [Setting up a TURN server](#setting-up-a-turn-server)
  - [URL previews](#url-previews)
 # Choosing your server name
 It is important to choose the name for your server before you install Synapse,
 because it cannot be changed later.
 The server name determines the "domain" part of user-ids for users on your
 server: these will all be of the format `@user:my.domain.name`. It also
 determines how other matrix servers will reach yours for federation.
 For a test configuration, set this to the hostname of your server. For a more
 production-ready setup, you will probably want to specify your domain
 (`example.com`) rather than a matrix-specific hostname here (in the same way
 that your email address is probably `user@example.com` rather than
 `user@email.example.com`) - but doing so may require more advanced setup: see
 [Setting up Federation](docs/federate.md).
 # Installing Synapse
 ## Installing from source
 (Prebuilt packages are available for some platforms - see [Prebuilt packages](#prebuilt-packages).)
 System requirements:
 - POSIX-compliant system (tested on Linux & OS X)
 - Python 3.5, 3.6, or 3.7
 - At least 1GB of free RAM if you want to join large public rooms like #matrix:matrix.org
 Synapse is written in Python but some of the libraries it uses are written in
 C. So before we can install Synapse itself we need a working C compiler and the
 header files for Python C extensions. See [Platform-Specific
 Instructions](#platform-specific-instructions) for information on installing
 these on various platforms.
 To install the Synapse homeserver run:
 ```
 mkdir -p ~/synapse
 virtualenv -p python3 ~/synapse/env
 source ~/synapse/env/bin/activate
 pip install --upgrade pip
 pip install --upgrade setuptools
 pip install matrix-synapse
 ```
 This will download Synapse from [PyPI](https://pypi.org/project/matrix-synapse)
 and install it, along with the python libraries it uses, into a virtual environment
 under `~/synapse/env`.  Feel free to pick a different directory if you
 prefer.
 This Synapse installation can then be later upgraded by using pip again with the
 update flag:
 ```
 source ~/synapse/env/bin/activate
 pip install -U matrix-synapse
 ```
 Before you can start Synapse, you will need to generate a configuration
 file. To do this, run (in your virtualenv, as before)::
 ```
 cd ~/synapse
 python -m synapse.app.homeserver \
    --server-name my.domain.name \
    --config-path homeserver.yaml \
    --generate-config \
    --report-stats=[yes|no]
 ```
 ... substituting an appropriate value for `--server-name`.
 This command will generate you a config file that you can then customise, but it will
 also generate a set of keys for you. These keys will allow your Home Server to
 identify itself to other Home Servers, so don't lose or delete them. It would be
 wise to back them up somewhere safe. (If, for whatever reason, you do need to
 change your Home Server's keys, you may find that other Home Servers have the
 old key cached. If you update the signing key, you should change the name of the
 key in the `<server name>.signing.key` file (the second word) to something
 different. See the
 [spec](https://matrix.org/docs/spec/server_server/latest.html#retrieving-server-keys)
 for more information on key management.)
 To actually run your new homeserver, pick a working directory for Synapse to
 run (e.g. `~/synapse`), and::
    cd ~/synapse
    source env/bin/activate
    synctl start
 ### Platform-Specific Instructions
 #### Debian/Ubuntu/Raspbian
 Installing prerequisites on Ubuntu or Debian:
 ```
 sudo apt-get install build-essential python3-dev libffi-dev \
                     python-pip python-setuptools sqlite3 \
                     libssl-dev python-virtualenv libjpeg-dev libxslt1-dev
 ```
 #### ArchLinux
 Installing prerequisites on ArchLinux:
 ```
 sudo pacman -S base-devel python python-pip \
               python-setuptools python-virtualenv sqlite3
 ```
 #### CentOS/Fedora
 Installing prerequisites on CentOS 7 or Fedora 25:
 ```
 sudo yum install libtiff-devel libjpeg-devel libzip-devel freetype-devel \
                 lcms2-devel libwebp-devel tcl-devel tk-devel redhat-rpm-config \
                 python-virtualenv libffi-devel openssl-devel
 sudo yum groupinstall "Development Tools"
 ```
 #### Mac OS X
 Installing prerequisites on Mac OS X:
 ```
 xcode-select --install
 sudo easy_install pip
 sudo pip install virtualenv
 brew install pkg-config libffi
 ```
 #### OpenSUSE
 Installing prerequisites on openSUSE:
 ```
 sudo zypper in -t pattern devel_basis
 sudo zypper in python-pip python-setuptools sqlite3 python-virtualenv \
               python-devel libffi-devel libopenssl-devel libjpeg62-devel
 ```
 #### OpenBSD
 Installing prerequisites on OpenBSD:
 ```
 doas pkg_add python libffi py-pip py-setuptools sqlite3 py-virtualenv \
              libxslt jpeg
 ```
 There is currently no port for OpenBSD. Additionally, OpenBSD's security
 settings require a slightly more difficult installation process.
 XXX: I suspect this is out of date.
 1. Create a new directory in `/usr/local` called `_synapse`. Also, create a
   new user called `_synapse` and set that directory as the new user's home.
   This is required because, by default, OpenBSD only allows binaries which need
   write and execute permissions on the same memory space to be run from
   `/usr/local`.
 2. `su` to the new `_synapse` user and change to their home directory.
 3. Create a new virtualenv: `virtualenv -p python2.7 ~/.synapse`
 4. Source the virtualenv configuration located at
   `/usr/local/_synapse/.synapse/bin/activate`. This is done in `ksh` by
   using the `.` command, rather than `bash`'s `source`.
 5. Optionally, use `pip` to install `lxml`, which Synapse needs to parse
   webpages for their titles.
 6. Use `pip` to install this repository: `pip install matrix-synapse`
 7. Optionally, change `_synapse`'s shell to `/bin/false` to reduce the
   chance of a compromised Synapse server being used to take over your box.
 After this, you may proceed with the rest of the install directions.
 #### Windows
 If you wish to run or develop Synapse on Windows, the Windows Subsystem For
 Linux provides a Linux environment on Windows 10 which is capable of using the
 Debian, Fedora, or source installation methods. More information about WSL can
 be found at https://docs.microsoft.com/en-us/windows/wsl/install-win10 for
 Windows 10 and https://docs.microsoft.com/en-us/windows/wsl/install-on-server
 for Windows Server.
 ### Troubleshooting Installation
 XXX a bunch of this is no longer relevant.
 Synapse requires pip 8 or later, so if your OS provides too old a version you
 may need to manually upgrade it::
    sudo pip install --upgrade pip
 Installing may fail with `Could not find any downloads that satisfy the requirement pymacaroons-pynacl (from matrix-synapse==0.12.0)`.
 You can fix this by manually upgrading pip and virtualenv::
    sudo pip install --upgrade virtualenv
 You can next rerun `virtualenv -p python3 synapse` to update the virtual env.
 Installing may fail during installing virtualenv with `InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.`
 You can fix this  by manually installing ndg-httpsclient::
    pip install --upgrade ndg-httpsclient
 Installing may fail with `mock requires setuptools>=17.1. Aborting installation`.
 You can fix this by upgrading setuptools::
    pip install --upgrade setuptools
 If pip crashes mid-installation for reason (e.g. lost terminal), pip may
 refuse to run until you remove the temporary installation directory it
 created. To reset the installation::
    rm -rf /tmp/pip_install_matrix
 pip seems to leak *lots* of memory during installation.  For instance, a Linux
 host with 512MB of RAM may run out of memory whilst installing Twisted.  If this
 happens, you will have to individually install the dependencies which are
 failing, e.g.::
    pip install twisted
 ## Prebuilt packages
 As an alternative to installing from source, prebuilt packages are available
 for a number of platforms.
 ### Docker images and Ansible playbooks
 There is an offical synapse image available at
 https://hub.docker.com/r/matrixdotorg/synapse which can be used with
 the docker-compose file available at [contrib/docker](contrib/docker). Further information on
 this including configuration options is available in the README on
 hub.docker.com.
 Alternatively, Andreas Peters (previously Silvio Fricke) has contributed a
 Dockerfile to automate a synapse server in a single Docker image, at
 https://hub.docker.com/r/avhost/docker-matrix/tags/
 Slavi Pantaleev has created an Ansible playbook,
 which installs the offical Docker image of Matrix Synapse
 along with many other Matrix-related services (Postgres database, riot-web, coturn, mxisd, SSL support, etc.).
 For more details, see
 https://github.com/spantaleev/matrix-docker-ansible-deploy
 ### Debian/Ubuntu
 #### Matrix.org packages
 Matrix.org provides Debian/Ubuntu packages of the latest stable version of
 Synapse via https://packages.matrix.org/debian/. They are available for Debian
 9 (Stretch), Ubuntu 16.04 (Xenial), and later. To use them:
 ```
 sudo apt install -y lsb-release wget apt-transport-https
 sudo wget -O /usr/share/keyrings/matrix-org-archive-keyring.gpg https://packages.matrix.org/debian/matrix-org-archive-keyring.gpg
 echo "deb [signed-by=/usr/share/keyrings/matrix-org-archive-keyring.gpg] https://packages.matrix.org/debian/ $(lsb_release -cs) main" |
    sudo tee /etc/apt/sources.list.d/matrix-org.list
 sudo apt update
 sudo apt install matrix-synapse-py3
 ```
 **Note**: if you followed a previous version of these instructions which
 recommended using `apt-key add` to add an old key from
 `https://matrix.org/packages/debian/`, you should note that this key has been
 revoked. You should remove the old key with `sudo apt-key remove
 C35EB17E1EAE708E6603A9B3AD0592FE47F0DF61`, and follow the above instructions to
 update your configuration.
 The fingerprint of the repository signing key (as shown by `gpg
 /usr/share/keyrings/matrix-org-archive-keyring.gpg`) is
 `AAF9AE843A7584B5A3E4CD2BCF45A512DE2DA058`.
 #### Downstream Debian/Ubuntu packages
 For `buster` and `sid`, Synapse is available in the Debian repositories and
 it should be possible to install it with simply:
 ```
    sudo apt install matrix-synapse
 ```
 There is also a version of `matrix-synapse` in `stretch-backports`. Please see
 the [Debian documentation on
 backports](https://backports.debian.org/Instructions/) for information on how
 to use them.
 We do not recommend using the packages in downstream Ubuntu at this time, as
 they are old and suffer from known security vulnerabilities.
 ### Fedora
 Synapse is in the Fedora repositories as `matrix-synapse`:
 ```
 sudo dnf install matrix-synapse
 ```
 Oleg Girko provides Fedora RPMs at
 https://obs.infoserver.lv/project/monitor/matrix-synapse
 ### OpenSUSE
 Synapse is in the OpenSUSE repositories as `matrix-synapse`:
 ```
 sudo zypper install matrix-synapse
 ```
 ### SUSE Linux Enterprise Server
 Unofficial package are built for SLES 15 in the openSUSE:Backports:SLE-15 repository at
 https://download.opensuse.org/repositories/openSUSE:/Backports:/SLE-15/standard/
 ### ArchLinux
 The quickest way to get up and running with ArchLinux is probably with the community package
 https://www.archlinux.org/packages/community/any/matrix-synapse/, which should pull in most of
 the necessary dependencies.
 pip may be outdated (6.0.7-1 and needs to be upgraded to 6.0.8-1 ):
 ```
 sudo pip install --upgrade pip
 ```
 If you encounter an error with lib bcrypt causing an Wrong ELF Class:
 ELFCLASS32 (x64 Systems), you may need to reinstall py-bcrypt to correctly
 compile it under the right architecture. (This should not be needed if
 installing under virtualenv):
 ```
 sudo pip uninstall py-bcrypt
 sudo pip install py-bcrypt
 ```
 ### FreeBSD
 Synapse can be installed via FreeBSD Ports or Packages contributed by Brendan Molloy from:
 - Ports: `cd /usr/ports/net-im/py-matrix-synapse && make install clean`
 - Packages: `pkg install py27-matrix-synapse`
 ### NixOS
 Robin Lambertz has packaged Synapse for NixOS at:
 https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/services/misc/matrix-synapse.nix
 # Setting up Synapse
 Once you have installed synapse as above, you will need to configure it.
 ## TLS certificates
 The default configuration exposes a single HTTP port: http://localhost:8008. It
 is suitable for local testing, but for any practical use, you will either need
 to enable a reverse proxy, or configure Synapse to expose an HTTPS port.
 For information on using a reverse proxy, see
 [docs/reverse_proxy.rst](docs/reverse_proxy.rst).
 To configure Synapse to expose an HTTPS port, you will need to edit
 `homeserver.yaml`, as follows:
 * First, under the `listeners` section, uncomment the configuration for the
  TLS-enabled listener. (Remove the hash sign (`#`) at the start of
  each line). The relevant lines are like this:
  ```
    - port: 8448
      type: http
      tls: true
      resources:
        - names: [client, federation]
  ```
 * You will also need to uncomment the `tls_certificate_path` and
  `tls_private_key_path` lines under the `TLS` section. You can either
  point these settings at an existing certificate and key, or you can
  enable Synapse's built-in ACME (Let's Encrypt) support. Instructions
  for having Synapse automatically provision and renew federation
  certificates through ACME can be found at [ACME.md](docs/ACME.md). If you
  are using your own certificate, be sure to use a `.pem` file that includes
  the full certificate chain including any intermediate certificates (for
  instance, if using certbot, use `fullchain.pem` as your certificate, not
  `cert.pem`).
 For a more detailed guide to configuring your server for federation, see
 [federate.md](docs/federate.md)
 ## Email
 It is desirable for Synapse to have the capability to send email. For example,
 this is required to support the 'password reset' feature.
 To configure an SMTP server for Synapse, modify the configuration section
 headed ``email``, and be sure to have at least the ``smtp_host``, ``smtp_port``
 and ``notif_from`` fields filled out. You may also need to set ``smtp_user``,
 ``smtp_pass``, and ``require_transport_security``.
 If Synapse is not configured with an SMTP server, password reset via email will
 be disabled by default.
 ## Registering a user
 The easiest way to create a new user is to do so from a client like [Riot](https://riot.im).
 Alternatively you can do so from the command line if you have installed via pip.
 This can be done as follows:
 ```
 $ source ~/synapse/env/bin/activate
 $ synctl start # if not already running
 $ register_new_matrix_user -c homeserver.yaml http://localhost:8008
 New user localpart: erikj
 Password:
 Confirm password:
 Make admin [no]:
 Success!
 ```
 This process uses a setting `registration_shared_secret` in
 `homeserver.yaml`, which is shared between Synapse itself and the
 `register_new_matrix_user` script. It doesn't matter what it is (a random
 value is generated by `--generate-config`), but it should be kept secret, as
 anyone with knowledge of it can register users, including admin accounts,
 on your server even if `enable_registration` is `false`.
 ## Setting up a TURN server
 For reliable VoIP calls to be routed via this homeserver, you MUST configure
 a TURN server.  See [docs/turn-howto.rst](docs/turn-howto.rst) for details.
 ## URL previews
 Synapse includes support for previewing URLs, which is disabled by default.  To
 turn it on you must enable the `url_preview_enabled: True` config parameter
 and explicitly specify the IP ranges that Synapse is not allowed to spider for
 previewing in the `url_preview_ip_range_blacklist` configuration parameter.
 This is critical from a security perspective to stop arbitrary Matrix users
 spidering 'internal' URLs on your network.  At the very least we recommend that
 your loopback and RFC1918 IP addresses are blacklisted.
 This also requires the optional lxml and netaddr python dependencies to be
 installed.  This in turn requires the libxml2 library to be available - on
 Debian/Ubuntu this means `apt-get install libxml2-dev`, or equivalent for
 your OS.
--- a/MANIFEST.in
+++ b/MANIFEST.in
@@ -2,52 +2,13 @@ include synctl
 include LICENSE
 include VERSION
 include *.rst
 include *.md
 include demo/README
 include demo/demo.tls.dh
 include demo/*.py
 include demo/*.sh
 recursive-include synapse/storage/schema *.sql
 recursive-include synapse/storage/schema *.sql.postgres
 recursive-include synapse/storage/schema *.sql.sqlite
 recursive-include synapse/storage/schema *.py
 recursive-include synapse/storage/schema *.txt
 recursive-include demo *.dh
 recursive-include demo *.py
 recursive-include demo *.sh
 recursive-include docs *
 recursive-include scripts *
 recursive-include scripts-dev *
 recursive-include synapse *.pyi
 recursive-include tests *.py
 include tests/http/ca.crt
 include tests/http/ca.key
 include tests/http/server.key
 recursive-include synapse/res *
 recursive-include synapse/static *.css
 recursive-include synapse/static *.gif
 recursive-include synapse/static *.html
 recursive-include synapse/static *.js
 exclude Dockerfile
 exclude .dockerignore
 exclude test_postgresql.sh
 exclude .editorconfig
 exclude sytest-blacklist
 include pyproject.toml
 recursive-include changelog.d *
 prune .buildkite
 prune .circleci
 prune .codecov.yml
 prune .coveragerc
 prune .github
 prune debian
 prune demo/etc
 prune docker
 prune mypy.ini
 prune stubs
 exclude jenkins*
 recursive-exclude jenkins *.sh
--- a/MAP.rst
+++ b/MAP.rst
@@ -0,0 +1,35 @@
 Directory Structure
 ===================
 Warning: this may be a bit stale...
 ::
    .
    ├── cmdclient           Basic CLI python Matrix client
    ├── demo                Scripts for running standalone Matrix demos
    ├── docs                All doc, including the draft Matrix API spec
    │   ├── client-server       The client-server Matrix API spec
    │   ├── model               Domain-specific elements of the Matrix API spec
    │   ├── server-server       The server-server model of the Matrix API spec
    │   └── sphinx              The internal API doc of the Synapse homeserver
    ├── experiments         Early experiments of using Synapse's internal APIs
    ├── graph               Visualisation of Matrix's distributed message store 
    ├── synapse             The reference Matrix homeserver implementation
    │   ├── api                 Common building blocks for the APIs
    │   │   ├── events              Definition of state representation Events 
    │   │   └── streams             Definition of streamable Event objects
    │   ├── app                 The __main__ entry point for the homeserver
    │   ├── crypto              The PKI client/server used for secure federation
    │   │   └── resource            PKI helper objects (e.g. keys)
    │   ├── federation          Server-server state replication logic
    │   ├── handlers            The main business logic of the homeserver
    │   ├── http                Wrappers around Twisted's HTTP server & client
    │   ├── rest                Servlet-style RESTful API
    │   ├── storage             Persistence subsystem (currently only sqlite3)
    │   │   └── schema              sqlite persistence schema
    │   └── util                Synapse-specific utilities
    ├── tests               Unit tests for the Synapse homeserver
    └── webclient           Basic AngularJS Matrix web client
--- a/README.rst
+++ b/README.rst
@@ -1,5 +1,3 @@
 .. contents::
 Introduction
 ============
@@ -11,8 +9,8 @@ VoIP.  The basics you need to know to get up and running are:
  like ``#matrix:matrix.org`` or ``#test:localhost:8448``.
 - Matrix user IDs look like ``@matthew:matrix.org`` (although in the future
-  you will normally refer to yourself and others using a third party identifier
+  you will normally refer to yourself and others using a 3PID: email
-  (3PID): email address, phone number, etc rather than manipulating Matrix user IDs)
+  address, phone number, etc rather than manipulating Matrix user IDs)
 The overall architecture is::
@@ -20,8 +18,8 @@ The overall architecture is::
             https://somewhere.org/_matrix      https://elsewhere.net/_matrix
 ``#matrix:matrix.org`` is the official support room for Matrix, and can be
-accessed by any client from https://matrix.org/docs/projects/try-matrix-now.html or
+accessed by the web client at http://matrix.org/alpha or via an IRC bridge at
-via IRC bridge at irc://irc.freenode.net/matrix.
+irc://irc.freenode.net/matrix.
 Synapse is currently in rapid development, but as of version 0.5 we believe it 
 is sufficiently stable to be run as an internet-facing service for real usage!
@@ -52,235 +50,236 @@ generation of fully open and interoperable messaging and VoIP apps for the
 internet.
 Synapse is a reference "homeserver" implementation of Matrix from the core
-development team at matrix.org, written in Python/Twisted.  It is intended to
+development team at matrix.org, written in Python/Twisted for clarity and
-showcase the concept of Matrix and let folks see the spec in the context of a
+simplicity.  It is intended to showcase the concept of Matrix and let folks see
-codebase and let you run your own homeserver and generally help bootstrap the
+the spec in the context of a codebase and let you run your own homeserver and
-ecosystem.
+generally help bootstrap the ecosystem.
 In Matrix, every user runs one or more Matrix clients, which connect through to
-a Matrix homeserver. The homeserver stores all their personal chat history and
+a Matrix homeserver which stores all their personal chat history and user
-user account information - much as a mail client connects through to an
+account information - much as a mail client connects through to an IMAP/SMTP
-IMAP/SMTP server. Just like email, you can either run your own Matrix
+server. Just like email, you can either run your own Matrix homeserver and
-homeserver and control and own your own communications and history or use one
+control and own your own communications and history or use one hosted by
-hosted by someone else (e.g. matrix.org) - there is no single point of control
+someone else (e.g. matrix.org) - there is no single point of control or
-or mandatory service provider in Matrix, unlike WhatsApp, Facebook, Hangouts,
+mandatory service provider in Matrix, unlike WhatsApp, Facebook, Hangouts, etc.
 etc.
-We'd like to invite you to join #matrix:matrix.org (via
+Synapse ships with two basic demo Matrix clients: webclient (a basic group chat
-https://matrix.org/docs/projects/try-matrix-now.html), run a homeserver, take a look
+web client demo implemented in AngularJS) and cmdclient (a basic Python
-at the `Matrix spec <https://matrix.org/docs/spec>`_, and experiment with the
+command line utility which lets you easily see what the JSON APIs are up to).
-`APIs <https://matrix.org/docs/api>`_ and `Client SDKs
+
-<https://matrix.org/docs/projects/try-matrix-now.html#client-sdks>`_.
+Meanwhile, iOS and Android SDKs and clients are currently in development and available from:
 - https://github.com/matrix-org/matrix-ios-sdk
 - https://github.com/matrix-org/matrix-android-sdk
 We'd like to invite you to join #matrix:matrix.org (via http://matrix.org/alpha), run a homeserver, take a look at the Matrix spec at
 http://matrix.org/docs/spec, experiment with the APIs and the demo
 clients, and report any bugs via http://matrix.org/jira.
 Thanks for using Matrix!
-[1] End-to-end encryption is currently in beta: `blog post <https://matrix.org/blog/2016/11/21/matrixs-olm-end-to-end-encryption-security-assessment-released-and-implemented-cross-platform-on-riot-at-last>`_.
+[1] End-to-end encryption is currently in development
 Homeserver Installation
 =======================
-Synapse Installation
+System requirements:
-====================
+- POSIX-compliant system (tested on Linux & OSX)
 - Python 2.7
-.. _federation:
+Synapse is written in python but some of the libraries is uses are written in
 C. So before we can install synapse itself we need a working C compiler and the
 header files for python C extensions.
-* For details on how to install synapse, see `<INSTALL.md>`_.
+Installing prerequisites on Ubuntu or Debian::
 * For specific details on how to configure Synapse for federation see `docs/federate.md <docs/federate.md>`_
    $ sudo apt-get install build-essential python2.7-dev libffi-dev \
                           python-pip python-setuptools sqlite3 \
                           libssl-dev python-virtualenv libjpeg-dev
-Connecting to Synapse from a client
+Installing prerequisites on ArchLinux::
 ===================================
-The easiest way to try out your new Synapse installation is by connecting to it
+    $ sudo pacman -S base-devel python2 python-pip \
-from a web client.
+                     python-setuptools python-virtualenv sqlite3
-Unless you are running a test instance of Synapse on your local machine, in
+Installing prerequisites on Mac OS X::
 general, you will need to enable TLS support before you can successfully
 connect from a client: see `<INSTALL.md#tls-certificates>`_.
-An easy way to get started is to login or register via Riot at
+    $ xcode-select --install
-https://riot.im/app/#/login or https://riot.im/app/#/register respectively.
+    $ sudo pip install virtualenv
 You will need to change the server you are logging into from ``matrix.org``
 and instead specify a Homeserver URL of ``https://<server_name>:8448``
 (or just ``https://<server_name>`` if you are using a reverse proxy).
 (Leave the identity server as the default - see `Identity servers`_.)
 If you prefer to use another client, refer to our
 `client breakdown <https://matrix.org/docs/projects/clients-matrix>`_.
-If all goes well you should at least be able to log in, create a room, and
+To install the synapse homeserver run::
 start sending messages.
-.. _`client-user-reg`:
+    $ virtualenv ~/.synapse
    $ source ~/.synapse/bin/activate
    $ pip install --process-dependency-links https://github.com/matrix-org/synapse/tarball/master
-Registering a new user from a client
+This installs synapse, along with the libraries it uses, into a virtual
------------------------------------
+environment under ``~/.synapse``.
-By default, registration of new users via Matrix clients is disabled. To enable
+To set up your homeserver, run (in your virtualenv, as before)::
 it, specify ``enable_registration: true`` in ``homeserver.yaml``. (It is then
 recommended to also set up CAPTCHA - see `<docs/CAPTCHA_SETUP.rst>`_.)
-Once ``enable_registration`` is set to ``true``, it is possible to register a
+    $ cd ~/.synapse
-user via `riot.im <https://riot.im/app/#/register>`_ or other Matrix clients.
+    $ python -m synapse.app.homeserver \
        --server-name machine.my.domain.name \
        --config-path homeserver.yaml \
        --generate-config
-Your new user name will be formed partly from the ``server_name``, and partly
+Substituting your host and domain name as appropriate.
 from a localpart you specify when you create the account. Your name will take
 the form of::
-    @localpart:my.domain.name
+For reliable VoIP calls to be routed via this homeserver, you MUST configure
 a TURN server.  See docs/turn-howto.rst for details.
-(pronounced "at localpart on my dot domain dot name").
+Troubleshooting Installation
 ----------------------------
-As when logging in, you will need to specify a "Custom server".  Specify your
+Synapse requires pip 1.7 or later, so if your OS provides too old a version and 
-desired ``localpart`` in the 'User name' box.
+you get errors about ``error: no such option: --process-dependency-links`` you 
 may need to manually upgrade it::
-ACME setup
+    $ sudo pip install --upgrade pip
 ==========
-For details on having Synapse manage your federation TLS certificates
+If pip crashes mid-installation for reason (e.g. lost terminal), pip may
-automatically, please see `<docs/ACME.md>`_.
+refuse to run until you remove the temporary installation directory it
 created. To reset the installation::
    $ rm -rf /tmp/pip_install_matrix
-Security Note
+pip seems to leak *lots* of memory during installation.  For instance, a Linux 
-=============
+host with 512MB of RAM may run out of memory whilst installing Twisted.  If this 
 happens, you will have to individually install the dependencies which are 
 failing, e.g.::
-Matrix serves raw user generated data in some APIs - specifically the `content
+    $ pip install twisted
 repository endpoints <https://matrix.org/docs/spec/client_server/latest.html#get-matrix-media-r0-download-servername-mediaid>`_.
-Whilst we have tried to mitigate against possible XSS attacks (e.g.
+On OSX, if you encounter clang: error: unknown argument: '-mno-fused-madd' you
-https://github.com/matrix-org/synapse/pull/1021) we recommend running
+will need to export CFLAGS=-Qunused-arguments.
 matrix homeservers on a dedicated domain name, to limit any malicious user generated
 content served to web browsers a matrix API from being able to attack webapps hosted
 on the same domain.  This is particularly true of sharing a matrix webclient and
 server on the same domain.
-See https://github.com/vector-im/riot-web/issues/1977 and
+ArchLinux
-https://developer.github.com/changes/2014-04-25-user-content-security for more details.
+---------
 Installation on ArchLinux may encounter a few hiccups as Arch defaults to
 python 3, but synapse currently assumes python 2.7 by default.
-Upgrading an existing Synapse
+pip may be outdated (6.0.7-1 and needs to be upgraded to 6.0.8-1 )::
 =============================
-The instructions for upgrading synapse are in `UPGRADE.rst`_.
+    $ sudo pip2.7 install --upgrade pip
 Please check these instructions as upgrading may require extra steps for some
 versions of synapse.
-.. _UPGRADE.rst: UPGRADE.rst
+You also may need to explicitly specify python 2.7 again during the install
 request::
    $ pip2.7 install --process-dependency-links \
        https://github.com/matrix-org/synapse/tarball/master
-Using PostgreSQL
+If you encounter an error with lib bcrypt causing an Wrong ELF Class:
-================
+ELFCLASS32 (x64 Systems), you may need to reinstall py-bcrypt to correctly
 compile it under the right architecture. (This should not be needed if
 installing under virtualenv)::
-Synapse offers two database engines:
+    $ sudo pip2.7 uninstall py-bcrypt
- * `SQLite <https://sqlite.org/>`_
+    $ sudo pip2.7 install py-bcrypt
 * `PostgreSQL <https://www.postgresql.org>`_
-By default Synapse uses SQLite in and doing so trades performance for convenience.
+During setup of homeserver you need to call python2.7 directly again::
 SQLite is only recommended in Synapse for testing purposes or for servers with
 light workloads.
-Almost all installations should opt to use PostreSQL. Advantages include:
+    $ cd ~/.synapse
    $ python2.7 -m synapse.app.homeserver \
      --server-name machine.my.domain.name \
      --config-path homeserver.yaml \
      --generate-config
-* significant performance improvements due to the superior threading and
+...substituting your host and domain name as appropriate.
  caching model, smarter query optimiser
 * allowing the DB to be run on separate hardware
 * allowing basic active/backup high-availability with a "hot spare" synapse
  pointing at the same DB master, as well as enabling DB replication in
  synapse itself.
-For information on how to install and use PostgreSQL, please see
+Windows Install
-`docs/postgres.rst <docs/postgres.rst>`_.
+---------------
 Synapse can be installed on Cygwin. It requires the following Cygwin packages:
-.. _reverse-proxy:
+ - gcc
 - git
 - libffi-devel
 - openssl (and openssl-devel, python-openssl)
 - python
 - python-setuptools
-Using a reverse proxy with Synapse
+The content repository requires additional packages and will be unable to process
-==================================
+uploads without them:
 - libjpeg8
 - libjpeg8-devel
 - zlib
 If you choose to install Synapse without these packages, you will need to reinstall
 ``pillow`` for changes to be applied, e.g. ``pip uninstall pillow`` ``pip install
 pillow --user``
-It is recommended to put a reverse proxy such as
+Troubleshooting:
 `nginx <https://nginx.org/en/docs/http/ngx_http_proxy_module.html>`_,
 `Apache <https://httpd.apache.org/docs/current/mod/mod_proxy_http.html>`_,
 `Caddy <https://caddyserver.com/docs/proxy>`_ or
 `HAProxy <https://www.haproxy.org/>`_ in front of Synapse. One advantage of
 doing so is that it means that you can expose the default https port (443) to
 Matrix clients without needing to run Synapse with root privileges.
-For information on configuring one, see `<docs/reverse_proxy.rst>`_.
+- You may need to upgrade ``setuptools`` to get this to work correctly:
  ``pip install setuptools --upgrade``.
 - You may encounter errors indicating that ``ffi.h`` is missing, even with
  ``libffi-devel`` installed. If you do, copy the ``.h`` files:
  ``cp /usr/lib/libffi-3.0.13/include/*.h /usr/include``
 - You may need to install libsodium from source in order to install PyNacl. If
  you do, you may need to create a symlink to ``libsodium.a`` so ``ld`` can find
  it: ``ln -s /usr/local/lib/libsodium.a /usr/lib/libsodium.a``
-Identity Servers
+Running Your Homeserver
-================
+=======================
-Identity servers have the job of mapping email addresses and other 3rd Party
+To actually run your new homeserver, pick a working directory for Synapse to run 
-IDs (3PIDs) to Matrix user IDs, as well as verifying the ownership of 3PIDs
+(e.g. ``~/.synapse``), and::
 before creating that mapping.
-**They are not where accounts or credentials are stored - these live on home
+    $ cd ~/.synapse
-servers. Identity Servers are just for mapping 3rd party IDs to matrix IDs.**
+    $ source ./bin/activate
    $ synctl start
-This process is very security-sensitive, as there is obvious risk of spam if it
+Troubleshooting Running
-is too easy to sign up for Matrix accounts or harvest 3PID data. In the longer
+-----------------------
 term, we hope to create a decentralised system to manage it (`matrix-doc #712
 <https://github.com/matrix-org/matrix-doc/issues/712>`_), but in the meantime,
 the role of managing trusted identity in the Matrix ecosystem is farmed out to
 a cluster of known trusted ecosystem partners, who run 'Matrix Identity
 Servers' such as `Sydent <https://github.com/matrix-org/sydent>`_, whose role
 is purely to authenticate and track 3PID logins and publish end-user public
 keys.
-You can host your own copy of Sydent, but this will prevent you reaching other
+If synapse fails with ``missing "sodium.h"`` crypto errors, you may need 
-users in the Matrix ecosystem via their email address, and prevent them finding
+to manually upgrade PyNaCL, as synapse uses NaCl (http://nacl.cr.yp.to/) for 
-you. We therefore recommend that you use one of the centralised identity servers
+encryption and digital signatures.
-at ``https://matrix.org`` or ``https://vector.im`` for now.
+Unfortunately PyNACL currently has a few issues
 (https://github.com/pyca/pynacl/issues/53) and
 (https://github.com/pyca/pynacl/issues/79) that mean it may not install
 correctly, causing all tests to fail with errors about missing "sodium.h". To
 fix try re-installing from PyPI or directly from
 (https://github.com/pyca/pynacl)::
-To reiterate: the Identity server will only be used if you choose to associate
+    $ # Install from PyPI
-an email address with your account, or send an invite to another user via their
+    $ pip install --user --upgrade --force pynacl
-email address.
+    $ # Install from github
    $ pip install --user https://github.com/pyca/pynacl/tarball/master
 ArchLinux
 ---------
-Password reset
+If running `$ synctl start` fails wit 'returned non-zero exit status 1', you will need to explicitly call Python2.7 - either running as::
 ==============
-If a user has registered an email address to their account using an identity
+    $ python2.7 -m synapse.app.homeserver --daemonize -c homeserver.yaml --pid-file homeserver.pid
 server, they can request a password-reset token via clients such as Riot.
-A manual password reset can be done via direct database access as follows.
+...or by editing synctl with the correct python executable.
-First calculate the hash of the new password::
+Homeserver Development
 ======================
-    $ ~/synapse/env/bin/hash_password
+To check out a homeserver for development, clone the git repo into a working
    Password:
    Confirm password:
    $2a$12$xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
 Then update the `users` table in the database::
    UPDATE users SET password_hash='$2a$12$xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
        WHERE name='@test:test.com';
 Synapse Development
 ===================
 Before setting up a development environment for synapse, make sure you have the
 system dependencies (such as the python header files) installed - see
 `Installing from source <INSTALL.md#installing-from-source>`_.
 To check out a synapse for development, clone the git repo into a working
 directory of your choice::
-    git clone https://github.com/matrix-org/synapse.git
+    $ git clone https://github.com/matrix-org/synapse.git
-    cd synapse
+    $ cd synapse
-Synapse has a number of external dependencies, that are easiest
+The homeserver has a number of external dependencies, that are easiest
 to install using pip and a virtualenv::
-    virtualenv -p python3 env
+    $ virtualenv env
-    source env/bin/activate
+    $ source env/bin/activate
-    python -m pip install --no-use-pep517 -e .[all]
+    $ python synapse/python_dependencies.py | xargs -n1 pip install
    $ pip install setuptools_trial mock
 This will run a process of downloading and installing all the needed
 dependencies into a virtual env.
-Once this is done, you may wish to run Synapse's unit tests, to
+Once this is done, you may wish to run the homeserver's unit tests, to
 check that everything is installed as it should be::
-    python -m twisted.trial tests
+    $ python setup.py test
 This should end with a 'PASSED' result::
@@ -288,17 +287,147 @@ This should end with a 'PASSED' result::
    PASSED (successes=143)
 Running the Integration Tests
 =============================
-Synapse is accompanied by `SyTest <https://github.com/matrix-org/sytest>`_,
+Upgrading an existing homeserver
-a Matrix homeserver integration testing suite, which uses HTTP requests to
+================================
-access the API as a Matrix client would. It is able to run Synapse directly from
+
-the source tree, so installation of the server is not required.
+IMPORTANT: Before upgrading an existing homeserver to a new version, please
 refer to UPGRADE.rst for any additional instructions.
 Otherwise, simply re-install the new codebase over the current one - e.g.
 by ``pip install --process-dependency-links
 https://github.com/matrix-org/synapse/tarball/master``
 if using pip, or by ``git pull`` if running off a git working copy.
 Setting up Federation
 =====================
 In order for other homeservers to send messages to your server, it will need to
 be publicly visible on the internet, and they will need to know its host name.
 You have two choices here, which will influence the form of your Matrix user
 IDs:
 1) Use the machine's own hostname as available on public DNS in the form of
   its A or AAAA records. This is easier to set up initially, perhaps for
   testing, but lacks the flexibility of SRV.
 2) Set up a SRV record for your domain name. This requires you create a SRV
   record in DNS, but gives the flexibility to run the server on your own
   choice of TCP port, on a machine that might not be the same name as the
   domain name.
 For the first form, simply pass the required hostname (of the machine) as the
 --server-name parameter::
    $ python -m synapse.app.homeserver \
        --server-name machine.my.domain.name \
        --config-path homeserver.yaml \
        --generate-config
    $ python -m synapse.app.homeserver --config-path homeserver.yaml
 Alternatively, you can run ``synctl start`` to guide you through the process.
 For the second form, first create your SRV record and publish it in DNS. This
 needs to be named _matrix._tcp.YOURDOMAIN, and point at at least one hostname
 and port where the server is running.  (At the current time synapse does not
 support clustering multiple servers into a single logical homeserver).  The DNS
 record would then look something like::
    $ dig -t srv _matrix._tcp.machine.my.domaine.name
    _matrix._tcp    IN      SRV     10 0 8448 machine.my.domain.name.
 At this point, you should then run the homeserver with the hostname of this
 SRV record, as that is the name other machines will expect it to have::
    $ python -m synapse.app.homeserver \
        --server-name YOURDOMAIN \
        --bind-port 8448 \
        --config-path homeserver.yaml \
        --generate-config
    $ python -m synapse.app.homeserver --config-path homeserver.yaml
 You may additionally want to pass one or more "-v" options, in order to
 increase the verbosity of logging output; at least for initial testing.
 For the initial alpha release, the homeserver is not speaking TLS for
 either client-server or server-server traffic for ease of debugging. We have
 also not spent any time yet getting the homeserver to run behind loadbalancers.
 Running a Demo Federation of Homeservers
 ----------------------------------------
 If you want to get up and running quickly with a trio of homeservers in a
 private federation (``localhost:8080``, ``localhost:8081`` and
 ``localhost:8082``) which you can then access through the webclient running at
 http://localhost:8080. Simply run::
    $ demo/start.sh
 This is mainly useful just for development purposes.
 Running The Demo Web Client
 ===========================
 The homeserver runs a web client by default at https://localhost:8448/.
 If this is the first time you have used the client from that browser (it uses
 HTML5 local storage to remember its config), you will need to log in to your
 account. If you don't yet have an account, because you've just started the
 homeserver for the first time, then you'll need to register one.
 Registering A New Account
 -------------------------
 Your new user name will be formed partly from the hostname your server is
 running as, and partly from a localpart you specify when you create the
 account. Your name will take the form of::
    @localpart:my.domain.here
         (pronounced "at localpart on my dot domain dot here")
 Specify your desired localpart in the topmost box of the "Register for an
 account" form, and click the "Register" button. Hostnames can contain ports if
 required due to lack of SRV records (e.g. @matthew:localhost:8448 on an
 internal synapse sandbox running on localhost)
 Logging In To An Existing Account
 ---------------------------------
 Just enter the ``@localpart:my.domain.here`` Matrix user ID and password into
 the form and click the Login button.
 Identity Servers
 ================
 The job of authenticating 3PIDs and tracking which 3PIDs are associated with a
 given Matrix user is very security-sensitive, as there is obvious risk of spam
 if it is too easy to sign up for Matrix accounts or harvest 3PID data.
 Meanwhile the job of publishing the end-to-end encryption public keys for
 Matrix users is also very security-sensitive for similar reasons.
 Therefore the role of managing trusted identity in the Matrix ecosystem is
 farmed out to a cluster of known trusted ecosystem partners, who run 'Matrix
 Identity Servers' such as ``sydent``, whose role is purely to authenticate and
 track 3PID logins and publish end-user public keys.
 It's currently early days for identity servers as Matrix is not yet using 3PIDs
 as the primary means of identity and E2E encryption is not complete. As such,
 we are running a single identity server (http://matrix.org:8090) at the current
 time.
 Where's the spec?!
 ==================
 The source of the matrix spec lives at https://github.com/matrix-org/matrix-doc.  
 A recent HTML snapshot of this lives at http://matrix.org/docs/spec
 Testing with SyTest is recommended for verifying that changes related to the
 Client-Server API are functioning correctly. See the `installation instructions
 <https://github.com/matrix-org/sytest#installing>`_ for details.
 Building Internal API Documentation
 ===================================
@@ -306,78 +435,10 @@ Building Internal API Documentation
 Before building internal API documentation install sphinx and
 sphinxcontrib-napoleon::
-    pip install sphinx
+    $ pip install sphinx
-    pip install sphinxcontrib-napoleon
+    $ pip install sphinxcontrib-napoleon
 Building internal API documentation::
-    python setup.py build_sphinx
+    $ python setup.py build_sphinx
 Troubleshooting
 ===============
 Running out of File Handles
 ---------------------------
 If synapse runs out of file handles, it typically fails badly - live-locking
 at 100% CPU, and/or failing to accept new TCP connections (blocking the
 connecting client).  Matrix currently can legitimately use a lot of file handles,
 thanks to busy rooms like #matrix:matrix.org containing hundreds of participating
 servers.  The first time a server talks in a room it will try to connect
 simultaneously to all participating servers, which could exhaust the available
 file descriptors between DNS queries & HTTPS sockets, especially if DNS is slow
 to respond. (We need to improve the routing algorithm used to be better than
 full mesh, but as of March 2019 this hasn't happened yet).
 If you hit this failure mode, we recommend increasing the maximum number of
 open file handles to be at least 4096 (assuming a default of 1024 or 256).
 This is typically done by editing ``/etc/security/limits.conf``
 Separately, Synapse may leak file handles if inbound HTTP requests get stuck
 during processing - e.g. blocked behind a lock or talking to a remote server etc.
 This is best diagnosed by matching up the 'Received request' and 'Processed request'
 log lines and looking for any 'Processed request' lines which take more than
 a few seconds to execute. Please let us know at #synapse:matrix.org if
 you see this failure mode so we can help debug it, however.
 Help!! Synapse is slow and eats all my RAM/CPU!
 -----------------------------------------------
 First, ensure you are running the latest version of Synapse, using Python 3
 with a PostgreSQL database.
 Synapse's architecture is quite RAM hungry currently - we deliberately
 cache a lot of recent room data and metadata in RAM in order to speed up
 common requests. We'll improve this in the future, but for now the easiest
 way to either reduce the RAM usage (at the risk of slowing things down)
 is to set the almost-undocumented ``SYNAPSE_CACHE_FACTOR`` environment
 variable. The default is 0.5, which can be decreased to reduce RAM usage
 in memory constrained enviroments, or increased if performance starts to
 degrade.
 However, degraded performance due to a low cache factor, common on
 machines with slow disks, often leads to explosions in memory use due
 backlogged requests. In this case, reducing the cache factor will make
 things worse. Instead, try increasing it drastically. 2.0 is a good
 starting value.
 Using `libjemalloc <http://jemalloc.net/>`_ can also yield a significant
 improvement in overall memory use, and especially in terms of giving back
 RAM to the OS. To use it, the library must simply be put in the
 LD_PRELOAD environment variable when launching Synapse. On Debian, this
 can be done by installing the ``libjemalloc1`` package and adding this
 line to ``/etc/default/matrix-synapse``::
    LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libjemalloc.so.1
 This can make a significant difference on Python 2.7 - it's unclear how
 much of an improvement it provides on Python 3.x.
 If you're encountering high CPU use by the Synapse process itself, you
 may be affected by a bug with presence tracking that leads to a
 massive excess of outgoing federation requests (see `discussion
 <https://github.com/matrix-org/synapse/issues/3971>`_). If metrics
 indicate that your server is also issuing far more outgoing federation
 requests than can be accounted for by your users' activity, this is a
 likely cause. The misbehavior can be worked around by setting
 ``use_presence: false`` in the Synapse config file.
--- a/UPGRADE.rst
+++ b/UPGRADE.rst
@@ -1,371 +1,3 @@
 Upgrading Synapse
 =================
 Before upgrading check if any special steps are required to upgrade from the
 what you currently have installed to current version of synapse. The extra
 instructions that may be required are listed later in this document.
 1. If synapse was installed in a virtualenv then activate that virtualenv before
   upgrading. If synapse is installed in a virtualenv in ``~/synapse/env`` then
   run:
   .. code:: bash
       source ~/synapse/env/bin/activate
 2. If synapse was installed using pip then upgrade to the latest version by
   running:
   .. code:: bash
       pip install --upgrade matrix-synapse[all]
       # restart synapse
       synctl restart
   If synapse was installed using git then upgrade to the latest version by
   running:
   .. code:: bash
       # Pull the latest version of the master branch.
       git pull
       # Update synapse and its python dependencies.
       pip install --upgrade .[all]
       # restart synapse
       ./synctl restart
 To check whether your update was successful, you can check the Server header
 returned by the Client-Server API:
 .. code:: bash
    # replace <host.name> with the hostname of your synapse homeserver.
    # You may need to specify a port (eg, :8448) if your server is not
    # configured on port 443.
    curl -kv https://<host.name>/_matrix/client/versions 2>&1 | grep "Server:"
 Upgrading to v1.4.0
 ===================
 Config options
 --------------
 **Note: Registration by email address or phone number will not work in this release unless
 some config options are changed from their defaults.**
 This is due to Synapse v1.4.0 now defaulting to sending registration and password reset tokens
 itself. This is for security reasons as well as putting less reliance on identity servers.
 However, currently Synapse only supports sending emails, and does not have support for
 phone-based password reset or account registration. If Synapse is configured to handle these on
 its own, phone-based password resets and registration will be disabled. For Synapse to send
 emails, the ``email`` block of the config must be filled out. If not, then password resets and
 registration via email will be disabled entirely.
 This release also deprecates the ``email.trust_identity_server_for_password_resets`` option and
 replaces it with the ``account_threepid_delegates`` dictionary. This option defines whether the
 homeserver should delegate an external server (typically an `identity server
 <https://matrix.org/docs/spec/identity_service/r0.2.1>`_) to handle sending password reset or
 registration messages via email and SMS.
 If ``email.trust_identity_server_for_password_resets`` is set to ``true``, and
 ``account_threepid_delegates.email`` is not set, then the first entry in
 ``trusted_third_party_id_servers`` will be used as the account threepid delegate for email.
 This is to ensure compatibility with existing Synapse installs that set up external server
 handling for these tasks before v1.4.0. If ``email.trust_identity_server_for_password_resets``
 is ``true`` and no trusted identity server domains are configured, Synapse will throw an error.
 If ``email.trust_identity_server_for_password_resets`` is ``false`` or absent and a threepid
 type in ``account_threepid_delegates`` is not set to a domain, then Synapse will attempt to
 send password reset and registration messages for that type.
 Email templates
 ---------------
 If you have configured a custom template directory with the ``email.template_dir`` option, be
 aware that there are new templates regarding registration. ``registration.html`` and
 ``registration.txt`` have been added and contain the content that is sent to a client upon
 registering via an email address.
 ``registration_success.html`` and ``registration_failure.html`` are also new HTML templates
 that will be shown to the user when they click the link in their registration emai , either
 showing them a success or failure page (assuming a redirect URL is not configured).
 Synapse will expect these files to exist inside the configured template directory. To view the
 default templates, see `synapse/res/templates
 <https://github.com/matrix-org/synapse/tree/master/synapse/res/templates>`_.
 Upgrading to v1.2.0
 ===================
 Some counter metrics have been renamed, with the old names deprecated. See
 `the metrics documentation <docs/metrics-howto.rst#renaming-of-metrics--deprecation-of-old-names-in-12>`_
 for details.
 Upgrading to v1.1.0
 ===================
 Synapse v1.1.0 removes support for older Python and PostgreSQL versions, as
 outlined in `our deprecation notice <https://matrix.org/blog/2019/04/08/synapse-deprecating-postgres-9-4-and-python-2-x>`_.
 Minimum Python Version
 ----------------------
 Synapse v1.1.0 has a minimum Python requirement of Python 3.5. Python 3.6 or
 Python 3.7 are recommended as they have improved internal string handling,
 significantly reducing memory usage.
 If you use current versions of the Matrix.org-distributed Debian packages or
 Docker images, action is not required.
 If you install Synapse in a Python virtual environment, please see "Upgrading to
 v0.34.0" for notes on setting up a new virtualenv under Python 3.
 Minimum PostgreSQL Version
 --------------------------
 If using PostgreSQL under Synapse, you will need to use PostgreSQL 9.5 or above.
 Please see the
 `PostgreSQL documentation <https://www.postgresql.org/docs/11/upgrading.html>`_
 for more details on upgrading your database.
 Upgrading to v1.0
 =================
 Validation of TLS certificates
 ------------------------------
 Synapse v1.0 is the first release to enforce
 validation of TLS certificates for the federation API. It is therefore
 essential that your certificates are correctly configured. See the `FAQ
 <docs/MSC1711_certificates_FAQ.md>`_ for more information.
 Note, v1.0 installations will also no longer be able to federate with servers
 that have not correctly configured their certificates.
 In rare cases, it may be desirable to disable certificate checking: for
 example, it might be essential to be able to federate with a given legacy
 server in a closed federation. This can be done in one of two ways:-
 * Configure the global switch ``federation_verify_certificates`` to ``false``.
 * Configure a whitelist of server domains to trust via ``federation_certificate_verification_whitelist``.
 See the `sample configuration file <docs/sample_config.yaml>`_
 for more details on these settings.
 Email
 -----
 When a user requests a password reset, Synapse will send an email to the
 user to confirm the request.
 Previous versions of Synapse delegated the job of sending this email to an
 identity server. If the identity server was somehow malicious or became
 compromised, it would be theoretically possible to hijack an account through
 this means.
 Therefore, by default, Synapse v1.0 will send the confirmation email itself. If
 Synapse is not configured with an SMTP server, password reset via email will be
 disabled.
 To configure an SMTP server for Synapse, modify the configuration section
 headed ``email``, and be sure to have at least the ``smtp_host``, ``smtp_port``
 and ``notif_from`` fields filled out. You may also need to set ``smtp_user``,
 ``smtp_pass``, and ``require_transport_security``.
 If you are absolutely certain that you wish to continue using an identity
 server for password resets, set ``trust_identity_server_for_password_resets`` to ``true``.
 See the `sample configuration file <docs/sample_config.yaml>`_
 for more details on these settings.
 New email templates
 ---------------
 Some new templates have been added to the default template directory for the purpose of the
 homeserver sending its own password reset emails. If you have configured a custom
 ``template_dir`` in your Synapse config, these files will need to be added.
 ``password_reset.html`` and ``password_reset.txt`` are HTML and plain text templates
 respectively that contain the contents of what will be emailed to the user upon attempting to
 reset their password via email. ``password_reset_success.html`` and
 ``password_reset_failure.html`` are HTML files that the content of which (assuming no redirect
 URL is set) will be shown to the user after they attempt to click the link in the email sent
 to them.
 Upgrading to v0.99.0
 ====================
 Please be aware that, before Synapse v1.0 is released around March 2019, you
 will need to replace any self-signed certificates with those verified by a
 root CA. Information on how to do so can be found at `the ACME docs
 <docs/ACME.md>`_.
 For more information on configuring TLS certificates see the `FAQ <docs/MSC1711_certificates_FAQ.md>`_.
 Upgrading to v0.34.0
 ====================
 1. This release is the first to fully support Python 3. Synapse will now run on
   Python versions 3.5, or 3.6 (as well as 2.7). We recommend switching to
   Python 3, as it has been shown to give performance improvements.
   For users who have installed Synapse into a virtualenv, we recommend doing
   this by creating a new virtualenv. For example::
       virtualenv -p python3 ~/synapse/env3
       source ~/synapse/env3/bin/activate
       pip install matrix-synapse
   You can then start synapse as normal, having activated the new virtualenv::
       cd ~/synapse
       source env3/bin/activate
       synctl start
   Users who have installed from distribution packages should see the relevant
   package documentation. See below for notes on Debian packages.
   * When upgrading to Python 3, you **must** make sure that your log files are
     configured as UTF-8, by adding ``encoding: utf8`` to the
     ``RotatingFileHandler`` configuration (if you have one) in your
     ``<server>.log.config`` file. For example, if your ``log.config`` file
     contains::
       handlers:
         file:
           class: logging.handlers.RotatingFileHandler
           formatter: precise
           filename: homeserver.log
           maxBytes: 104857600
           backupCount: 10
           filters: [context]
         console:
           class: logging.StreamHandler
           formatter: precise
           filters: [context]
     Then you should update this to be::
       handlers:
         file:
           class: logging.handlers.RotatingFileHandler
           formatter: precise
           filename: homeserver.log
           maxBytes: 104857600
           backupCount: 10
           filters: [context]
           encoding: utf8
         console:
           class: logging.StreamHandler
           formatter: precise
           filters: [context]
     There is no need to revert this change if downgrading to Python 2.
   We are also making available Debian packages which will run Synapse on
   Python 3. You can switch to these packages with ``apt-get install
   matrix-synapse-py3``, however, please read `debian/NEWS
   <https://github.com/matrix-org/synapse/blob/release-v0.34.0/debian/NEWS>`_
   before doing so. The existing ``matrix-synapse`` packages will continue to
   use Python 2 for the time being.
 2. This release removes the ``riot.im`` from the default list of trusted
   identity servers.
   If ``riot.im`` is in your homeserver's list of
   ``trusted_third_party_id_servers``, you should remove it. It was added in
   case a hypothetical future identity server was put there. If you don't
   remove it, users may be unable to deactivate their accounts.
 3. This release no longer installs the (unmaintained) Matrix Console web client
   as part of the default installation. It is possible to re-enable it by
   installing it separately and setting the ``web_client_location`` config
   option, but please consider switching to another client.
 Upgrading to v0.33.7
 ====================
 This release removes the example email notification templates from
 ``res/templates`` (they are now internal to the python package). This should
 only affect you if you (a) deploy your Synapse instance from a git checkout or
 a github snapshot URL, and (b) have email notifications enabled.
 If you have email notifications enabled, you should ensure that
 ``email.template_dir`` is either configured to point at a directory where you
 have installed customised templates, or leave it unset to use the default
 templates.
 Upgrading to v0.27.3
 ====================
 This release expands the anonymous usage stats sent if the opt-in
 ``report_stats`` configuration is set to ``true``. We now capture RSS memory
 and cpu use at a very coarse level. This requires administrators to install
 the optional ``psutil`` python module.
 We would appreciate it if you could assist by ensuring this module is available
 and ``report_stats`` is enabled. This will let us see if performance changes to
 synapse are having an impact to the general community.
 Upgrading to v0.15.0
 ====================
 If you want to use the new URL previewing API (/_matrix/media/r0/preview_url)
 then you have to explicitly enable it in the config and update your dependencies
 dependencies.  See README.rst for details.
 Upgrading to v0.11.0
 ====================
 This release includes the option to send anonymous usage stats to matrix.org,
 and requires that administrators explictly opt in or out by setting the
 ``report_stats`` option to either ``true`` or ``false``.
 We would really appreciate it if you could help our project out by reporting
 anonymized usage statistics from your homeserver. Only very basic aggregate
 data (e.g. number of users) will be reported, but it helps us to track the
 growth of the Matrix community, and helps us to make Matrix a success, as well
 as to convince other networks that they should peer with us.
 Upgrading to v0.9.0
 ===================
 Application services have had a breaking API change in this version.
 They can no longer register themselves with a home server using the AS HTTP API. This
 decision was made because a compromised application service with free reign to register
 any regex in effect grants full read/write access to the home server if a regex of ``.*``
 is used. An attack where a compromised AS re-registers itself with ``.*`` was deemed too
 big of a security risk to ignore, and so the ability to register with the HS remotely has
 been removed.
 It has been replaced by specifying a list of application service registrations in
 ``homeserver.yaml``::
  app_service_config_files: ["registration-01.yaml", "registration-02.yaml"]
 Where ``registration-01.yaml`` looks like::
  url: <String>  # e.g. "https://my.application.service.com"
  as_token: <String>
  hs_token: <String>
  sender_localpart: <String>  # This is a new field which denotes the user_id localpart when using the AS token
  namespaces:
    users:
      - exclusive: <Boolean>
        regex: <String>  # e.g. "@prefix_.*"
    aliases:
      - exclusive: <Boolean>
        regex: <String>
    rooms:
      - exclusive: <Boolean>
        regex: <String>
 Upgrading to v0.8.0
 ===================
--- a/changelog.d/.gitignore
+++ b/changelog.d/.gitignore
@@ -1 +0,0 @@
 !.gitignore
--- a/changelog.d/5633.bugfix
+++ b/changelog.d/5633.bugfix
@@ -1 +0,0 @@
 Don't create broken room when power_level_content_override.users does not contain creator_id.
--- a/changelog.d/5680.misc
+++ b/changelog.d/5680.misc
@@ -1 +0,0 @@
 Lay the groundwork for structured logging output.
--- a/changelog.d/5771.feature
+++ b/changelog.d/5771.feature
@@ -1 +0,0 @@
 Make Opentracing work in worker mode.
--- a/changelog.d/5776.misc
+++ b/changelog.d/5776.misc
@@ -1 +0,0 @@
 Update opentracing docs to use the unified `trace` method.
--- a/changelog.d/5835.feature
+++ b/changelog.d/5835.feature
@@ -1 +0,0 @@
 Add the ability to send registration emails from the homeserver rather than delegating to an identity server.
--- a/changelog.d/5844.misc
+++ b/changelog.d/5844.misc
@@ -1 +0,0 @@
 Retry well-known lookup before the cache expires, giving a grace period where the remote well-known can be down but we still use the old result.
--- a/changelog.d/5845.feature
+++ b/changelog.d/5845.feature
@@ -1 +0,0 @@
 Add an admin API to purge old rooms from the database.
--- a/changelog.d/5850.feature
+++ b/changelog.d/5850.feature
@@ -1 +0,0 @@
 Add retry to well-known lookups if we have recently seen a valid well-known record for the server.
--- a/changelog.d/5852.feature
+++ b/changelog.d/5852.feature
@@ -1 +0,0 @@
 Pass opentracing contexts between servers when transmitting EDUs.
--- a/changelog.d/5853.feature
+++ b/changelog.d/5853.feature
@@ -1 +0,0 @@
 Opentracing for device list updates.
--- a/changelog.d/5855.misc
+++ b/changelog.d/5855.misc
@@ -1 +0,0 @@
 Opentracing for room and e2e keys.
--- a/changelog.d/5856.feature
+++ b/changelog.d/5856.feature
@@ -1 +0,0 @@
 Add a tag recording a request's authenticated entity and corresponding servlet in opentracing.
--- a/changelog.d/5857.bugfix
+++ b/changelog.d/5857.bugfix
@@ -1 +0,0 @@
 Fix database index so that different backup versions can have the same sessions.
--- a/changelog.d/5859.feature
+++ b/changelog.d/5859.feature
@@ -1 +0,0 @@
 Add unstable support for MSC2197 (filtered search requests over federation), in order to allow upcoming room directory query performance improvements.
--- a/changelog.d/5860.misc
+++ b/changelog.d/5860.misc
@@ -1 +0,0 @@
 Remove log line for debugging issue #5407.
--- a/changelog.d/5863.bugfix
+++ b/changelog.d/5863.bugfix
@@ -1 +0,0 @@
 Fix Synapse looking for config options `password_reset_failure_template` and `password_reset_success_template`, when they are actually `password_reset_template_failure_html`, `password_reset_template_success_html`.
--- a/changelog.d/5864.feature
+++ b/changelog.d/5864.feature
@@ -1 +0,0 @@
 Correctly retry all hosts returned from SRV when we fail to connect.
--- a/changelog.d/5868.feature
+++ b/changelog.d/5868.feature
@@ -1 +0,0 @@
 Add `m.require_identity_server` key to `/versions`'s `unstable_features` section.
--- a/changelog.d/5875.misc
+++ b/changelog.d/5875.misc
@@ -1 +0,0 @@
 Deprecate the `trusted_third_party_id_servers` option.
--- a/changelog.d/5876.feature
+++ b/changelog.d/5876.feature
@@ -1 +0,0 @@
 Replace `trust_identity_server_for_password_resets` config option with `account_threepid_delegates`.
--- a/changelog.d/5877.removal
+++ b/changelog.d/5877.removal
@@ -1 +0,0 @@
 Remove shared secret registration from client/r0/register endpoint. Contributed by Awesome Technologies Innovationslabor GmbH.
--- a/changelog.d/5878.feature
+++ b/changelog.d/5878.feature
@@ -1 +0,0 @@
 Add admin API endpoint for setting whether or not a user is a server administrator.
--- a/changelog.d/5885.bugfix
+++ b/changelog.d/5885.bugfix
@@ -1 +0,0 @@
 Fix stack overflow when recovering an appservice which had an outage.
--- a/changelog.d/5886.misc
+++ b/changelog.d/5886.misc
@@ -1 +0,0 @@
 Refactor the Appservice scheduler code.
--- a/changelog.d/5892.misc
+++ b/changelog.d/5892.misc
@@ -1 +0,0 @@
 Compatibility with v2 Identity Service APIs other than /lookup.
--- a/changelog.d/5893.misc
+++ b/changelog.d/5893.misc
@@ -1 +0,0 @@
 Drop some unused tables.
--- a/changelog.d/5894.misc
+++ b/changelog.d/5894.misc
@@ -1 +0,0 @@
 Add missing index on users_in_public_rooms to improve the performance of directory queries.
--- a/changelog.d/5895.feature
+++ b/changelog.d/5895.feature
@@ -1 +0,0 @@
 Add config option to sign remote key query responses with a separate key.
--- a/changelog.d/5896.misc
+++ b/changelog.d/5896.misc
@@ -1 +0,0 @@
 Improve the logging when we have an error when fetching signing keys.
--- a/changelog.d/5897.feature
+++ b/changelog.d/5897.feature
@@ -1 +0,0 @@
 Switch to using the v2 Identity Service `/lookup` API where available, with fallback to v1. (Implements [MSC2134](https://github.com/matrix-org/matrix-doc/pull/2134) plus id_access_token authentication for v2 Identity Service APIs from [MSC2140](https://github.com/matrix-org/matrix-doc/pull/2140)).
--- a/changelog.d/5900.feature
+++ b/changelog.d/5900.feature
@@ -1 +0,0 @@
 Add support for config templating.
--- a/changelog.d/5902.feature
+++ b/changelog.d/5902.feature
@@ -1 +0,0 @@
 Users with the type of "support" or "bot" are no longer required to consent.
--- a/changelog.d/5904.feature
+++ b/changelog.d/5904.feature
@@ -1 +0,0 @@
 Let synctl accept a directory of config files.
--- a/changelog.d/5906.feature
+++ b/changelog.d/5906.feature
@@ -1 +0,0 @@
 Increase max display name size to 256.
--- a/changelog.d/5909.misc
+++ b/changelog.d/5909.misc
@@ -1 +0,0 @@
 Fix error message which referred to public_base_url instead of public_baseurl. Thanks to @aaronraimist for the fix!
--- a/changelog.d/5911.misc
+++ b/changelog.d/5911.misc
@@ -1 +0,0 @@
 Add support for database engine-specific schema deltas, based on file extension.
--- a/changelog.d/5914.feature
+++ b/changelog.d/5914.feature
@@ -1 +0,0 @@
 Add admin API endpoint for getting whether or not a user is a server administrator.
--- a/changelog.d/5915.bugfix
+++ b/changelog.d/5915.bugfix
@@ -1 +0,0 @@
 Fix 404 for thumbnail download when `dynamic_thumbnails` is `false` and the thumbnail was dynamically generated. Fix reported by rkfg.
--- a/changelog.d/5920.bugfix
+++ b/changelog.d/5920.bugfix
@@ -1 +0,0 @@
 Fix a cache-invalidation bug for worker-based deployments.
--- a/changelog.d/5922.misc
+++ b/changelog.d/5922.misc
@@ -1 +0,0 @@
 Update Buildkite pipeline to use plugins instead of buildkite-agent commands.
--- a/changelog.d/5926.misc
+++ b/changelog.d/5926.misc
@@ -1 +0,0 @@
 Add link in sample config to the logging config schema.
--- a/changelog.d/5931.misc
+++ b/changelog.d/5931.misc
@@ -1 +0,0 @@
 Remove unnecessary parentheses in return statements.
--- a/changelog.d/5934.feature
+++ b/changelog.d/5934.feature
@@ -1 +0,0 @@
 Redact events in the database that have been redacted for a month.
--- a/changelog.d/5938.misc
+++ b/changelog.d/5938.misc
@@ -1 +0,0 @@
 Remove unused jenkins/prepare_sytest.sh file.
--- a/changelog.d/5940.feature
+++ b/changelog.d/5940.feature
@@ -1 +0,0 @@
 Add the ability to send registration emails from the homeserver rather than delegating to an identity server.
--- a/changelog.d/5943.misc
+++ b/changelog.d/5943.misc
@@ -1 +0,0 @@
 Move Buildkite pipeline config to the pipelines repo.
--- a/changelog.d/5953.misc
+++ b/changelog.d/5953.misc
@@ -1 +0,0 @@
 Update INSTALL.md to say that Python 2 is no longer supported.
--- a/changelog.d/5962.misc
+++ b/changelog.d/5962.misc
@@ -1 +0,0 @@
 Remove unnecessary return statements in the codebase which were the result of a regex run.
--- a/changelog.d/5963.misc
+++ b/changelog.d/5963.misc
@@ -1 +0,0 @@
 Remove left-over methods from C/S registration API.
--- a/changelog.d/5964.feature
+++ b/changelog.d/5964.feature
@@ -1 +0,0 @@
 Remove `bind_email` and `bind_msisdn` parameters from /register ala MSC2140.
--- a/changelog.d/5966.bugfix
+++ b/changelog.d/5966.bugfix
@@ -1 +0,0 @@
 Fix admin API for listing media in a room not being available with an external media repo.
--- a/changelog.d/5967.bugfix
+++ b/changelog.d/5967.bugfix
@@ -1 +0,0 @@
 Fix list media admin API always returning an error.
--- a/changelog.d/5969.feature
+++ b/changelog.d/5969.feature
@@ -1 +0,0 @@
 Replace `trust_identity_server_for_password_resets` config option with `account_threepid_delegates`.
--- a/changelog.d/5970.docker
+++ b/changelog.d/5970.docker
@@ -1 +0,0 @@
 Avoid changing UID/GID if they are already correct.
--- a/changelog.d/5971.bugfix
+++ b/changelog.d/5971.bugfix
@@ -1 +0,0 @@
 Fix room and user stats tracking.
--- a/changelog.d/5975.misc
+++ b/changelog.d/5975.misc
@@ -1 +0,0 @@
 Cleanup event auth type initialisation.
--- a/changelog.d/5980.feature
+++ b/changelog.d/5980.feature
@@ -1 +0,0 @@
 Add POST /_matrix/client/r0/account/3pid/unbind endpoint from MSC2140 for unbinding a 3PID from an identity server without removing it from the homeserver user account.
--- a/changelog.d/5981.feature
+++ b/changelog.d/5981.feature
@@ -1 +0,0 @@
 Setting metrics_flags.known_servers to True in the configuration will publish the synapse_federation_known_servers metric over Prometheus. This represents the total number of servers your server knows about (i.e. is in rooms with), including itself.
--- a/changelog.d/5982.bugfix
+++ b/changelog.d/5982.bugfix
@@ -1 +0,0 @@
 Include missing opentracing contexts in outbout replication requests.
--- a/changelog.d/5983.feature
+++ b/changelog.d/5983.feature
@@ -1 +0,0 @@
 Add minimum opentracing for client servlets.
--- a/changelog.d/5984.bugfix
+++ b/changelog.d/5984.bugfix
@@ -1 +0,0 @@
 Fix sending of EDUs when opentracing is enabled with an empty whitelist.
--- a/changelog.d/5985.feature
+++ b/changelog.d/5985.feature
@@ -1 +0,0 @@
 Check at setup that opentracing is installed if it's enabled in the config.
--- a/changelog.d/5986.feature
+++ b/changelog.d/5986.feature
@@ -1 +0,0 @@
 Trace replication send times.
--- a/changelog.d/5988.bugfix
+++ b/changelog.d/5988.bugfix
@@ -1 +0,0 @@
 Fix invalid references to None while opentracing if the log context slips.
--- a/changelog.d/5989.misc
+++ b/changelog.d/5989.misc
@@ -1 +0,0 @@
 Clean up dependency checking at setup.
--- a/changelog.d/5991.bugfix
+++ b/changelog.d/5991.bugfix
@@ -1 +0,0 @@
 Fix invalid references to None while opentracing if the log context slips.
--- a/changelog.d/5993.feature
+++ b/changelog.d/5993.feature
@@ -1 +0,0 @@
 Add the ability to send registration emails from the homeserver rather than delegating to an identity server.
--- a/changelog.d/5994.feature
+++ b/changelog.d/5994.feature
@@ -1 +0,0 @@
 Add the ability to send registration emails from the homeserver rather than delegating to an identity server.
--- a/changelog.d/5995.bugfix
+++ b/changelog.d/5995.bugfix
@@ -1 +0,0 @@
 Return a M_MISSING_PARAM if `sid` is not provided to `/account/3pid`.
--- a/changelog.d/5998.bugfix
+++ b/changelog.d/5998.bugfix
@@ -1 +0,0 @@
 Fix room and user stats tracking.
--- a/changelog.d/6003.misc
+++ b/changelog.d/6003.misc
@@ -1 +0,0 @@
 Add opentracing span over HTTP push processing.
--- a/Show More
+++ b/Show More
Author	SHA1	Message	Date
Erik Johnston	75bf48b905	Update tracer to give more information	2015-03-13 11:38:57 +00:00
Erik Johnston	d1ae594ae5	Add a utility class that can be used to generate a twisted deferred aware call graph	2015-03-12 16:52:02 +00:00
		`@@ -1 +0,0 @@`
			`Don't create broken room when power_level_content_override.users does not contain creator_id.`
		`@@ -1 +0,0 @@`
			`Lay the groundwork for structured logging output.`
		`@@ -1 +0,0 @@`
			Update opentracing docs to use the unified `trace` method.
		`@@ -1 +0,0 @@`
			`Add the ability to send registration emails from the homeserver rather than delegating to an identity server.`
		`@@ -1 +0,0 @@`
			`Retry well-known lookup before the cache expires, giving a grace period where the remote well-known can be down but we still use the old result.`
		`@@ -1 +0,0 @@`
			`Add an admin API to purge old rooms from the database.`
		`@@ -1 +0,0 @@`
			`Add retry to well-known lookups if we have recently seen a valid well-known record for the server.`
		`@@ -1 +0,0 @@`
			`Pass opentracing contexts between servers when transmitting EDUs.`
		`@@ -1 +0,0 @@`
			`Add a tag recording a request's authenticated entity and corresponding servlet in opentracing.`