Bump versions and changelog

This uses the ignoreSynchronization flag because of the longpoll on the event
stream. It would be better to use $interval, but couldn't get that to
*reliably* work when testing. I suspect that $interval won't help us here,
since there is genuinely an open $http connection, as we're doing a long
poll. https://github.com/angular/protractor/issues/49 for more info.

.gitignore

@@ -10,10 +10,21 @@ docs/build/
 *.egg-info
 
 cmdclient_config.json
-homeserver.db
+homeserver*.db
 
 .coverage
 htmlcov
 
 demo/*.db
 demo/*.log
+demo/*.pid
+demo/etc
+
+graph/*.svg
+graph/*.png
+graph/*.dot
+
+webclient/config.js
+webclient/test/environment-protractor.js
+
+uploads

CHANGES.rst

@@ -0,0 +1,218 @@
+Changes in synapse 0.3.3 (2014-09-22)
+=====================================
+
+Homeserver:
+ * Fix bug where you continued to get events for rooms you had left.
+
+Webclient:
+ * Add support for video calls with basic UI.
+ * Fix bug where one to one chats were named after your display name rather
+   than the other person's.
+ * Fix bug which caused lag when typing in the textarea.
+ * Refuse to run on browsers we know won't work.
+ * Trigger pagination when joining new rooms.
+ * Fix bug where we sometimes didn't display invitations in recents.
+ * Automatically join room when accepting a VoIP call.
+ * Disable outgoing and reject incoming calls on browsers we don't support
+   VoIP in.
+ * Don't display desktop notifications for messages in the room you are
+   non-idle and speaking in.
+
+Changes in synapse 0.3.2 (2014-09-18)
+=====================================
+
+Webclient:
+ * Fix bug where an empty "bing words" list in old accounts didn't send
+   notifications when it should have done.
+
+Changes in synapse 0.3.1 (2014-09-18)
+=====================================
+This is a release to hotfix v0.3.0 to fix two regressions.
+
+Webclient:
+ * Fix a regression where we sometimes displayed duplicate events.
+ * Fix a regression where we didn't immediately remove rooms you were
+   banned in from the recents list.
+
+Changes in synapse 0.3.0 (2014-09-18)
+=====================================
+See UPGRADE for information about changes to the client server API, including
+breaking backwards compatibility with VoIP calls and registration API.
+
+Homeserver:
+ * When a user changes their displayname or avatar the server will now update 
+   all their join states to reflect this.
+ * The server now adds "age" key to events to indicate how old they are. This
+   is clock independent, so at no point does any server or webclient have to
+   assume their clock is in sync with everyone else.
+ * Fix bug where we didn't correctly pull in missing PDUs.
+ * Fix bug where prev_content key wasn't always returned.
+ * Add support for password resets.
+
+Webclient:
+ * Improve page content loading.
+ * Join/parts now trigger desktop notifications.
+ * Always show room aliases in the UI if one is present.
+ * No longer show user-count in the recents side panel.
+ * Add up & down arrow support to the text box for message sending to step
+   through your sent history.
+ * Don't display notifications for our own messages.
+ * Emotes are now formatted correctly in desktop notifications.
+ * The recents list now differentiates between public & private rooms.
+ * Fix bug where when switching between rooms the pagination flickered before
+   the view jumped to the bottom of the screen.
+ * Add bing word support.
+
+Registration API:
+ * The registration API has been overhauled to function like the login API. In
+   practice, this means registration requests must now include the following:
+   'type':'m.login.password'. See UPGRADE for more information on this.
+ * The 'user_id' key has been renamed to 'user' to better match the login API.
+ * There is an additional login type: 'm.login.email.identity'.
+ * The command client and web client have been updated to reflect these changes.
+
+Changes in synapse 0.2.3 (2014-09-12)
+=====================================
+
+Homeserver:
+ * Fix bug where we stopped sending events to remote home servers if a
+   user from that home server left, even if there were some still in the
+   room.
+ * Fix bugs in the state conflict resolution where it was incorrectly
+   rejecting events.
+
+Webclient:
+ * Display room names and topics.
+ * Allow setting/editing of room names and topics.
+ * Display information about rooms on the main page.
+ * Handle ban and kick events in real time.
+ * VoIP UI and reliability improvements.
+ * Add glare support for VoIP.
+ * Improvements to initial startup speed.
+ * Don't display duplicate join events.
+ * Local echo of messages.
+ * Differentiate sending and sent of local echo.
+ * Various minor bug fixes.
+
+Changes in synapse 0.2.2 (2014-09-06)
+=====================================
+
+Homeserver:
+ * When the server returns state events it now also includes the previous 
+   content.
+ * Add support for inviting people when creating a new room.
+ * Make the homeserver inform the room via `m.room.aliases` when a new alias
+   is added for a room.
+ * Validate `m.room.power_level` events.
+
+Webclient:
+ * Add support for captchas on registration.
+ * Handle `m.room.aliases` events.
+ * Asynchronously send messages and show a local echo.
+ * Inform the UI when a message failed to send.
+ * Only autoscroll on receiving a new message if the user was already at the 
+   bottom of the screen.
+ * Add support for ban/kick reasons.
+
+Changes in synapse 0.2.1 (2014-09-03)
+=====================================
+
+Homeserver:
+ * Added support for signing up with a third party id.
+ * Add synctl scripts.
+ * Added rate limiting.
+ * Add option to change the external address the content repo uses.
+ * Presence bug fixes.
+
+Webclient:
+ * Added support for signing up with a third party id.
+ * Added support for banning and kicking users.
+ * Added support for displaying and setting ops.
+ * Added support for room names.
+ * Fix bugs with room membership event display.
+
+Changes in synapse 0.2.0 (2014-09-02)
+=====================================
+This update changes many configuration options, updates the
+database schema and mandates SSL for server-server connections.
+
+Homeserver:
+ * Require SSL for server-server connections.
+ * Add SSL listener for client-server connections.
+ * Add ability to use config files.
+ * Add support for kicking/banning and power levels.
+ * Allow setting of room names and topics on creation.
+ * Change presence to include last seen time of the user.
+ * Change url path prefix to /_matrix/...
+ * Bug fixes to presence.
+
+Webclient:
+ * Reskin the CSS for registration and login.
+ * Various improvements to rooms CSS.
+ * Support changes in client-server API.
+ * Bug fixes to VOIP UI.
+ * Various bug fixes to handling of changes to room member list.
+
+Changes in synapse 0.1.2 (2014-08-29)
+=====================================
+
+Webclient:
+ * Add basic call state UI for VoIP calls.
+
+Changes in synapse 0.1.1 (2014-08-29)
+=====================================
+
+Homeserver:
+    * Fix bug that caused the event stream to not notify some clients about
+      changes.
+
+Changes in synapse 0.1.0 (2014-08-29)
+=====================================
+Presence has been reenabled in this release.
+
+Homeserver:
+ * Update client to server API, including:
+    - Use a more consistent url scheme.
+    - Provide more useful information in the initial sync api.
+ * Change the presence handling to be much more efficient.
+ * Change the presence server to server API to not require explicit polling of
+   all users who share a room with a user.
+ * Fix races in the event streaming logic.
+
+Webclient:
+ * Update to use new client to server API.
+ * Add basic VOIP support.
+ * Add idle timers that change your status to away.
+ * Add recent rooms column when viewing a room.
+ * Various network efficiency improvements.
+ * Add basic mobile browser support.
+ * Add a settings page.
+
+Changes in synapse 0.0.1 (2014-08-22)
+=====================================
+Presence has been disabled in this release due to a bug that caused the
+homeserver to spam other remote homeservers.
+
+Homeserver:
+ * Completely change the database schema to support generic event types.
+ * Improve presence reliability.
+ * Improve reliability of joining remote rooms.
+ * Fix bug where room join events were duplicated.
+ * Improve initial sync API to return more information to the client.
+ * Stop generating fake messages for room membership events.
+
+Webclient:
+ * Add tab completion of names.
+ * Add ability to upload and send images.
+ * Add profile pages.
+ * Improve CSS layout of room.
+ * Disambiguate identical display names.
+ * Don't get remote users display names and avatars individually.
+ * Use the new initial sync API to reduce number of round trips to the homeserver.
+ * Change url scheme to use room aliases instead of room ids where known.
+ * Increase longpoll timeout.
+
+Changes in synapse 0.0.0 (2014-08-13)
+=====================================
+
+ * Initial alpha release

README.rst

@@ -1,12 +1,12 @@
-Quick Start
-===========
+Introduction
+============
 
 Matrix is an ambitious new ecosystem for open federated Instant Messaging and
-VoIP[1].  The basics you need to know to get up and running are:
+VoIP.  The basics you need to know to get up and running are:
 
     - Chatrooms are distributed and do not exist on any single server.  Rooms 
-      can be found using names like ``#matrix:matrix.org`` or 
-      ``#test:localhost:8080`` or they can be ephemeral.
+      can be found using aliases like ``#matrix:matrix.org`` or 
+      ``#test:localhost:8008`` or they can be ephemeral.
     
     - Matrix user IDs look like ``@matthew:matrix.org`` (although in the future
       you will normally refer to yourself and others using a 3PID: email
@@ -14,29 +14,53 @@ VoIP[1].  The basics you need to know to get up and running are:
 
 The overall architecture is::
 
-      client <----> homeserver <=================> homeserver <-----> client
-                e.g. matrix.org:8080        e.g. mydomain.net:8080
+      client <----> homeserver <=====================> homeserver <----> client
+             https://somewhere.org/_matrix      https://elsewhere.net/_matrix
+
+WARNING
+=======
+
+**Synapse is currently in a state of rapid development, and not all features are yet functional.
+Critically, some security features are still in development, which means Synapse can *not*
+be considered secure or reliable at this point.**  For instance:
+
+- **SSL Certificates used by server-server federation are not yet validated.**
+- **Room permissions are not yet enforced on traffic received via federation.**
+- **Homeservers do not yet cryptographically sign their events to avoid tampering**
+- Default configuration provides open signup to the service from the internet
+
+Despite this, we believe Synapse is more than useful as a way for experimenting and
+exploring Synapse, and the missing features will land shortly.  **Until then, please do *NOT*
+use Synapse for any remotely important or secure communication.**
+
+
+Quick Start
+===========
+
+System requirements:
+    - POSIX-compliant system (tested on Linux & OSX)
+    - Python 2.7
 
 To get up and running:
       
     - To simply play with an **existing** homeserver you can
       just go straight to http://matrix.org/alpha.
     
-    - To run your own **private** homeserver on localhost:8080, install synapse 
-      with ``python setup.py develop --user`` and then run one with
-      ``python synapse/app/homeserver.py``
-      
-    - To run your own webclient:
-      ``cd webclient; python -m SimpleHTTPServer`` and hit http://localhost:8000
-      in your web browser (a recent Chrome, Safari or Firefox for now,
-      please...)
+    - To run your own **private** homeserver on localhost:8008, install synapse with
+      ``python setup.py develop --user`` and then run ``./synctl start`` twice (once to
+      generate a config; once to actually run) - you will find a webclient running at
+      http://localhost:8008. Please use a recent Chrome, Safari or Firefox for now...
              
-    - To make the homeserver **public** and let it exchange messages with 
-      other homeservers and participate in the overall Matrix federation, open 
-      up port 8080 and run ``python synapse/app/homeserver.py --host 
-      machine.my.domain.name``.  Then come join ``#matrix:matrix.org`` and
-      say hi! :)
-    
+    - To run a **public** homeserver and let it exchange messages with other homeservers
+      and participate in the global Matrix federation, you must expose port 8448 to the
+      internet and edit homeserver.yaml to specify server_name (the public DNS entry for
+      this server) and then run ``synctl start``. If you changed the server_name, you may
+      need to move the old database (homeserver.db) out of the way first. Then come join
+      ``#matrix:matrix.org`` and say hi! :)
+
+For more detailed setup instructions, please see further down this document.
+
+   
 About Matrix
 ============
 
@@ -45,15 +69,15 @@ which handle:
 
     - Creating and managing fully distributed chat rooms with no
       single points of control or failure
-    - Eventually-consistent cryptographically secure[2] synchronisation of room 
+    - Eventually-consistent cryptographically secure[1] synchronisation of room 
       state across a global open network of federated servers and services
     - Sending and receiving extensible messages in a room with (optional)
-      end-to-end encryption[3]
+      end-to-end encryption[2]
     - Inviting, joining, leaving, kicking, banning room members
     - Managing user accounts (registration, login, logout)
     - Using 3rd Party IDs (3PIDs) such as email addresses, phone numbers,
       Facebook accounts to authenticate, identify and discover users on Matrix.
-    - Placing 1:1 VoIP and Video calls (in development)
+    - Placing 1:1 VoIP and Video calls
 
 These APIs are intended to be implemented on a wide range of servers, services
 and clients, letting developers build messaging and VoIP functionality on top of
@@ -78,7 +102,7 @@ service provider in Matrix, unlike WhatsApp, Facebook, Hangouts, etc.
 
 Synapse ships with two basic demo Matrix clients: webclient (a basic group chat
 web client demo implemented in AngularJS) and cmdclient (a basic Python
-commandline utility which lets you easily see what the JSON APIs are up to).
+command line utility which lets you easily see what the JSON APIs are up to).
 
 We'd like to invite you to take a look at the Matrix spec, try to run a
 homeserver, and join the existing Matrix chatrooms already out there, experiment
@@ -87,11 +111,9 @@ https://github.com/matrix-org/synapse/issues or at matrix@matrix.org.
 
 Thanks for trying Matrix!
 
-[1] VoIP currently in development
+[1] Cryptographic signing of messages isn't turned on yet
 
-[2] Cryptographic signing of messages isn't turned on yet
-
-[3] End-to-end encryption is currently in development
+[2] End-to-end encryption is currently in development
 
 
 Homeserver Installation
@@ -99,9 +121,8 @@ Homeserver Installation
 
 First, the dependencies need to be installed.  Start by installing 
 'python2.7-dev' and the various tools of the compiler toolchain.
-N.B. synapse requires python 2.x where x >= 7
 
-  Installing prerequisites on ubuntu::
+  Installing prerequisites on Ubuntu::
 
     $ sudo apt-get install build-essential python2.7-dev libffi-dev
 
@@ -120,12 +141,19 @@ may need to also run:
     $ sudo apt-get install python-pip
     $ sudo pip install --upgrade setuptools
     
+If you don't have access to github, then you may need to install ``syutil``
+manually by checking it out and running ``python setup.py develop --user`` on it
+too.
+    
 If you get errors about ``sodium.h`` being missing, you may also need to
 manually install a newer PyNaCl via pip as setuptools installs an old one. Or
 you can check PyNaCl out of git directly (https://github.com/pyca/pynacl) and
 installing it. Installing PyNaCl using pip may also work (remember to remove any
 other versions installed by setuputils in, for example, ~/.local/lib).
 
+On OSX, if you encounter ``clang: error: unknown argument: '-mno-fused-madd'`` 
+you will need to ``export CFLAGS=-Qunused-arguments``.
+
 This will run a process of downloading and installing into your
 user's .local/lib directory all of the required dependencies that are
 missing.
@@ -142,6 +170,13 @@ This should end with a 'PASSED' result::
     PASSED (successes=143)
 
 
+Upgrading an existing homeserver
+================================
+
+Before upgrading an existing homeserver to a new version, please refer to
+UPGRADE.rst for any additional instructions.
+ 
+
 Setting up Federation
 =====================
 
@@ -162,7 +197,16 @@ IDs:
 For the first form, simply pass the required hostname (of the machine) as the
 --host parameter::
 
-    $ python synapse/app/homeserver.py --host machine.my.domain.name
+    $ python synapse/app/homeserver.py \
+        --server-name machine.my.domain.name \
+        --config-path homeserver.config \
+        --generate-config
+    $ python synapse/app/homeserver.py --config-path homeserver.config
+    
+Alternatively, you can run synapse via synctl - running ``synctl start`` to 
+generate a homeserver.yaml config file, where you can then edit server-name to 
+specify machine.my.domain.name, and then set the actual server running again 
+with synctl start.
 
 For the second form, first create your SRV record and publish it in DNS. This
 needs to be named _matrix._tcp.YOURDOMAIN, and point at at least one hostname
@@ -175,7 +219,13 @@ record would then look something like::
 At this point, you should then run the homeserver with the hostname of this
 SRV record, as that is the name other machines will expect it to have::
 
-    $ python synapse/app/homeserver.py --host my.domain.name --port 8448
+    $ python synapse/app/homeserver.py \
+        --server-name YOURDOMAIN \
+        --bind-port 8448 \
+        --config-path homeserver.config \
+        --generate-config
+    $ python synapse/app/homeserver.py --config-path homeserver.config
+
 
 You may additionally want to pass one or more "-v" options, in order to
 increase the verbosity of logging output; at least for initial testing.
@@ -189,22 +239,15 @@ Running a Demo Federation of Homeservers
 
 If you want to get up and running quickly with a trio of homeservers in a
 private federation (``localhost:8080``, ``localhost:8081`` and
-``localhost:8082``) which you can then access through the webclient running at http://localhost:8080.  Simply run::
+``localhost:8082``) which you can then access through the webclient running at
+http://localhost:8080. Simply run::
 
     $ demo/start.sh
 
 Running The Demo Web Client
 ===========================
 
-At the present time, the web client is not directly served by the homeserver's
-HTTP server. To serve this in a form the web browser can reach, arrange for the
-'webclient' sub-directory to be made available by any sort of HTTP server that
-can serve static files. For example, python's SimpleHTTPServer will suffice::
-
-    $ cd webclient
-    $ python -m SimpleHTTPServer
-
-You can now point your browser at  http://localhost:8000/  to find the client.
+The homeserver runs a web client by default at http://localhost:8080.
 
 If this is the first time you have used the client from that browser (it uses
 HTML5 local storage to remember its config), you will need to log in to your
@@ -251,7 +294,8 @@ track 3PID logins and publish end-user public keys.
 
 It's currently early days for identity servers as Matrix is not yet using 3PIDs
 as the primary means of identity and E2E encryption is not complete. As such,
-we're not yet running an identity server in public.
+we are running a single identity server (http://matrix.org:8090) at the current 
+time.
 
 
 Where's the spec?!

UPGRADE.rst

@@ -0,0 +1,81 @@
+Upgrading to v0.3.0
+===================
+
+This registration API now closely matches the login API. This introduces a bit
+more backwards and forwards between the HS and the client, but this improves
+the overall flexibility of the API. You can now GET on /register to retrieve a list
+of valid registration flows. Upon choosing one, they are submitted in the same
+way as login, e.g::
+
+  {
+    type: m.login.password,
+    user: foo,
+    password: bar
+  }
+
+The default HS supports 2 flows, with and without Identity Server email
+authentication. Enabling captcha on the HS will add in an extra step to all
+flows: ``m.login.recaptcha`` which must be completed before you can transition
+to the next stage. There is a new login type: ``m.login.email.identity`` which
+contains the ``threepidCreds`` key which were previously sent in the original
+register request. For more information on this, see the specification.
+
+Web Client
+----------
+
+The VoIP specification has changed between v0.2.0 and v0.3.0. Users should
+refresh any browser tabs to get the latest web client code. Users on
+v0.2.0 of the web client will not be able to call those on v0.3.0 and
+vice versa.
+
+
+Upgrading to v0.2.0
+===================
+
+The home server now requires setting up of SSL config before it can run. To
+automatically generate default config use::
+
+    $ python synapse/app/homeserver.py \
+        --server-name machine.my.domain.name \
+        --bind-port 8448 \
+        --config-path homeserver.config \
+        --generate-config
+
+This config can be edited if desired, for example to specify a different SSL 
+certificate to use. Once done you can run the home server using::
+
+    $ python synapse/app/homeserver.py --config-path homeserver.config
+
+See the README.rst for more information.
+
+Also note that some config options have been renamed, including:
+
+- "host" to "server-name"
+- "database" to "database-path"
+- "port" to "bind-port" and "unsecure-port"
+
+
+Upgrading to v0.0.1
+===================
+
+This release completely changes the database schema and so requires upgrading
+it before starting the new version of the homeserver.
+
+The script "database-prepare-for-0.0.1.sh" should be used to upgrade the
+database. This will save all user information, such as logins and profiles, 
+but will otherwise purge the database. This includes messages, which
+rooms the home server was a member of and room alias mappings.
+
+Before running the command the homeserver should be first completely 
+shutdown. To run it, simply specify the location of the database, e.g.:
+
+  ./database-prepare-for-0.0.1.sh "homeserver.db"
+
+Once this has successfully completed it will be safe to restart the 
+homeserver. You may notice that the homeserver takes a few seconds longer to 
+restart than usual as it reinitializes the database.
+
+On startup of the new version, users can either rejoin remote rooms using room
+aliases or by being reinvited. Alternatively, if any other homeserver sends a
+message to a room that the homeserver was previously in the local HS will 
+automatically rejoin the room.

VERSION

@@ -0,0 +1 @@
+0.3.3

WISHLIST.rst

@@ -0,0 +1,9 @@
+Broad-sweeping stuff which would be nice to have
+================================================
+
+ - Additional SQL backends beyond sqlite
+ - homeserver implementation in go
+ - homeserver implementation in node.js
+ - client SDKs
+ - libpurple library
+ - irssi plugin?

cmdclient/console.py

@@ -1,6 +1,6 @@
 #!/usr/bin/env python
 
-# Copyright 2014 matrix.org
+# Copyright 2014 OpenMarket Ltd
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
@@ -60,8 +60,8 @@ class SynapseCmd(cmd.Cmd):
             "complete_usernames": "on",
             "send_delivery_receipts": "on"
         }
-        self.path_prefix = "/matrix/client/api/v1"
-        self.event_stream_token = "START"
+        self.path_prefix = "/_matrix/client/api/v1"
+        self.event_stream_token = "END"
         self.prompt = ">>> "
 
     def do_EOF(self, line):  # allows CTRL+D quitting
@@ -88,6 +88,8 @@ class SynapseCmd(cmd.Cmd):
         return False
 
     def _domain(self):
+        if "user" not in self.config or not self.config["user"]:
+            return None
         return self.config["user"].split(":")[1]
 
     def do_config(self, line):
@@ -143,35 +145,50 @@ class SynapseCmd(cmd.Cmd):
         <noupdate> : Do not automatically clobber config values.
         """
         args = self._parse(line, ["userid", "noupdate"])
-        path = "/register"
 
         password = None
         pwd = None
         pwd2 = "_"
         while pwd != pwd2:
-            pwd = getpass.getpass("(Optional) Type a password for this user: ")
-            if len(pwd) == 0:
-                print "Not using a password for this user."
-                break
+            pwd = getpass.getpass("Type a password for this user: ")
             pwd2 = getpass.getpass("Retype the password: ")
-            if pwd != pwd2:
+            if pwd != pwd2 or len(pwd) == 0:
                 print "Password mismatch."
+                pwd = None
             else:
                 password = pwd
 
-        body = {}
+        body = {
+            "type": "m.login.password"
+        }
         if "userid" in args:
-            body["user_id"] = args["userid"]
+            body["user"] = args["userid"]
         if password:
             body["password"] = password
 
-        reactor.callFromThread(self._do_register, "POST", path, body,
+        reactor.callFromThread(self._do_register, body,
                                "noupdate" not in args)
 
     @defer.inlineCallbacks
-    def _do_register(self, method, path, data, update_config):
-        url = self._url() + path
-        json_res = yield self.http_client.do_request(method, url, data=data)
+    def _do_register(self, data, update_config):
+        # check the registration flows
+        url = self._url() + "/register"
+        json_res = yield self.http_client.do_request("GET", url)
+        print json.dumps(json_res, indent=4)
+
+        passwordFlow = None
+        for flow in json_res["flows"]:
+            if flow["type"] == "m.login.recaptcha" or ("stages" in flow and "m.login.recaptcha" in flow["stages"]):
+                print "Unable to register: Home server requires captcha."
+                return
+            if flow["type"] == "m.login.password" and "stages" not in flow:
+                passwordFlow = flow
+                break
+
+        if not passwordFlow:
+            return
+
+        json_res = yield self.http_client.do_request("POST", url, data=data)
         print json.dumps(json_res, indent=4)
         if update_config and "user_id" in json_res:
             self.config["user"] = json_res["user_id"]
@@ -191,10 +208,12 @@ class SynapseCmd(cmd.Cmd):
                 p = getpass.getpass("Enter your password: ")
                 user = args["user_id"]
                 if self._is_on("complete_usernames") and not user.startswith("@"):
-                    user = "@" + user + ":" + self._domain()
-
+                    domain = self._domain()
+                    if domain:
+                        user = "@" + user + ":" + domain
+                
                 reactor.callFromThread(self._do_login, user, p)
-                print " got %s " % p
+                #print " got %s " % p
         except Exception as e:
             print e
 
@@ -225,59 +244,76 @@ class SynapseCmd(cmd.Cmd):
         json_res = yield self.http_client.do_request("GET", url)
         print json_res
 
-        if ("type" not in json_res or "m.login.password" != json_res["type"] or
-                "stages" in json_res):
+        if "flows" not in json_res:
+            print "Failed to find any login flows."
+            defer.returnValue(False)
+
+        flow = json_res["flows"][0] # assume first is the one we want.
+        if ("type" not in flow or "m.login.password" != flow["type"] or
+                "stages" in flow):
             fallback_url = self._url() + "/login/fallback"
             print ("Unable to login via the command line client. Please visit "
                 "%s to login." % fallback_url)
             defer.returnValue(False)
         defer.returnValue(True)
 
-    def do_3pidrequest(self, line):
+    def do_emailrequest(self, line):
         """Requests the association of a third party identifier
-        <medium> The medium of the identifer (currently only 'email')
-        <address> The address of the identifer (ie. the email address)
+        <address> The email address)
+        <clientSecret> A string of characters generated when requesting an email that you'll supply in subsequent calls to identify yourself
+        <sendAttempt> The number of times the user has requested an email. Leave this the same between requests to retry the request at the transport level. Increment it to request that the email be sent again.
         """
-        args = self._parse(line, ['medium', 'address'])
+        args = self._parse(line, ['address', 'clientSecret', 'sendAttempt'])
 
-        if not args['medium'] == 'email':
-            print "Only email is supported currently"
-            return
+        postArgs = {'email': args['address'], 'clientSecret': args['clientSecret'], 'sendAttempt': args['sendAttempt']}
 
-        postArgs = {'email': args['address'], 'clientSecret': '____'}
-
-        reactor.callFromThread(self._do_3pidrequest, postArgs)
+        reactor.callFromThread(self._do_emailrequest, postArgs)
 
     @defer.inlineCallbacks
-    def _do_3pidrequest(self, args):
-        url = self._identityServerUrl()+"/matrix/identity/api/v1/validate/email/requestToken"
+    def _do_emailrequest(self, args):
+        url = self._identityServerUrl()+"/_matrix/identity/api/v1/validate/email/requestToken"
 
         json_res = yield self.http_client.do_request("POST", url, data=urllib.urlencode(args), jsonreq=False,
                                                      headers={'Content-Type': ['application/x-www-form-urlencoded']})
         print json_res
-        if 'tokenId' in json_res:
-            print "Token ID %s sent" % (json_res['tokenId'])
+        if 'sid' in json_res:
+            print "Token sent. Your session ID is %s" % (json_res['sid'])
 
-    def do_3pidvalidate(self, line):
+    def do_emailvalidate(self, line):
         """Validate and associate a third party ID
-        <medium> The medium of the identifer (currently only 'email')
-        <tokenId> The identifier iof the token given in 3pidrequest
+        <sid> The session ID (sid) given to you in the response to requestToken
         <token> The token sent to your third party identifier address
+        <clientSecret> The same clientSecret you supplied in requestToken
         """
-        args = self._parse(line, ['medium', 'tokenId', 'token'])
+        args = self._parse(line, ['sid', 'token', 'clientSecret'])
 
-        if not args['medium'] == 'email':
-            print "Only email is supported currently"
-            return
+        postArgs = { 'sid' : args['sid'], 'token' : args['token'], 'clientSecret': args['clientSecret'] }
 
-        postArgs = { 'tokenId' : args['tokenId'], 'token' : args['token'] }
-        postArgs['mxId'] = self.config["user"]
-
-        reactor.callFromThread(self._do_3pidvalidate, postArgs)
+        reactor.callFromThread(self._do_emailvalidate, postArgs)
 
     @defer.inlineCallbacks
-    def _do_3pidvalidate(self, args):
-        url = self._identityServerUrl()+"/matrix/identity/api/v1/validate/email/submitToken"
+    def _do_emailvalidate(self, args):
+        url = self._identityServerUrl()+"/_matrix/identity/api/v1/validate/email/submitToken"
+
+        json_res = yield self.http_client.do_request("POST", url, data=urllib.urlencode(args), jsonreq=False,
+                                                     headers={'Content-Type': ['application/x-www-form-urlencoded']})
+        print json_res
+
+    def do_3pidbind(self, line):
+        """Validate and associate a third party ID
+        <sid> The session ID (sid) given to you in the response to requestToken
+        <clientSecret> The same clientSecret you supplied in requestToken
+        """
+        args = self._parse(line, ['sid', 'clientSecret'])
+
+        postArgs = { 'sid' : args['sid'], 'clientSecret': args['clientSecret'] }
+        postArgs['mxid'] = self.config["user"]
+
+        reactor.callFromThread(self._do_3pidbind, postArgs)
+
+    @defer.inlineCallbacks
+    def _do_3pidbind(self, args):
+        url = self._identityServerUrl()+"/_matrix/identity/api/v1/3pid/bind"
 
         json_res = yield self.http_client.do_request("POST", url, data=urllib.urlencode(args), jsonreq=False,
                                                      headers={'Content-Type': ['application/x-www-form-urlencoded']})
@@ -295,7 +331,7 @@ class SynapseCmd(cmd.Cmd):
         try:
             args = self._parse(line, ["roomname"], force_keys=True)
             path = "/join/%s" % urllib.quote(args["roomname"])
-            reactor.callFromThread(self._run_and_pprint, "PUT", path, {})
+            reactor.callFromThread(self._run_and_pprint, "POST", path, {})
         except Exception as e:
             print e
 
@@ -343,14 +379,14 @@ class SynapseCmd(cmd.Cmd):
     def _do_invite(self, roomid, userstring):
         if (not userstring.startswith('@') and
                     self._is_on("complete_usernames")):
-            url = self._identityServerUrl()+"/matrix/identity/api/v1/lookup"
+            url = self._identityServerUrl()+"/_matrix/identity/api/v1/lookup"
 
             json_res = yield self.http_client.do_request("GET", url, qparams={'medium':'email','address':userstring})
 
             mxid = None
 
             if 'mxid' in json_res and 'signatures' in json_res:
-                url = self._identityServerUrl()+"/matrix/identity/api/v1/pubkey/ed25519"
+                url = self._identityServerUrl()+"/_matrix/identity/api/v1/pubkey/ed25519"
 
                 pubKey = None
                 pubKeyObj = yield self.http_client.do_request("GET", url)
@@ -390,19 +426,16 @@ class SynapseCmd(cmd.Cmd):
         """Leaves a room: "leave <roomid>" """
         try:
             args = self._parse(line, ["roomid"], force_keys=True)
-            path = ("/rooms/%s/members/%s/state" %
-                    (urllib.quote(args["roomid"]), self._usr()))
-            reactor.callFromThread(self._run_and_pprint, "DELETE", path)
+            self._do_membership_change(args["roomid"], "leave", self._usr())
         except Exception as e:
             print e
 
     def do_send(self, line):
         """Sends a message. "send <roomid> <body>" """
         args = self._parse(line, ["roomid", "body"])
-        msg_id = "m%s" % int(time.time())
-        path = "/rooms/%s/messages/%s/%s" % (urllib.quote(args["roomid"]),
-                                             self._usr(),
-                                             msg_id)
+        txn_id = "txn%s" % int(time.time())
+        path = "/rooms/%s/send/m.room.message/%s" % (urllib.quote(args["roomid"]),
+                                             txn_id)
         body_json = {
             "msgtype": "m.text",
             "body": args["body"]
@@ -426,7 +459,7 @@ class SynapseCmd(cmd.Cmd):
             print "Unrecognised type: %s" % args["type"]
             return
         room_id = args["roomid"]
-        path = "/rooms/%s/%s/list" % (urllib.quote(room_id), args["type"])
+        path = "/rooms/%s/%s" % (urllib.quote(room_id), args["type"])
 
         qp = {"access_token": self._tok()}
         if "qp" in args:
@@ -462,7 +495,7 @@ class SynapseCmd(cmd.Cmd):
             room_name = args["vis"]
             body["room_alias_name"] = room_name
 
-        reactor.callFromThread(self._run_and_pprint, "POST", "/rooms", body)
+        reactor.callFromThread(self._run_and_pprint, "POST", "/createRoom", body)
 
     def do_raw(self, line):
         """Directly send a JSON object: "raw <method> <path> <data> <notoken>"
@@ -555,7 +588,7 @@ class SynapseCmd(cmd.Cmd):
                                alt_text="Sent receipt for %s" % event["msg_id"])
 
     def _do_membership_change(self, roomid, membership, userid):
-        path = "/rooms/%s/members/%s/state" % (urllib.quote(roomid), userid)
+        path = "/rooms/%s/state/m.room.member/%s" % (urllib.quote(roomid), urllib.quote(userid))
         data = {
             "membership": membership
         }
@@ -686,7 +719,7 @@ def main(server_url, identity_server_url, username, token, config_path):
 if __name__ == '__main__':
     parser = argparse.ArgumentParser("Starts a synapse client.")
     parser.add_argument(
-        "-s", "--server", dest="server", default="http://localhost:8080",
+        "-s", "--server", dest="server", default="http://localhost:8008",
         help="The URL of the home server to talk to.")
     parser.add_argument(
         "-i", "--identity-server", dest="identityserver", default="http://localhost:8090",

cmdclient/http.py

@@ -1,5 +1,5 @@
 # -*- coding: utf-8 -*-
-# Copyright 2014 matrix.org
+# Copyright 2014 OpenMarket Ltd
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.

database-prepare-for-0.0.1.sh

@@ -0,0 +1,21 @@
+#!/bin/bash
+
+# This is will prepare a synapse database for running with v0.0.1 of synapse. 
+# It will store all the user information, but will *delete* all messages and
+# room data.
+
+set -e
+
+cp "$1" "$1.bak"
+
+DUMP=$(sqlite3 "$1" << 'EOF'
+.dump users
+.dump access_tokens
+.dump presence
+.dump profiles
+EOF
+)
+
+rm "$1"
+
+sqlite3 "$1" <<< "$DUMP" 

database-save.sh

@@ -8,7 +8,7 @@
 #
 #   $ sqlite3 homeserver.db < table-save.sql
 
-sqlite3 homeserver.db <<'EOF' >table-save.sql
+sqlite3 "$1" <<'EOF' >table-save.sql
 .dump users
 .dump access_tokens
 .dump presence

demo/demo.tls.dh

@@ -0,0 +1,9 @@
+2048-bit DH parameters taken from rfc3526
+-----BEGIN DH PARAMETERS-----
+MIIBCAKCAQEA///////////JD9qiIWjCNMTGYouA3BzRKQJOCIpnzHQCC76mOxOb
+IlFKCHmONATd75UZs806QxswKwpt8l8UN0/hNW1tUcJF5IW1dmJefsb0TELppjft
+awv/XLb0Brft7jhr+1qJn6WunyQRfEsf5kkoZlHs5Fs9wgB8uKFjvwWY2kg2HFXT
+mmkWP6j9JM9fg2VdI9yjrZYcYvNWIIVSu57VKQdwlpZtZww1Tkq8mATxdGwIyhgh
+fDKQXkYuNs474553LBgOhgObJ4Oi7Aeij7XFXfBvTFLJ3ivL9pVYFxg5lUl86pVq
+5RXSJhiY+gUQFXKOWoqsqmj//////////wIBAg==
+-----END DH PARAMETERS-----

demo/start.sh

@@ -6,16 +6,29 @@ CWD=$(pwd)
 
 cd "$DIR/.."
 
-for port in "8080" "8081" "8082"; do
+mkdir -p demo/etc
+
+for port in 8080 8081 8082; do
     echo "Starting server on port $port... "
 
+    https_port=$((port + 400))
+
     python -m synapse.app.homeserver \
-        -p "$port" \
-        -H "localhost:$port" \
+        --generate-config \
+        --config-path "demo/etc/$port.config" \
+        -p "$https_port" \
+        --unsecure-port "$port" \
+        -H "localhost:$https_port" \
         -f "$DIR/$port.log" \
         -d "$DIR/$port.db" \
+        -D --pid-file "$DIR/$port.pid" \
+        --manhole $((port + 1000)) \
+        --tls-dh-params-path "demo/demo.tls.dh"
+
+    python -m synapse.app.homeserver \
+        --config-path "demo/etc/$port.config" \
         -vv \
-        -D --pid-file "$DIR/$port.pid"
+
 done
 
 echo "Starting webclient on port 8000..."

demo/webserver.py

@@ -2,9 +2,32 @@ import argparse
 import BaseHTTPServer
 import os
 import SimpleHTTPServer
+import cgi, logging
 
 from daemonize import Daemonize
 
+class SimpleHTTPRequestHandlerWithPOST(SimpleHTTPServer.SimpleHTTPRequestHandler):
+    UPLOAD_PATH = "upload"
+
+    """
+    Accept all post request as file upload
+    """
+    def do_POST(self):
+
+        path = os.path.join(self.UPLOAD_PATH, os.path.basename(self.path))
+        length = self.headers['content-length']
+        data = self.rfile.read(int(length))
+
+        with open(path, 'wb') as fh:
+            fh.write(data)
+
+        self.send_response(200)
+        self.send_header('Content-Type', 'application/json')
+        self.end_headers()
+
+        # Return the absolute path of the uploaded file
+        self.wfile.write('{"url":"/%s"}' % path)
+
 
 def setup():
     parser = argparse.ArgumentParser()
@@ -19,7 +42,7 @@ def setup():
 
     httpd = BaseHTTPServer.HTTPServer(
         ('', args.port),
-        SimpleHTTPServer.SimpleHTTPRequestHandler
+        SimpleHTTPRequestHandlerWithPOST
     )
 
     def run():

docs/client-server/OLD_specification.rst

@@ -1,6 +1,20 @@
-=========================
-Synapse Client-Server API
-=========================
+========================
+Matrix Client-Server API
+========================
+
+
+.. WARNING::
+  This specification is old. Please see /docs/specification.rst instead.
+
+
+
+
+
+
+
+
+
+
 
 The following specification outlines how a client can send and receive data from 
 a home server.
@@ -262,7 +276,10 @@ the error, but the keys 'error' and 'errcode' will always be present.
 Some standard error codes are below:
 
 M_FORBIDDEN:
-Forbidden access, e.g. bad access token, failed login.
+Forbidden access, e.g. joining a room without permission, failed login.
+
+M_UNKNOWN_TOKEN:
+The access token specified was not recognised.
 
 M_BAD_JSON:
 Request contained valid JSON, but it was malformed in some way, e.g. missing
@@ -303,11 +320,11 @@ POST requests MUST be submitted as application/json.
 All paths MUST be namespaced by the version of the API being used. This should
 be:
 
-/matrix/client/api/v1
+/_matrix/client/api/v1
 
 All REST paths in this section MUST be prefixed with this. E.g.
   REST Path: /rooms/$room_id
-  Absolute Path: /matrix/client/api/v1/rooms/$room_id
+  Absolute Path: /_matrix/client/api/v1/rooms/$room_id
 
 Registration
 ============
@@ -411,6 +428,9 @@ The server checks this, finds it is valid, and returns:
 {
   "access_token": "abcdef0123456789"
 }
+The server may optionally return "user_id" to confirm or change the user's ID.
+This is particularly useful if the home server wishes to support localpart entry
+of usernames (e.g. "bob" rather than "@bob:matrix.org").
 
 OAuth2-based
 ------------
@@ -688,6 +708,16 @@ Invite/Joining/Leaving a room
   Required keys:
     membership : [join|invite] - The membership state of $user_id in room 
                                  $room_id.
+  Optional keys:
+    displayname,
+    avatar_url : String fields from the member user's profile
+    state,
+    status_msg,
+    mtime_age : Presence information
+
+  These optional keys provide extra information that the client is likely to
+  be interested in so it doesn't have to perform an additional profile or
+  presence information fetch.
 
 Where:
   join - Indicate you ($user_id) are joining the room $room_id.
@@ -991,26 +1021,15 @@ for users from other servers entirely.
 Presence
 ========
 
-In the following messages, the presence state is an integer enumeration of the
-following states:
-  0 : OFFLINE
-  1 : BUSY
-  2 : ONLINE
-  3 : FREE_TO_CHAT
-
-Aside from OFFLINE, the protocol doesn't assign any special meaning to these
-states; they are provided as an approximate signal for users to give to other
-users and for clients to present them in some way that may be useful. Clients
-could have different behaviours for different states of the user's presence, for
-example to decide how much prominence or sound to use for incoming event
-notifications.
+In the following messages, the presence state is a presence string as described in
+the main specification document.
 
 Getting/Setting your own presence state
 ---------------------------------------
   REST Path: /presence/$user_id/status
   Valid methods: GET/PUT
   Required keys:
-    state : [0|1|2|3] - The user's new presence state
+    presence : <string> - The user's new presence state
   Optional keys:
     status_msg : text string provided by the user to explain their status
 
@@ -1023,7 +1042,7 @@ Fetching your presence list
     following keys:
       {
         "user_id" : string giving the observed user's ID
-        "state" : int giving their status
+        "presence" : int giving their status
         "status_msg" : optional text string
         "displayname" : optional text string from the user's profile
         "avatar_url" : optional text string from the user's profile

docs/client-server/howto.rst

@@ -0,0 +1,636 @@
+.. TODO kegan
+  Room config (specifically: message history,
+  public rooms). /register seems super simplistic compared to /login, maybe it
+  would be better if /register used the same technique as /login? /register should
+  be "user" not "user_id".
+
+
+How to use the client-server API
+================================
+
+This guide focuses on how the client-server APIs *provided by the reference 
+home server* can be used. Since this is specific to a home server 
+implementation, there may be variations in relation to registering/logging in
+which are not covered in extensive detail in this guide.
+
+If you haven't already, get a home server up and running on 
+``http://localhost:8008``.
+
+
+Accounts
+========
+Before you can send and receive messages, you must **register** for an account. 
+If you already have an account, you must **login** into it.
+
+`Try out the fiddle`__
+
+.. __: http://jsfiddle.net/gh/get/jquery/1.8.3/matrix-org/synapse/tree/master/jsfiddles/register_login
+
+Registration
+------------
+The aim of registration is to get a user ID and access token which you will need
+when accessing other APIs::
+
+    curl -XPOST -d '{"user_id":"example", "password":"wordpass"}' "http://localhost:8008/_matrix/client/api/v1/register"
+
+    {
+        "access_token": "QGV4YW1wbGU6bG9jYWxob3N0.AqdSzFmFYrLrTmteXc", 
+        "home_server": "localhost", 
+        "user_id": "@example:localhost"
+    }
+
+NB: If a ``user_id`` is not specified, one will be randomly generated for you. 
+If you do not specify a ``password``, you will be unable to login to the account
+if you forget the ``access_token``.
+
+Implementation note: The matrix specification does not enforce how users 
+register with a server. It just specifies the URL path and absolute minimum 
+keys. The reference home server uses a username/password to authenticate user,
+but other home servers may use different methods.
+
+Login
+-----
+The aim when logging in is to get an access token for your existing user ID::
+
+    curl -XGET "http://localhost:8008/_matrix/client/api/v1/login"
+
+    {
+        "flows": [
+            {
+                "type": "m.login.password"
+            }
+        ]
+    }
+
+    curl -XPOST -d '{"type":"m.login.password", "user":"example", "password":"wordpass"}' "http://localhost:8008/_matrix/client/api/v1/login"
+
+    {
+        "access_token": "QGV4YW1wbGU6bG9jYWxob3N0.vRDLTgxefmKWQEtgGd", 
+        "home_server": "localhost", 
+        "user_id": "@example:localhost"
+    }
+    
+Implementation note: Different home servers may implement different methods for 
+logging in to an existing account. In order to check that you know how to login 
+to this home server, you must perform a ``GET`` first and make sure you 
+recognise the login type. If you do not know how to login, you can 
+``GET /login/fallback`` which will return a basic webpage which you can use to 
+login. The reference home server implementation support username/password login,
+but other home servers may support different login methods (e.g. OAuth2).
+
+
+Communicating
+=============
+
+In order to communicate with another user, you must **create a room** with that 
+user and **send a message** to that room. 
+
+`Try out the fiddle`__
+
+.. __: http://jsfiddle.net/gh/get/jquery/1.8.3/matrix-org/synapse/tree/master/jsfiddles/create_room_send_msg
+
+Creating a room
+---------------
+If you want to send a message to someone, you have to be in a room with them. To
+create a room::
+
+    curl -XPOST -d '{"room_alias_name":"tutorial"}' "http://localhost:8008/_matrix/client/api/v1/createRoom?access_token=YOUR_ACCESS_TOKEN"
+
+    {
+        "room_alias": "#tutorial:localhost", 
+        "room_id": "!CvcvRuDYDzTOzfKKgh:localhost"
+    }
+    
+The "room alias" is a human-readable string which can be shared with other users
+so they can join a room, rather than the room ID which is a randomly generated
+string. You can have multiple room aliases per room.
+
+.. TODO(kegan)
+  How to add/remove aliases from an existing room.
+    
+
+Sending messages
+----------------
+You can now send messages to this room::
+
+    curl -XPOST -d '{"msgtype":"m.text", "body":"hello"}' "http://localhost:8008/_matrix/client/api/v1/rooms/%21CvcvRuDYDzTOzfKKgh%3Alocalhost/send/m.room.message?access_token=YOUR_ACCESS_TOKEN"
+    
+    {
+        "event_id": "YUwRidLecu"
+    }
+    
+The event ID returned is a unique ID which identifies this message.
+    
+NB: There are no limitations to the types of messages which can be exchanged.
+The only requirement is that ``"msgtype"`` is specified. The Matrix 
+specification outlines the following standard types: ``m.text``, ``m.image``,
+``m.audio``, ``m.video``, ``m.location``, ``m.emote``. See the specification for
+more information on these types.
+
+Users and rooms
+===============
+
+Each room can be configured to allow or disallow certain rules. In particular,
+these rules may specify if you require an **invitation** from someone already in
+the room in order to **join the room**. In addition, you may also be able to 
+join a room **via a room alias** if one was set up.
+
+`Try out the fiddle`__
+
+.. __: http://jsfiddle.net/gh/get/jquery/1.8.3/matrix-org/synapse/tree/master/jsfiddles/room_memberships
+
+Inviting a user to a room
+-------------------------
+You can directly invite a user to a room like so::
+
+    curl -XPOST -d '{"user_id":"@myfriend:localhost"}' "http://localhost:8008/_matrix/client/api/v1/rooms/%21CvcvRuDYDzTOzfKKgh%3Alocalhost/invite?access_token=YOUR_ACCESS_TOKEN"
+    
+This informs ``@myfriend:localhost`` of the room ID 
+``!CvcvRuDYDzTOzfKKgh:localhost`` and allows them to join the room.
+
+Joining a room via an invite
+----------------------------
+If you receive an invite, you can join the room::
+
+    curl -XPOST -d '{}' "http://localhost:8008/_matrix/client/api/v1/rooms/%21CvcvRuDYDzTOzfKKgh%3Alocalhost/join?access_token=YOUR_ACCESS_TOKEN"
+    
+NB: Only the person invited (``@myfriend:localhost``) can change the membership
+state to ``"join"``. Repeatedly joining a room does nothing.
+
+Joining a room via an alias
+---------------------------
+Alternatively, if you know the room alias for this room and the room config 
+allows it, you can directly join a room via the alias::
+
+    curl -XPOST -d '{}' "http://localhost:8008/_matrix/client/api/v1/join/%23tutorial%3Alocalhost?access_token=YOUR_ACCESS_TOKEN"
+    
+    {
+        "room_id": "!CvcvRuDYDzTOzfKKgh:localhost"
+    }
+    
+You will need to use the room ID when sending messages, not the room alias.
+
+NB: If the room is configured to be an invite-only room, you will still require
+an invite in order to join the room even though you know the room alias. As a
+result, it is more common to see a room alias in relation to a public room, 
+which do not require invitations.
+
+Getting events
+==============
+An event is some interesting piece of data that a client may be interested in. 
+It can be a message in a room, a room invite, etc. There are many different ways
+of getting events, depending on what the client already knows.
+
+`Try out the fiddle`__
+
+.. __: http://jsfiddle.net/gh/get/jquery/1.8.3/matrix-org/synapse/tree/master/jsfiddles/event_stream
+
+Getting all state
+-----------------
+If the client doesn't know any information on the rooms the user is 
+invited/joined on, they can get all the user's state for all rooms::
+
+    curl -XGET "http://localhost:8008/_matrix/client/api/v1/initialSync?access_token=YOUR_ACCESS_TOKEN"
+    
+    {
+        "end": "s39_18_0", 
+        "presence": [
+            {
+                "content": {
+                    "last_active_ago": 1061436, 
+                    "user_id": "@example:localhost"
+                }, 
+                "type": "m.presence"
+            }
+        ], 
+        "rooms": [
+            {
+                "membership": "join", 
+                "messages": {
+                    "chunk": [
+                        {
+                            "content": {
+                                "@example:localhost": 10, 
+                                "default": 0
+                            }, 
+                            "event_id": "wAumPSTsWF", 
+                            "required_power_level": 10, 
+                            "room_id": "!MkDbyRqnvTYnoxjLYx:localhost", 
+                            "state_key": "", 
+                            "ts": 1409665585188, 
+                            "type": "m.room.power_levels", 
+                            "user_id": "@example:localhost"
+                        }, 
+                        {
+                            "content": {
+                                "join_rule": "public"
+                            }, 
+                            "event_id": "jrLVqKHKiI", 
+                            "required_power_level": 10, 
+                            "room_id": "!MkDbyRqnvTYnoxjLYx:localhost", 
+                            "state_key": "", 
+                            "ts": 1409665585188, 
+                            "type": "m.room.join_rules", 
+                            "user_id": "@example:localhost"
+                        }, 
+                        {
+                            "content": {
+                                "level": 10
+                            }, 
+                            "event_id": "WpmTgsNWUZ", 
+                            "required_power_level": 10, 
+                            "room_id": "!MkDbyRqnvTYnoxjLYx:localhost", 
+                            "state_key": "", 
+                            "ts": 1409665585188, 
+                            "type": "m.room.add_state_level", 
+                            "user_id": "@example:localhost"
+                        }, 
+                        {
+                            "content": {
+                                "level": 0
+                            }, 
+                            "event_id": "qUMBJyKsTQ", 
+                            "required_power_level": 10, 
+                            "room_id": "!MkDbyRqnvTYnoxjLYx:localhost", 
+                            "state_key": "", 
+                            "ts": 1409665585188, 
+                            "type": "m.room.send_event_level", 
+                            "user_id": "@example:localhost"
+                        }, 
+                        {
+                            "content": {
+                                "ban_level": 5, 
+                                "kick_level": 5
+                            }, 
+                            "event_id": "YAaDmKvoUW", 
+                            "required_power_level": 10, 
+                            "room_id": "!MkDbyRqnvTYnoxjLYx:localhost", 
+                            "state_key": "", 
+                            "ts": 1409665585188, 
+                            "type": "m.room.ops_levels", 
+                            "user_id": "@example:localhost"
+                        }, 
+                        {
+                            "content": {
+                                "avatar_url": null, 
+                                "displayname": null, 
+                                "membership": "join"
+                            }, 
+                            "event_id": "RJbPMtCutf", 
+                            "membership": "join", 
+                            "room_id": "!MkDbyRqnvTYnoxjLYx:localhost", 
+                            "state_key": "@example:localhost", 
+                            "ts": 1409665586730, 
+                            "type": "m.room.member", 
+                            "user_id": "@example:localhost"
+                        }, 
+                        {
+                            "content": {
+                                "body": "hello", 
+                                "hsob_ts": 1409665660439, 
+                                "msgtype": "m.text"
+                            }, 
+                            "event_id": "YUwRidLecu", 
+                            "room_id": "!MkDbyRqnvTYnoxjLYx:localhost", 
+                            "ts": 1409665660439, 
+                            "type": "m.room.message", 
+                            "user_id": "@example:localhost"
+                        }, 
+                        {
+                            "content": {
+                                "membership": "invite"
+                            }, 
+                            "event_id": "YjNuBKnPsb", 
+                            "membership": "invite", 
+                            "room_id": "!MkDbyRqnvTYnoxjLYx:localhost", 
+                            "state_key": "@myfriend:localhost", 
+                            "ts": 1409666426819, 
+                            "type": "m.room.member", 
+                            "user_id": "@example:localhost"
+                        }, 
+                        {
+                            "content": {
+                                "avatar_url": null, 
+                                "displayname": null, 
+                                "membership": "join", 
+                                "prev": "join"
+                            }, 
+                            "event_id": "KWwdDjNZnm", 
+                            "membership": "join", 
+                            "room_id": "!MkDbyRqnvTYnoxjLYx:localhost", 
+                            "state_key": "@example:localhost", 
+                            "ts": 1409666551582, 
+                            "type": "m.room.member", 
+                            "user_id": "@example:localhost"
+                        }, 
+                        {
+                            "content": {
+                                "avatar_url": null, 
+                                "displayname": null, 
+                                "membership": "join"
+                            }, 
+                            "event_id": "JFLVteSvQc", 
+                            "membership": "join", 
+                            "room_id": "!MkDbyRqnvTYnoxjLYx:localhost", 
+                            "state_key": "@example:localhost", 
+                            "ts": 1409666587265, 
+                            "type": "m.room.member", 
+                            "user_id": "@example:localhost"
+                        }
+                    ], 
+                    "end": "s39_18_0", 
+                    "start": "t1-11_18_0"
+                }, 
+                "room_id": "!MkDbyRqnvTYnoxjLYx:localhost", 
+                "state": [
+                    {
+                        "content": {
+                            "creator": "@example:localhost"
+                        }, 
+                        "event_id": "dMUoqVTZca", 
+                        "required_power_level": 10, 
+                        "room_id": "!MkDbyRqnvTYnoxjLYx:localhost", 
+                        "state_key": "", 
+                        "ts": 1409665585188, 
+                        "type": "m.room.create", 
+                        "user_id": "@example:localhost"
+                    }, 
+                    {
+                        "content": {
+                            "@example:localhost": 10, 
+                            "default": 0
+                        }, 
+                        "event_id": "wAumPSTsWF", 
+                        "required_power_level": 10, 
+                        "room_id": "!MkDbyRqnvTYnoxjLYx:localhost", 
+                        "state_key": "", 
+                        "ts": 1409665585188, 
+                        "type": "m.room.power_levels", 
+                        "user_id": "@example:localhost"
+                    }, 
+                    {
+                        "content": {
+                            "join_rule": "public"
+                        }, 
+                        "event_id": "jrLVqKHKiI", 
+                        "required_power_level": 10, 
+                        "room_id": "!MkDbyRqnvTYnoxjLYx:localhost", 
+                        "state_key": "", 
+                        "ts": 1409665585188, 
+                        "type": "m.room.join_rules", 
+                        "user_id": "@example:localhost"
+                    }, 
+                    {
+                        "content": {
+                            "level": 10
+                        }, 
+                        "event_id": "WpmTgsNWUZ", 
+                        "required_power_level": 10, 
+                        "room_id": "!MkDbyRqnvTYnoxjLYx:localhost", 
+                        "state_key": "", 
+                        "ts": 1409665585188, 
+                        "type": "m.room.add_state_level", 
+                        "user_id": "@example:localhost"
+                    }, 
+                    {
+                        "content": {
+                            "level": 0
+                        }, 
+                        "event_id": "qUMBJyKsTQ", 
+                        "required_power_level": 10, 
+                        "room_id": "!MkDbyRqnvTYnoxjLYx:localhost", 
+                        "state_key": "", 
+                        "ts": 1409665585188, 
+                        "type": "m.room.send_event_level", 
+                        "user_id": "@example:localhost"
+                    }, 
+                    {
+                        "content": {
+                            "ban_level": 5, 
+                            "kick_level": 5
+                        }, 
+                        "event_id": "YAaDmKvoUW", 
+                        "required_power_level": 10, 
+                        "room_id": "!MkDbyRqnvTYnoxjLYx:localhost", 
+                        "state_key": "", 
+                        "ts": 1409665585188, 
+                        "type": "m.room.ops_levels", 
+                        "user_id": "@example:localhost"
+                    }, 
+                    {
+                        "content": {
+                            "membership": "invite"
+                        }, 
+                        "event_id": "YjNuBKnPsb", 
+                        "membership": "invite", 
+                        "room_id": "!MkDbyRqnvTYnoxjLYx:localhost", 
+                        "state_key": "@myfriend:localhost", 
+                        "ts": 1409666426819, 
+                        "type": "m.room.member", 
+                        "user_id": "@example:localhost"
+                    }, 
+                    {
+                        "content": {
+                            "avatar_url": null, 
+                            "displayname": null, 
+                            "membership": "join"
+                        }, 
+                        "event_id": "JFLVteSvQc", 
+                        "membership": "join", 
+                        "room_id": "!MkDbyRqnvTYnoxjLYx:localhost", 
+                        "state_key": "@example:localhost", 
+                        "ts": 1409666587265, 
+                        "type": "m.room.member", 
+                        "user_id": "@example:localhost"
+                    }
+                ]
+            }
+        ]
+    }
+    
+This returns all the room information the user is invited/joined on, as well as
+all of the presences relevant for these rooms. This can be a LOT of data. You
+may just want the most recent event for each room. This can be achieved by 
+applying query parameters to ``limit`` this request::
+
+    curl -XGET "http://localhost:8008/_matrix/client/api/v1/initialSync?limit=1&access_token=YOUR_ACCESS_TOKEN"
+    
+    {
+        "end": "s39_18_0", 
+        "presence": [
+            {
+                "content": {
+                    "last_active_ago": 1279484, 
+                    "user_id": "@example:localhost"
+                }, 
+                "type": "m.presence"
+            }
+        ], 
+        "rooms": [
+            {
+                "membership": "join", 
+                "messages": {
+                    "chunk": [
+                        {
+                            "content": {
+                                "avatar_url": null, 
+                                "displayname": null, 
+                                "membership": "join"
+                            }, 
+                            "event_id": "JFLVteSvQc", 
+                            "membership": "join", 
+                            "room_id": "!MkDbyRqnvTYnoxjLYx:localhost", 
+                            "state_key": "@example:localhost", 
+                            "ts": 1409666587265, 
+                            "type": "m.room.member", 
+                            "user_id": "@example:localhost"
+                        }
+                    ], 
+                    "end": "s39_18_0", 
+                    "start": "t10-30_18_0"
+                }, 
+                "room_id": "!MkDbyRqnvTYnoxjLYx:localhost", 
+                "state": [
+                    {
+                        "content": {
+                            "creator": "@example:localhost"
+                        }, 
+                        "event_id": "dMUoqVTZca", 
+                        "required_power_level": 10, 
+                        "room_id": "!MkDbyRqnvTYnoxjLYx:localhost", 
+                        "state_key": "", 
+                        "ts": 1409665585188, 
+                        "type": "m.room.create", 
+                        "user_id": "@example:localhost"
+                    }, 
+                    {
+                        "content": {
+                            "@example:localhost": 10, 
+                            "default": 0
+                        }, 
+                        "event_id": "wAumPSTsWF", 
+                        "required_power_level": 10, 
+                        "room_id": "!MkDbyRqnvTYnoxjLYx:localhost", 
+                        "state_key": "", 
+                        "ts": 1409665585188, 
+                        "type": "m.room.power_levels", 
+                        "user_id": "@example:localhost"
+                    }, 
+                    {
+                        "content": {
+                            "join_rule": "public"
+                        }, 
+                        "event_id": "jrLVqKHKiI", 
+                        "required_power_level": 10, 
+                        "room_id": "!MkDbyRqnvTYnoxjLYx:localhost", 
+                        "state_key": "", 
+                        "ts": 1409665585188, 
+                        "type": "m.room.join_rules", 
+                        "user_id": "@example:localhost"
+                    }, 
+                    {
+                        "content": {
+                            "level": 10
+                        }, 
+                        "event_id": "WpmTgsNWUZ", 
+                        "required_power_level": 10, 
+                        "room_id": "!MkDbyRqnvTYnoxjLYx:localhost", 
+                        "state_key": "", 
+                        "ts": 1409665585188, 
+                        "type": "m.room.add_state_level", 
+                        "user_id": "@example:localhost"
+                    }, 
+                    {
+                        "content": {
+                            "level": 0
+                        }, 
+                        "event_id": "qUMBJyKsTQ", 
+                        "required_power_level": 10, 
+                        "room_id": "!MkDbyRqnvTYnoxjLYx:localhost", 
+                        "state_key": "", 
+                        "ts": 1409665585188, 
+                        "type": "m.room.send_event_level", 
+                        "user_id": "@example:localhost"
+                    }, 
+                    {
+                        "content": {
+                            "ban_level": 5, 
+                            "kick_level": 5
+                        }, 
+                        "event_id": "YAaDmKvoUW", 
+                        "required_power_level": 10, 
+                        "room_id": "!MkDbyRqnvTYnoxjLYx:localhost", 
+                        "state_key": "", 
+                        "ts": 1409665585188, 
+                        "type": "m.room.ops_levels", 
+                        "user_id": "@example:localhost"
+                    }, 
+                    {
+                        "content": {
+                            "membership": "invite"
+                        }, 
+                        "event_id": "YjNuBKnPsb", 
+                        "membership": "invite", 
+                        "room_id": "!MkDbyRqnvTYnoxjLYx:localhost", 
+                        "state_key": "@myfriend:localhost", 
+                        "ts": 1409666426819, 
+                        "type": "m.room.member", 
+                        "user_id": "@example:localhost"
+                    }, 
+                    {
+                        "content": {
+                            "avatar_url": null, 
+                            "displayname": null, 
+                            "membership": "join"
+                        }, 
+                        "event_id": "JFLVteSvQc", 
+                        "membership": "join", 
+                        "room_id": "!MkDbyRqnvTYnoxjLYx:localhost", 
+                        "state_key": "@example:localhost", 
+                        "ts": 1409666587265, 
+                        "type": "m.room.member", 
+                        "user_id": "@example:localhost"
+                    }
+                ]
+            }
+        ]
+    }
+
+Getting live state
+------------------
+Once you know which rooms the client has previously interacted with, you need to
+listen for incoming events. This can be done like so::
+
+    curl -XGET "http://localhost:8008/_matrix/client/api/v1/events?access_token=YOUR_ACCESS_TOKEN"
+    
+    {
+        "chunk": [], 
+        "end": "s39_18_0", 
+        "start": "s39_18_0"
+    }
+    
+This will block waiting for an incoming event, timing out after several seconds.
+Even if there are no new events (as in the example above), there will be some
+pagination stream response keys. The client should make subsequent requests 
+using the value of the ``"end"`` key (in this case ``s39_18_0``) as the ``from`` 
+query parameter e.g. ``http://localhost:8008/_matrix/client/api/v1/events?access
+_token=YOUR_ACCESS_TOKEN&from=s39_18_0``. This value should be stored so when the 
+client reopens your app after a period of inactivity, you can resume from where 
+you got up to in the event stream. If it has been a long period of inactivity, 
+there may be LOTS of events waiting for the user. In this case, you may wish to 
+get all state instead and then resume getting live state from a newer end token.
+
+NB: The timeout can be changed by adding a ``timeout`` query parameter, which is
+in milliseconds. A timeout of 0 will not block.
+
+
+Example application
+-------------------
+The following example demonstrates registration and login, live event streaming,
+creating and joining rooms, sending messages, getting member lists and getting 
+historical messages for a room. This covers most functionality of a messaging
+application.
+
+`Try out the fiddle`__
+
+.. __: http://jsfiddle.net/gh/get/jquery/1.8.3/matrix-org/synapse/tree/master/jsfiddles/example_app

docs/client-server/swagger_matrix/api-docs

@@ -0,0 +1,46 @@
+{
+  "apiVersion": "1.0.0",
+  "swaggerVersion": "1.2",
+  "apis": [
+    {
+      "path": "-login",
+      "description": "Login operations"
+    },
+    {
+      "path": "-registration",
+      "description": "Registration operations"
+    },
+    {
+      "path": "-rooms",
+      "description": "Room operations"
+    },
+    {
+      "path": "-profile",
+      "description": "Profile operations"
+    },
+    {
+      "path": "-presence",
+      "description": "Presence operations"
+    },
+    {
+      "path": "-events",
+      "description": "Event operations"
+    },
+    {
+      "path": "-directory",
+      "description": "Directory operations"
+    }
+  ],
+  "authorizations": {
+    "token": {
+      "scopes": []
+    }
+  },
+  "info": {
+    "title": "Matrix Client-Server API Reference",
+    "description": "This contains the client-server API for the reference implementation of the home server",
+    "termsOfServiceUrl": "http://matrix.org",
+    "license": "Apache 2.0",
+    "licenseUrl": "http://www.apache.org/licenses/LICENSE-2.0.html"
+  }
+}

docs/client-server/swagger_matrix/api-docs-directory

@@ -0,0 +1,85 @@
+{
+  "apiVersion": "1.0.0",
+  "swaggerVersion": "1.2",
+  "basePath": "http://localhost:8008/_matrix/client/api/v1",
+  "resourcePath": "/directory",
+  "produces": [
+    "application/json"
+  ],
+  "apis": [
+    {
+      "path": "/directory/room/{roomAlias}",
+      "operations": [
+        {
+          "method": "GET",
+          "summary": "Get the room ID corresponding to this room alias.",
+          "notes": "Volatile: This API is likely to change.",
+          "type": "DirectoryResponse",
+          "nickname": "get_room_id_for_alias",
+          "parameters": [
+            {
+              "name": "roomAlias",
+              "description": "The room alias.",
+              "required": true,
+              "type": "string",
+              "paramType": "path"
+            }
+          ]
+        },
+        {
+          "method": "PUT",
+          "summary": "Create a new mapping from room alias to room ID.",
+          "notes": "Volatile: This API is likely to change.",
+          "type": "void",
+          "nickname": "add_room_alias",
+          "parameters": [
+            {
+              "name": "roomAlias",
+              "description": "The room alias to set.",
+              "required": true,
+              "type": "string",
+              "paramType": "path"
+            },
+            {
+              "name": "body",
+              "description": "The room ID to set.",
+              "required": true,
+              "type": "RoomAliasRequest",
+              "paramType": "body"
+            }
+          ]
+        }
+      ]
+    }
+  ],
+  "models": {
+    "DirectoryResponse": {
+      "id": "DirectoryResponse",
+      "properties": {
+        "room_id": {
+          "type": "string",
+          "description": "The fully-qualified room ID.",
+          "required": true
+        },
+        "servers": {
+          "type": "array",
+          "items": {
+            "$ref": "string"
+          },
+          "description": "A list of servers that know about this room.",
+          "required": true
+        }
+      }
+    },
+    "RoomAliasRequest": {
+      "id": "RoomAliasRequest",
+      "properties": {
+        "room_id": {
+          "type": "string",
+          "description": "The room ID to map the alias to.",
+          "required": true
+        }
+      }
+    }
+  }
+}

docs/client-server/swagger_matrix/api-docs-events

@@ -0,0 +1,247 @@
+{
+  "apiVersion": "1.0.0",
+  "swaggerVersion": "1.2",
+  "basePath": "http://localhost:8008/_matrix/client/api/v1",
+  "resourcePath": "/events",
+  "produces": [
+    "application/json"
+  ],
+  "apis": [
+    {
+      "path": "/events",
+      "operations": [
+        {
+          "method": "GET",
+          "summary": "Listen on the event stream",
+          "notes": "This can only be done by the logged in user. This will block until an event is received, or until the timeout is reached.",
+          "type": "PaginationChunk",
+          "nickname": "get_event_stream",
+          "parameters": [
+            {
+              "name": "from",
+              "description": "The token to stream from.",
+              "required": false,
+              "type": "string",
+              "paramType": "query"
+            },
+            {
+              "name": "timeout",
+              "description": "The maximum time in milliseconds to wait for an event.",
+              "required": false,
+              "type": "integer",
+              "paramType": "query"
+            }
+          ]
+        }
+      ],
+      
+      "responseMessages": [
+        {
+          "code": 400,
+          "message": "Bad pagination token."
+        }
+      ]
+    },
+    {
+      "path": "/events/{eventId}",
+      "operations": [
+        {
+          "method": "GET",
+          "summary": "Get information about a single event.",
+          "notes": "Get information about a single event.",
+          "type": "Event",
+          "nickname": "get_event",
+          "parameters": [
+            {
+              "name": "eventId",
+              "description": "The event ID to get.",
+              "required": true,
+              "type": "string",
+              "paramType": "path"
+            }
+          ],
+          "responseMessages": [
+            {
+              "code": 404,
+              "message": "Event not found."
+            }
+          ]
+        }
+      ]
+    },
+    {
+      "path": "/initialSync",
+      "operations": [
+        {
+          "method": "GET",
+          "summary": "Get this user's current state.",
+          "notes": "Get this user's current state.",
+          "type": "InitialSyncResponse",
+          "nickname": "initial_sync",
+          "parameters": [
+            {
+              "name": "limit",
+              "description": "The maximum number of messages to return for each room.",
+              "type": "integer",
+              "paramType": "query",
+              "required": false
+            }
+          ]
+        }
+      ]
+    },
+    {
+      "path": "/publicRooms",
+      "operations": [
+        {
+          "method": "GET",
+          "summary": "Get a list of publicly visible rooms.",
+          "type": "PublicRoomsPaginationChunk",
+          "nickname": "get_public_room_list"
+        }
+      ]
+    }
+  ],
+  "models": {
+    "PaginationChunk": {
+      "id": "PaginationChunk",
+      "properties": {
+        "start": {
+          "type": "string",
+          "description": "A token which correlates to the first value in \"chunk\" for paginating.",
+          "required": true
+        },
+        "end": {
+          "type": "string",
+          "description": "A token which correlates to the last value in \"chunk\" for paginating.",
+          "required": true
+        },
+        "chunk": {
+          "type": "array",
+          "description": "An array of events.",
+          "required": true,
+          "items": {
+            "$ref": "Event"
+          }
+        }
+      }
+    },
+    "Event": {
+      "id": "Event",
+      "properties": {
+        "event_id": {
+          "type": "string",
+          "description": "An ID which uniquely identifies this event.",
+          "required": true
+        },
+        "room_id": {
+          "type": "string",
+          "description": "The room in which this event occurred.",
+          "required": true
+        }
+      }
+    },
+    "PublicRoomInfo": {
+      "id": "PublicRoomInfo",
+      "properties": {
+        "aliases": {
+          "type": "array",
+          "description": "A list of room aliases for this room.",
+          "items": {
+            "$ref": "string"
+          }
+        },
+        "name": {
+          "type": "string",
+          "description": "The name of the room, as given by the m.room.name state event."
+        },
+        "room_id": {
+          "type": "string",
+          "description": "The room ID for this public room.",
+          "required": true
+        },
+        "topic": {
+          "type": "string",
+          "description": "The topic of this room, as given by the m.room.topic state event."
+        }
+      }
+    },
+    "PublicRoomsPaginationChunk": {
+      "id": "PublicRoomsPaginationChunk",
+      "properties": {
+        "start": {
+          "type": "string",
+          "description": "A token which correlates to the first value in \"chunk\" for paginating.",
+          "required": true
+        },
+        "end": {
+          "type": "string",
+          "description": "A token which correlates to the last value in \"chunk\" for paginating.",
+          "required": true
+        },
+        "chunk": {
+          "type": "array",
+          "description": "A list of public room data.",
+          "required": true,
+          "items": {
+            "$ref": "PublicRoomInfo"
+          }
+        }
+      }
+    },
+    "InitialSyncResponse": {
+      "id": "InitialSyncResponse",
+      "properties": {
+        "end": {
+          "type": "string",
+          "description": "A streaming token which can be used with /events to continue from this snapshot of data.",
+          "required": true
+        },
+        "presence": {
+          "type": "array",
+          "description": "A list of presence events.",
+          "items": {
+            "$ref": "Event"
+          },
+          "required": false
+        },
+        "rooms": {
+          "type": "array",
+          "description": "A list of initial sync room data.",
+          "required": false,
+          "items": {
+            "$ref": "InitialSyncRoomData"
+          }
+        }
+      }
+    },
+    "InitialSyncRoomData": {
+      "id": "InitialSyncRoomData",
+      "properties": {
+        "membership": {
+          "type": "string",
+          "description": "This user's membership state in this room.",
+          "required": true
+        },
+        "room_id": {
+          "type": "string",
+          "description": "The ID of this room.",
+          "required": true
+        },
+        "messages": {
+          "type": "PaginationChunk",
+          "description": "The most recent messages for this room, governed by the limit parameter.",
+          "required": false
+        },
+        "state": {
+          "type": "array",
+          "description": "A list of state events representing the current state of the room.",
+          "required": false,
+          "items": {
+            "$ref": "Event"
+          }
+        }
+      }
+    }
+  }
+}

docs/client-server/swagger_matrix/api-docs-login

@@ -0,0 +1,120 @@
+{
+  "apiVersion": "1.0.0", 
+  "apis": [
+    {
+      "operations": [
+        {
+          "method": "GET", 
+          "nickname": "get_login_info", 
+          "notes": "All login stages MUST be mentioned if there is >1 login type.", 
+          "summary": "Get the login mechanism to use when logging in.", 
+          "type": "LoginFlows"
+        }, 
+        {
+          "method": "POST", 
+          "nickname": "submit_login", 
+          "notes": "If this is part of a multi-stage login, there MUST be a 'session' key.", 
+          "parameters": [
+            {
+              "description": "A login submission", 
+              "name": "body", 
+              "paramType": "body", 
+              "required": true, 
+              "type": "LoginSubmission"
+            }
+          ], 
+          "responseMessages": [
+            {
+              "code": 400, 
+              "message": "Bad login type"
+            }, 
+            {
+              "code": 400, 
+              "message": "Missing JSON keys"
+            }
+          ], 
+          "summary": "Submit a login action.", 
+          "type": "LoginResult"
+        }
+      ], 
+      "path": "/login"
+    }
+  ], 
+  "basePath": "http://localhost:8008/_matrix/client/api/v1", 
+  "consumes": [
+    "application/json"
+  ], 
+  "models": {
+    "LoginFlows": {
+      "id": "LoginFlows",
+      "properties": {
+        "flows": {
+          "description": "A list of valid login flows.",
+          "type": "array",
+          "items": {
+            "$ref": "LoginInfo"
+          }
+        }
+      }
+    },
+    "LoginInfo": {
+      "id": "LoginInfo", 
+      "properties": {
+        "stages": {
+          "description": "Multi-stage login only: An array of all the login types required to login.", 
+          "items": {
+            "$ref": "string"
+          }, 
+          "type": "array"
+        }, 
+        "type": {
+          "description": "The login type that must be used when logging in.", 
+          "type": "string"
+        }
+      }
+    }, 
+    "LoginResult": {
+      "id": "LoginResult", 
+      "properties": {
+        "access_token": {
+          "description": "The access token for this user's login if this is the final stage of the login process.", 
+          "type": "string"
+        },
+        "user_id": {
+          "description": "The user's fully-qualified user ID.",
+          "type": "string"
+        }, 
+        "next": {
+          "description": "Multi-stage login only: The next login type to submit.", 
+          "type": "string"
+        },
+        "session": {
+          "description": "Multi-stage login only: The session token to send when submitting the next login type.",
+          "type": "string"
+        }
+      }
+    }, 
+    "LoginSubmission": {
+      "id": "LoginSubmission", 
+      "properties": {
+        "type": {
+          "description": "The type of login being submitted.", 
+          "type": "string"
+        },
+        "session": {
+          "description": "Multi-stage login only: The session token from an earlier login stage.",
+          "type": "string"
+        },
+        "_login_type_defined_keys_": {
+          "description": "Keys as defined by the specified login type, e.g. \"user\", \"password\""
+        }
+      }
+    }
+  }, 
+  "produces": [
+    "application/json"
+  ], 
+  "resourcePath": "/login", 
+  "swaggerVersion": "1.2"
+}
+

docs/client-server/swagger_matrix/api-docs-presence

@@ -0,0 +1,164 @@
+{
+  "apiVersion": "1.0.0",
+  "swaggerVersion": "1.2",
+  "basePath": "http://localhost:8008/_matrix/client/api/v1",
+  "resourcePath": "/presence",
+  "produces": [
+    "application/json"
+  ],
+  "consumes": [
+    "application/json"
+  ],
+  "apis": [
+    {
+      "path": "/presence/{userId}/status",
+      "operations": [
+        {
+          "method": "PUT",
+          "summary": "Update this user's presence state.",
+          "notes": "This can only be done by the logged in user.",
+          "type": "void",
+          "nickname": "update_presence",
+          "parameters": [
+            {
+              "name": "body",
+              "description": "The new presence state",
+              "required": true,
+              "type": "PresenceUpdate",
+              "paramType": "body"
+            },
+            {
+              "name": "userId",
+              "description": "The user whose presence to set.",
+              "required": true,
+              "type": "string",
+              "paramType": "path"
+            }
+          ]
+        },
+        {
+          "method": "GET",
+          "summary": "Get this user's presence state.",
+          "notes": "Get this user's presence state.",
+          "type": "PresenceUpdate",
+          "nickname": "get_presence",
+          "parameters": [
+            {
+              "name": "userId",
+              "description": "The user whose presence to get.",
+              "required": true,
+              "type": "string",
+              "paramType": "path"
+            }
+          ]
+        }
+      ]
+    },
+    {
+      "path": "/presence/list/{userId}",
+      "operations": [
+        {
+          "method": "GET",
+          "summary": "Retrieve a list of presences for all of this user's friends.",
+          "notes": "",
+          "type": "array",
+          "items": {
+            "$ref": "Presence"
+          },
+          "nickname": "get_presence_list",
+          "parameters": [
+            {
+              "name": "userId",
+              "description": "The user whose presence list to get.",
+              "required": true,
+              "type": "string",
+              "paramType": "path"
+            }
+          ]
+        },
+        {
+          "method": "POST",
+          "summary": "Add or remove users from this presence list.",
+          "notes": "Add or remove users from this presence list.",
+          "type": "void",
+          "nickname": "modify_presence_list",
+          "parameters": [
+            {
+              "name": "userId",
+              "description": "The user whose presence list is being modified.",
+              "required": true,
+              "type": "string",
+              "paramType": "path"
+            },
+            {
+              "name": "body",
+              "description": "The modifications to make to this presence list.",
+              "required": true,
+              "type": "PresenceListModifications",
+              "paramType": "body"
+            }
+          ]
+        }
+      ]
+    }
+  ],
+  "models": {
+    "PresenceUpdate": {
+      "id": "PresenceUpdate",
+      "properties": {
+        "presence": {
+          "type": "string",
+          "description": "Enum: The presence state.",
+          "enum": [
+            "offline",
+            "unavailable",
+            "online",
+            "free_for_chat"
+          ]
+        },
+        "status_msg": {
+          "type": "string",
+          "description": "The user-defined message associated with this presence state."
+        }
+      },
+      "subTypes": [
+        "Presence"
+      ]
+    },
+    "Presence": {
+      "id": "Presence",
+      "properties": {
+        "last_active_ago": {
+          "type": "integer",
+          "format": "int64",
+          "description": "The last time this user performed an action on their home server."
+        },
+        "user_id": {
+          "type": "string",
+          "description": "The fully qualified user ID"
+        }
+      }
+    },
+    "PresenceListModifications": {
+      "id": "PresenceListModifications",
+      "properties": {
+        "invite": {
+          "type": "array",
+          "description": "A list of user IDs to add to the list.",
+          "items": {
+            "type": "string",
+            "description": "A fully qualified user ID."
+          }
+        },
+        "drop": {
+          "type": "array",
+          "description": "A list of user IDs to remove from the list.",
+          "items": {
+            "type": "string",
+            "description": "A fully qualified user ID."
+          }
+        }
+      }
+    }
+  }
+}

docs/client-server/swagger_matrix/api-docs-profile

@@ -0,0 +1,122 @@
+{
+  "apiVersion": "1.0.0",
+  "swaggerVersion": "1.2",
+  "basePath": "http://localhost:8008/_matrix/client/api/v1",
+  "resourcePath": "/profile",
+  "produces": [
+    "application/json"
+  ],
+  "consumes": [
+    "application/json"
+  ],
+  "apis": [
+    {
+      "path": "/profile/{userId}/displayname",
+      "operations": [
+        {
+          "method": "PUT",
+          "summary": "Set a display name.",
+          "notes": "This can only be done by the logged in user.",
+          "type": "void",
+          "nickname": "set_display_name",
+          "parameters": [
+            {
+              "name": "body",
+              "description": "The new display name for this user.",
+              "required": true,
+              "type": "DisplayName",
+              "paramType": "body"
+            },
+            {
+              "name": "userId",
+              "description": "The user whose display name to set.",
+              "required": true,
+              "type": "string",
+              "paramType": "path"
+            }
+          ]
+        },
+        {
+          "method": "GET",
+          "summary": "Get a display name.",
+          "notes": "This can be done by anyone.",
+          "type": "DisplayName",
+          "nickname": "get_display_name",
+          "parameters": [
+            {
+              "name": "userId",
+              "description": "The user whose display name to get.",
+              "required": true,
+              "type": "string",
+              "paramType": "path"
+            }
+          ]
+        }
+      ]
+    },
+    {
+      "path": "/profile/{userId}/avatar_url",
+      "operations": [
+        {
+          "method": "PUT",
+          "summary": "Set an avatar URL.",
+          "notes": "This can only be done by the logged in user.",
+          "type": "void",
+          "nickname": "set_avatar_url",
+          "parameters": [
+            {
+              "name": "body",
+              "description": "The new avatar url for this user.",
+              "required": true,
+              "type": "AvatarUrl",
+              "paramType": "body"
+            },
+            {
+              "name": "userId",
+              "description": "The user whose avatar url to set.",
+              "required": true,
+              "type": "string",
+              "paramType": "path"
+            }
+          ]
+        },
+        {
+          "method": "GET",
+          "summary": "Get an avatar url.",
+          "notes": "This can be done by anyone.",
+          "type": "AvatarUrl",
+          "nickname": "get_avatar_url",
+          "parameters": [
+            {
+              "name": "userId",
+              "description": "The user whose avatar url to get.",
+              "required": true,
+              "type": "string",
+              "paramType": "path"
+            }
+          ]
+        }
+      ]
+    }
+  ],
+  "models": {
+    "DisplayName": {
+      "id": "DisplayName",
+      "properties": {
+        "displayname": {
+          "type": "string",
+          "description": "The textual display name"
+        }
+      }
+    },
+    "AvatarUrl": {
+      "id": "AvatarUrl",
+      "properties": {
+        "avatar_url": {
+          "type": "string",
+          "description": "A url to an image representing an avatar."
+        }
+      }
+    }
+  }
+}

docs/client-server/swagger_matrix/api-docs-registration

@@ -0,0 +1,120 @@
+{
+  "apiVersion": "1.0.0", 
+  "apis": [
+    {
+      "operations": [
+        {
+          "method": "GET", 
+          "nickname": "get_registration_info", 
+          "notes": "All login stages MUST be mentioned if there is >1 login type.", 
+          "summary": "Get the login mechanism to use when registering.", 
+          "type": "RegistrationFlows"
+        }, 
+        {
+          "method": "POST", 
+          "nickname": "submit_registration", 
+          "notes": "If this is part of a multi-stage registration, there MUST be a 'session' key.", 
+          "parameters": [
+            {
+              "description": "A registration submission", 
+              "name": "body", 
+              "paramType": "body", 
+              "required": true, 
+              "type": "RegistrationSubmission"
+            }
+          ], 
+          "responseMessages": [
+            {
+              "code": 400, 
+              "message": "Bad login type"
+            }, 
+            {
+              "code": 400, 
+              "message": "Missing JSON keys"
+            }
+          ], 
+          "summary": "Submit a registration action.", 
+          "type": "RegistrationResult"
+        }
+      ], 
+      "path": "/register"
+    }
+  ], 
+  "basePath": "http://localhost:8008/_matrix/client/api/v1", 
+  "consumes": [
+    "application/json"
+  ], 
+  "models": {
+    "RegistrationFlows": {
+      "id": "RegistrationFlows",
+      "properties": {
+        "flows": {
+          "description": "A list of valid registration flows.",
+          "type": "array",
+          "items": {
+            "$ref": "RegistrationInfo"
+          }
+        }
+      }
+    },
+    "RegistrationInfo": {
+      "id": "RegistrationInfo", 
+      "properties": {
+        "stages": {
+          "description": "Multi-stage registration only: An array of all the login types required to registration.", 
+          "items": {
+            "$ref": "string"
+          }, 
+          "type": "array"
+        }, 
+        "type": {
+          "description": "The first login type that must be used when logging in.", 
+          "type": "string"
+        }
+      }
+    }, 
+    "RegistrationResult": {
+      "id": "RegistrationResult", 
+      "properties": {
+        "access_token": {
+          "description": "The access token for this user's registration if this is the final stage of the registration process.", 
+          "type": "string"
+        },
+        "user_id": {
+          "description": "The user's fully-qualified user ID.",
+          "type": "string"
+        }, 
+        "next": {
+          "description": "Multi-stage registration only: The next registration type to submit.", 
+          "type": "string"
+        },
+        "session": {
+          "description": "Multi-stage registration only: The session token to send when submitting the next registration type.",
+          "type": "string"
+        }
+      }
+    }, 
+    "RegistrationSubmission": {
+      "id": "RegistrationSubmission", 
+      "properties": {
+        "type": {
+          "description": "The type of registration being submitted.", 
+          "type": "string"
+        },
+        "session": {
+          "description": "Multi-stage registration only: The session token from an earlier registration stage.",
+          "type": "string"
+        },
+        "_registration_type_defined_keys_": {
+          "description": "Keys as defined by the specified registration type, e.g. \"user\", \"password\""
+        }
+      }
+    }
+  }, 
+  "produces": [
+    "application/json"
+  ], 
+  "resourcePath": "/register", 
+  "swaggerVersion": "1.2"
+}
+

docs/client-server/swagger_matrix/api-docs-rooms

@@ -0,0 +1,977 @@
+{
+  "apiVersion": "1.0.0",
+  "swaggerVersion": "1.2",
+  "basePath": "http://localhost:8008/_matrix/client/api/v1", 
+  "resourcePath": "/rooms",
+  "produces": [
+    "application/json"
+  ],
+  "consumes": [
+    "application/json"
+  ],
+  "authorizations": {
+    "token": []
+  },
+  "apis": [
+    {
+      "path": "/rooms/{roomId}/send/{eventType}",
+      "operations": [
+        {
+          "method": "POST",
+          "summary": "Send a generic non-state event to this room.",
+          "notes": "This operation can also be done as a PUT by suffixing /{txnId}.",
+          "type": "EventId",
+          "nickname": "send_non_state_event",
+          "consumes": [
+            "application/json"
+          ],
+          "parameters": [
+            {
+              "name": "body",
+              "description": "The event contents",
+              "required": true,
+              "type": "EventContent",
+              "paramType": "body"
+            },
+            {
+              "name": "roomId",
+              "description": "The room to send the message in.",
+              "required": true,
+              "type": "string",
+              "paramType": "path"
+            },
+            {
+              "name": "eventType",
+              "description": "The type of event to send.",
+              "required": true,
+              "type": "string",
+              "paramType": "path"
+            }
+          ]
+        }
+      ]
+    },
+    {
+      "path": "/rooms/{roomId}/state/{eventType}/{stateKey}",
+      "operations": [
+        {
+          "method": "PUT",
+          "summary": "Send a generic state event to this room.",
+          "notes": "The state key can be omitted, such that you can PUT to /rooms/{roomId}/state/{eventType}. The state key defaults to a 0 length string in this case.",
+          "type": "void",
+          "nickname": "send_state_event",
+          "consumes": [
+            "application/json"
+          ],
+          "parameters": [
+            {
+              "name": "body",
+              "description": "The event contents",
+              "required": true,
+              "type": "EventContent",
+              "paramType": "body"
+            },
+            {
+              "name": "roomId",
+              "description": "The room to send the message in.",
+              "required": true,
+              "type": "string",
+              "paramType": "path"
+            },
+            {
+              "name": "eventType",
+              "description": "The type of event to send.",
+              "required": true,
+              "type": "string",
+              "paramType": "path"
+            },
+            {
+              "name": "stateKey",
+              "description": "An identifier used to specify clobbering semantics. State events with the same (roomId, eventType, stateKey) will be replaced.",
+              "required": true,
+              "type": "string",
+              "paramType": "path"
+            }
+          ]
+        }
+      ]
+    },
+    {
+      "path": "/rooms/{roomId}/send/m.room.message",
+      "operations": [
+        {
+          "method": "POST",
+          "summary": "Send a message in this room.",
+          "notes": "This operation can also be done as a PUT by suffixing /{txnId}.",
+          "type": "EventId",
+          "nickname": "send_message",
+          "consumes": [
+            "application/json"
+          ],
+          "parameters": [
+            {
+              "name": "body",
+              "description": "The message contents",
+              "required": true,
+              "type": "Message",
+              "paramType": "body"
+            },
+            {
+              "name": "roomId",
+              "description": "The room to send the message in.",
+              "required": true,
+              "type": "string",
+              "paramType": "path"
+            }
+          ]
+        }
+      ]
+    },
+    {
+      "path": "/rooms/{roomId}/state/m.room.topic",
+      "operations": [
+        {
+          "method": "PUT",
+          "summary": "Set the topic for this room.",
+          "notes": "Set the topic for this room.",
+          "type": "void",
+          "nickname": "set_topic",
+          "consumes": [
+            "application/json"
+          ],
+          "parameters": [
+            {
+              "name": "body",
+              "description": "The topic contents",
+              "required": true,
+              "type": "Topic",
+              "paramType": "body"
+            },
+            {
+              "name": "roomId",
+              "description": "The room to set the topic in.",
+              "required": true,
+              "type": "string",
+              "paramType": "path"
+            }
+          ]
+        },
+        {
+          "method": "GET",
+          "summary": "Get the topic for this room.",
+          "notes": "Get the topic for this room.",
+          "type": "Topic",
+          "nickname": "get_topic",
+          "parameters": [
+            {
+              "name": "roomId",
+              "description": "The room to get topic in.",
+              "required": true,
+              "type": "string",
+              "paramType": "path"
+            }
+          ],
+          "responseMessages": [
+            {
+              "code": 404,
+              "message": "Topic not found."
+            }
+          ]
+        }
+      ]
+    },
+    {
+      "path": "/rooms/{roomId}/state/m.room.name",
+      "operations": [
+        {
+          "method": "PUT",
+          "summary": "Set the name of this room.",
+          "notes": "Set the name of this room.",
+          "type": "void",
+          "nickname": "set_room_name",
+          "consumes": [
+            "application/json"
+          ],
+          "parameters": [
+            {
+              "name": "body",
+              "description": "The name contents",
+              "required": true,
+              "type": "RoomName",
+              "paramType": "body"
+            },
+            {
+              "name": "roomId",
+              "description": "The room to set the name of.",
+              "required": true,
+              "type": "string",
+              "paramType": "path"
+            }
+          ]
+        },
+        {
+          "method": "GET",
+          "summary": "Get the room's name.",
+          "notes": "",
+          "type": "RoomName",
+          "nickname": "get_room_name",
+          "parameters": [
+            {
+              "name": "roomId",
+              "description": "The room to get the name of.",
+              "required": true,
+              "type": "string",
+              "paramType": "path"
+            }
+          ],
+          "responseMessages": [
+            {
+              "code": 404,
+              "message": "Name not found."
+            }
+          ]
+        }
+      ]
+    },
+    {
+      "path": "/rooms/{roomId}/send/m.room.message.feedback",
+      "operations": [
+        {
+          "method": "POST",
+          "summary": "Send feedback to a message.",
+          "notes": "This operation can also be done as a PUT by suffixing /{txnId}.",
+          "type": "EventId",
+          "nickname": "send_feedback",
+          "consumes": [
+            "application/json"
+          ],
+          "parameters": [
+            {
+              "name": "body",
+              "description": "The feedback contents",
+              "required": true,
+              "type": "Feedback",
+              "paramType": "body"
+            },
+            {
+              "name": "roomId",
+              "description": "The room to send the feedback in.",
+              "required": true,
+              "type": "string",
+              "paramType": "path"
+            }
+          ],
+          "responseMessages": [
+            {
+              "code": 400,
+              "message": "Bad feedback type."
+            }
+          ]
+        }
+      ]
+    },
+    {
+      "path": "/rooms/{roomId}/invite",
+      "operations": [
+        {
+          "method": "POST",
+          "summary": "Invite a user to this room.",
+          "notes": "This operation can also be done as a PUT by suffixing /{txnId}.",
+          "type": "void",
+          "nickname": "invite",
+          "consumes": [
+            "application/json"
+          ],
+          "parameters": [
+            {
+              "name": "roomId",
+              "description": "The room which has this user.",
+              "required": true,
+              "type": "string",
+              "paramType": "path"
+            },
+            {
+              "name": "body",
+              "description": "The user to invite.",
+              "required": true,
+              "type": "InviteRequest",
+              "paramType": "body"
+            }
+          ]
+        }  
+      ]
+    },
+    {
+      "path": "/rooms/{roomId}/join",
+      "operations": [
+        {
+          "method": "POST",
+          "summary": "Join this room.",
+          "notes": "This operation can also be done as a PUT by suffixing /{txnId}.",
+          "type": "void",
+          "nickname": "join_room",
+          "consumes": [
+            "application/json"
+          ],
+          "parameters": [
+            {
+              "name": "roomId",
+              "description": "The room to join.",
+              "required": true,
+              "type": "string",
+              "paramType": "path"
+            },
+            {
+              "name": "body",
+              "required": true,
+              "type": "JoinRequest",
+              "paramType": "body"
+            }
+          ]
+        }  
+      ]
+    },
+    {
+      "path": "/rooms/{roomId}/leave",
+      "operations": [
+        {
+          "method": "POST",
+          "summary": "Leave this room.",
+          "notes": "This operation can also be done as a PUT by suffixing /{txnId}.",
+          "type": "void",
+          "nickname": "leave",
+          "consumes": [
+            "application/json"
+          ],
+          "parameters": [
+            {
+              "name": "roomId",
+              "description": "The room to leave.",
+              "required": true,
+              "type": "string",
+              "paramType": "path"
+            },
+            {
+              "name": "body",
+              "required": true,
+              "type": "LeaveRequest",
+              "paramType": "body"
+            }
+          ]
+        }  
+      ]
+    },
+    {
+      "path": "/rooms/{roomId}/ban",
+      "operations": [
+        {
+          "method": "POST",
+          "summary": "Ban a user in the room.",
+          "notes": "This operation can also be done as a PUT by suffixing /{txnId}. The caller must have the required power level to do this operation.",
+          "type": "void",
+          "nickname": "ban",
+          "consumes": [
+            "application/json"
+          ],
+          "parameters": [
+            {
+              "name": "roomId",
+              "description": "The room which has the user to ban.",
+              "required": true,
+              "type": "string",
+              "paramType": "path"
+            },
+            {
+              "name": "body",
+              "description": "The user to ban.",
+              "required": true,
+              "type": "BanRequest",
+              "paramType": "body"
+            }
+          ]
+        }  
+      ]
+    },
+    {
+      "path": "/rooms/{roomId}/state/m.room.member/{userId}",
+      "operations": [
+        {
+          "method": "PUT",
+          "summary": "Change the membership state for a user in a room.",
+          "notes": "Change the membership state for a user in a room.",
+          "type": "void",
+          "nickname": "set_membership",
+          "consumes": [
+            "application/json"
+          ],
+          "parameters": [
+            {
+              "name": "body",
+              "description": "The new membership state",
+              "required": true,
+              "type": "Member",
+              "paramType": "body"
+            },
+            {
+              "name": "userId",
+              "description": "The user whose membership is being changed.",
+              "required": true,
+              "type": "string",
+              "paramType": "path"
+            },
+            {
+              "name": "roomId",
+              "description": "The room which has this user.",
+              "required": true,
+              "type": "string",
+              "paramType": "path"
+            }
+          ],
+          "responseMessages": [
+            {
+              "code": 400,
+              "message": "No membership key."
+            },
+            {
+              "code": 400,
+              "message": "Bad membership value."
+            },
+            {
+              "code": 403,
+              "message": "When inviting: You are not in the room."
+            },
+            {
+              "code": 403,
+              "message": "When inviting: <target> is already in the room."
+            },
+            {
+              "code": 403,
+              "message": "When joining: Cannot force another user to join."
+            },
+            {
+              "code": 403,
+              "message": "When joining: You are not invited to this room."
+            }
+          ]
+        },
+        {
+          "method": "GET",
+          "summary": "Get the membership state of a user in a room.",
+          "notes": "Get the membership state of a user in a room.",
+          "type": "Member",
+          "nickname": "get_membership",
+          "parameters": [
+            {
+              "name": "userId",
+              "description": "The user whose membership state you want to get.",
+              "required": true,
+              "type": "string",
+              "paramType": "path"
+            },
+            {
+              "name": "roomId",
+              "description": "The room which has this user.",
+              "required": true,
+              "type": "string",
+              "paramType": "path"
+            }
+          ],
+          "responseMessages": [
+            {
+              "code": 404,
+              "message": "Member not found."
+            }
+          ]
+        }
+      ]
+    },
+    {
+      "path": "/join/{roomAliasOrId}",
+      "operations": [
+        {
+          "method": "POST",
+          "summary": "Join a room via a room alias or room ID.",
+          "notes": "Join a room via a room alias or room ID.",
+          "type": "JoinRoomInfo",
+          "nickname": "join",
+          "consumes": [
+            "application/json"
+          ],
+          "parameters": [
+            {
+              "name": "roomAliasOrId",
+              "description": "The room alias or room ID to join.",
+              "required": true,
+              "type": "string",
+              "paramType": "path"
+            }
+          ],
+          "responseMessages": [
+            {
+              "code": 400,
+              "message": "Bad room alias."
+            }
+          ]
+        }
+      ]
+    },
+    {
+      "path": "/createRoom",
+      "operations": [
+        {
+          "method": "POST",
+          "summary": "Create a room.",
+          "notes": "Create a room.",
+          "type": "RoomInfo",
+          "nickname": "create_room",
+          "consumes": [
+            "application/json"
+          ],
+          "parameters": [
+            {
+              "name": "body",
+              "description": "The desired configuration for the room. This operation can also be done as a PUT by suffixing /{txnId}.",
+              "required": true,
+              "type": "RoomConfig",
+              "paramType": "body"
+            }
+          ],
+          "responseMessages": [
+            {
+              "code": 400,
+              "message": "Body must be JSON."
+            },
+            {
+              "code": 400,
+              "message": "Room alias already taken."
+            }
+          ]
+        }
+      ]
+    },
+    {
+      "path": "/rooms/{roomId}/messages",
+      "operations": [
+        {
+          "method": "GET",
+          "summary": "Get a list of messages for this room.",
+          "notes": "Get a list of messages for this room.",
+          "type": "MessagePaginationChunk",
+          "nickname": "get_messages",
+          "parameters": [
+            {
+              "name": "roomId",
+              "description": "The room to get messages in.",
+              "required": true,
+              "type": "string",
+              "paramType": "path"
+            },
+            {
+              "name": "from",
+              "description": "The token to start getting results from.",
+              "required": false,
+              "type": "string",
+              "paramType": "query"
+            },
+            {
+              "name": "to",
+              "description": "The token to stop getting results at.",
+              "required": false,
+              "type": "string",
+              "paramType": "query"
+            },
+            {
+              "name": "limit",
+              "description": "The maximum number of messages to return.",
+              "required": false,
+              "type": "integer",
+              "paramType": "query"
+            }
+          ]
+        }
+      ]
+    },
+    {
+      "path": "/rooms/{roomId}/members",
+      "operations": [
+        {
+          "method": "GET",
+          "summary": "Get a list of members for this room.",
+          "notes": "Get a list of members for this room.",
+          "type": "MemberPaginationChunk",
+          "nickname": "get_members",
+          "parameters": [
+            {
+              "name": "roomId",
+              "description": "The room to get a list of members from.",
+              "required": true,
+              "type": "string",
+              "paramType": "path"
+            },
+            {
+              "name": "from",
+              "description": "The token to start getting results from.",
+              "required": false,
+              "type": "string",
+              "paramType": "query"
+            },
+            {
+              "name": "to",
+              "description": "The token to stop getting results at.",
+              "required": false,
+              "type": "string",
+              "paramType": "query"
+            },
+            {
+              "name": "limit",
+              "description": "The maximum number of members to return.",
+              "required": false,
+              "type": "integer",
+              "paramType": "query"
+            }
+          ]
+        }
+      ]
+    },
+    {
+      "path": "/rooms/{roomId}/state",
+      "operations": [
+        {
+          "method": "GET",
+          "summary": "Get a list of all the current state events for this room.",
+          "notes": "NOT YET IMPLEMENTED.",
+          "type": "array",
+          "items": {
+            "$ref": "Event"
+          },
+          "nickname": "get_state_events",
+          "parameters": [
+            {
+              "name": "roomId",
+              "description": "The room to get a list of current state events from.",
+              "required": true,
+              "type": "string",
+              "paramType": "path"
+            }
+          ]
+        }
+      ]
+    },
+    {
+      "path": "/rooms/{roomId}/initialSync",
+      "operations": [
+        {
+          "method": "GET",
+          "summary": "Get all the current information for this room, including messages and state events.",
+          "notes": "NOT YET IMPLEMENTED.",
+          "type": "InitialSyncRoomData",
+          "nickname": "get_room_sync_data",
+          "parameters": [
+            {
+              "name": "roomId",
+              "description": "The room to get information for.",
+              "required": true,
+              "type": "string",
+              "paramType": "path"
+            }
+          ]
+        }
+      ]
+    }
+  ],
+  "models": {
+    "Topic": {
+      "id": "Topic",
+      "properties": {
+        "topic": {
+          "type": "string",
+          "description": "The topic text"
+        }
+      }
+    },
+    "RoomName": {
+      "id": "RoomName",
+      "properties": {
+        "name": {
+          "type": "string",
+          "description": "The human-readable name for the room. Can contain spaces."
+        }
+      }
+    },
+    "Message": {
+      "id": "Message",
+      "properties": {
+        "msgtype": {
+          "type": "string",
+          "description": "The type of message being sent, e.g. \"m.text\"",
+          "required": true
+        },
+        "_msgtype_defined_keys_": {
+          "description": "Additional keys as defined by the msgtype, e.g. \"body\""
+        }
+      }
+    },
+    "Feedback": {
+      "id": "Feedback",
+      "properties": {
+        "target_event_id": {
+          "type": "string",
+          "description": "The event ID being acknowledged.",
+          "required": true
+        },
+        "type": {
+          "type": "string",
+          "description": "The type of feedback. Either 'delivered' or 'read'.",
+          "required": true
+        }
+      }
+    },
+    "Member": {
+      "id": "Member",
+      "properties": {
+        "membership": {
+          "type": "string",
+          "description": "Enum: The membership state of this member.",
+          "enum": [
+            "invite",
+            "join",
+            "leave",
+            "ban"
+          ]
+        }
+      }
+    },
+    "RoomInfo": {
+      "id": "RoomInfo",
+      "properties": {
+        "room_id": {
+          "type": "string",
+          "description": "The allocated room ID.",
+          "required": true
+        },
+        "room_alias": {
+          "type": "string",
+          "description": "The alias for the room.",
+          "required": false
+        }
+      }
+    },
+    "JoinRoomInfo": {
+      "id": "JoinRoomInfo",
+      "properties": {
+        "room_id": {
+          "type": "string",
+          "description": "The room ID joined, if joined via a room alias only.",
+          "required": true
+        }
+      }
+    },
+    "RoomConfig": {
+      "id": "RoomConfig",
+      "properties": {
+        "visibility": {
+          "type": "string",
+          "description": "Enum: The room visibility.",
+          "required": false,
+          "enum": [
+            "public",
+            "private"
+          ]
+        },
+        "room_alias_name": {
+          "type": "string",
+          "description": "The alias to give the new room.",
+          "required": false
+        },
+        "name": {
+          "type": "string",
+          "description": "Sets the name of the room. Send a m.room.name event after creating the room with the 'name' key specified.",
+          "required": false
+        },
+        "topic": {
+          "type": "string",
+          "description": "Sets the topic for the room. Send a m.room.topic event after creating the room with the 'topic' key specified.",
+          "required": false
+        }
+      }
+    },
+    "PaginationRequest": {
+      "id": "PaginationRequest",
+      "properties": {
+        "from": {
+          "type": "string",
+          "description": "The token to start getting results from."
+        },
+        "to": {
+          "type": "string",
+          "description": "The token to stop getting results at."
+        },
+        "limit": {
+          "type": "integer",
+          "description": "The maximum number of entries to return."
+        }
+      }
+    },
+    "PaginationChunk": {
+      "id": "PaginationChunk",
+      "properties": {
+        "start": {
+          "type": "string",
+          "description": "A token which correlates to the first value in \"chunk\" for paginating.",
+          "required": true
+        },
+        "end": {
+          "type": "string",
+          "description": "A token which correlates to the last value in \"chunk\" for paginating.",
+          "required": true
+        }
+      },
+      "subTypes": [
+        "MessagePaginationChunk"
+      ]
+    },
+    "MessagePaginationChunk": {
+      "id": "MessagePaginationChunk",
+      "properties": {
+        "chunk": {
+          "type": "array",
+          "description": "A list of message events.",
+          "items": {
+            "$ref": "MessageEvent"
+          },
+          "required": true
+        }
+      }
+    },
+    "MemberPaginationChunk": {
+      "id": "MemberPaginationChunk",
+      "properties": {
+        "chunk": {
+          "type": "array",
+          "description": "A list of member events.",
+          "items": {
+            "$ref": "MemberEvent"
+          },
+          "required": true
+        }
+      }
+    },
+    "Event": {
+      "id": "Event",
+      "properties": {
+        "event_id": {
+          "type": "string",
+          "description": "An ID which uniquely identifies this event. This is automatically set by the server.",
+          "required": true
+        },
+        "room_id": {
+          "type": "string",
+          "description": "The room in which this event occurred. This is automatically set by the server.",
+          "required": true
+        },
+        "type": {
+          "type": "string",
+          "description": "The event type.",
+          "required": true
+        }
+      },
+      "subTypes": [
+        "MessageEvent"
+      ]
+    },
+    "EventId": {
+      "id": "EventId",
+      "properties": {
+        "event_id": {
+          "type": "string",
+          "description": "The allocated event ID for this event.",
+          "required": true
+        }
+      }
+    },
+    "EventContent": {
+      "id": "EventContent",
+      "properties": {
+        "__event_content_keys__": {
+          "type": "string",
+          "description": "Event-specific content keys and values.",
+          "required": false
+        }
+      }
+    },
+    "MessageEvent": {
+      "id": "MessageEvent",
+      "properties": {
+        "content": {
+          "type": "Message"
+        }
+      }
+    },
+    "MemberEvent": {
+      "id": "MemberEvent",
+      "properties": {
+        "content": {
+          "type": "Member"
+        }
+      }
+    },
+    "InviteRequest": {
+      "id": "InviteRequest",
+      "properties": {
+        "user_id": {
+          "type": "string",
+          "description": "The fully-qualified user ID."
+        }
+      }
+    },
+    "JoinRequest": {
+      "id": "JoinRequest",
+      "properties": {}
+    },
+    "LeaveRequest": {
+      "id": "LeaveRequest",
+      "properties": {}
+    },
+    "BanRequest": {
+      "id": "BanRequest",
+      "properties": {
+        "user_id": {
+          "type": "string",
+          "description": "The fully-qualified user ID."
+        },
+        "reason": {
+          "type": "string",
+          "description": "The reason for the ban."
+        }
+      }
+    },
+    "InitialSyncRoomData": {
+      "id": "InitialSyncRoomData",
+      "properties": {
+        "membership": {
+          "type": "string",
+          "description": "This user's membership state in this room.",
+          "required": true
+        },
+        "room_id": {
+          "type": "string",
+          "description": "The ID of this room.",
+          "required": true
+        },
+        "messages": {
+          "type": "MessagePaginationChunk",
+          "description": "The most recent messages for this room, governed by the limit parameter.",
+          "required": false
+        },
+        "state": {
+          "type": "array",
+          "description": "A list of state events representing the current state of the room.",
+          "required": false,
+          "items": {
+            "$ref": "Event"
+          }
+        }
+      }
+    }
+  }
+}

docs/client-server/urls.rst

@@ -1,92 +0,0 @@
-=========================
-Client-Server URL Summary
-=========================
-
-A brief overview of the URL scheme involved in the Synapse Client-Server API.
-
-
-URLs
-====
-
-Fetch events:
-  GET /events
-
-Registering an account
-  POST /register
-
-Unregistering an account
-  POST /unregister
-
-Rooms
------
-
-Creating a room by ID
-  PUT /rooms/$roomid
-
-Creating an anonymous room
-  POST /rooms
-
-Room topic
-  GET /rooms/$roomid/topic
-  PUT /rooms/$roomid/topic
-
-List rooms
-  GET /rooms/list
-
-Invite/Join/Leave
-  GET    /rooms/$roomid/members/$userid/state
-  PUT    /rooms/$roomid/members/$userid/state
-  DELETE /rooms/$roomid/members/$userid/state
-
-List members
-  GET  /rooms/$roomid/members/list
-
-Sending/reading messages
-  PUT /rooms/$roomid/messages/$sender/$msgid
-
-Feedback
-  GET /rooms/$roomid/messages/$sender/$msgid/feedback/$feedbackuser/$feedback
-  PUT /rooms/$roomid/messages/$sender/$msgid/feedback/$feedbackuser/$feedback
-
-Paginating messages
-  GET /rooms/$roomid/messages/list
-
-Profiles
---------
-
-Display name
-  GET /profile/$userid/displayname
-  PUT /profile/$userid/displayname
-
-Avatar URL
-  GET /profile/$userid/avatar_url
-  PUT /profile/$userid/avatar_url
-
-Metadata
-  GET  /profile/$userid/metadata
-  POST /profile/$userid/metadata
-
-Presence
---------
-
-My state or status message
-  GET /presence/$userid/status
-  PUT /presence/$userid/status
-    also 'GET' for fetching others
-
-TODO(paul): per-device idle time, device type; similar to above
-
-My presence list
-  GET  /presence_list/$myuserid
-  POST /presence_list/$myuserid
-    body is JSON-encoded dict of keys:
-      invite: list of UserID strings to invite
-      drop: list of UserID strings to remove
-      TODO(paul): define other ops: accept, group management, ordering?
-
-Presence polling start/stop
-  POST /presence_list/$myuserid?op=start
-  POST /presence_list/$myuserid?op=stop
-
-Presence invite
-  POST /presence_list/$myuserid/invite/$targetuserid

docs/client-server/web/README

@@ -0,0 +1,5 @@
+To get this running:
+  ln -s ../swagger_matrix
+  python -m SimpleHTTPServer
+
+Go to http://localhost:8000/swagger.html

docs/client-server/web/files/backbone-min.js

@@ -0,0 +1,38 @@
+// Backbone.js 0.9.2
+
+// (c) 2010-2012 Jeremy Ashkenas, DocumentCloud Inc.
+// Backbone may be freely distributed under the MIT license.
+// For all details and documentation:
+// http://backbonejs.org
+(function(){var l=this,y=l.Backbone,z=Array.prototype.slice,A=Array.prototype.splice,g;g="undefined"!==typeof exports?exports:l.Backbone={};g.VERSION="0.9.2";var f=l._;!f&&"undefined"!==typeof require&&(f=require("underscore"));var i=l.jQuery||l.Zepto||l.ender;g.setDomLibrary=function(a){i=a};g.noConflict=function(){l.Backbone=y;return this};g.emulateHTTP=!1;g.emulateJSON=!1;var p=/\s+/,k=g.Events={on:function(a,b,c){var d,e,f,g,j;if(!b)return this;a=a.split(p);for(d=this._callbacks||(this._callbacks=
+{});e=a.shift();)f=(j=d[e])?j.tail:{},f.next=g={},f.context=c,f.callback=b,d[e]={tail:g,next:j?j.next:f};return this},off:function(a,b,c){var d,e,h,g,j,q;if(e=this._callbacks){if(!a&&!b&&!c)return delete this._callbacks,this;for(a=a?a.split(p):f.keys(e);d=a.shift();)if(h=e[d],delete e[d],h&&(b||c))for(g=h.tail;(h=h.next)!==g;)if(j=h.callback,q=h.context,b&&j!==b||c&&q!==c)this.on(d,j,q);return this}},trigger:function(a){var b,c,d,e,f,g;if(!(d=this._callbacks))return this;f=d.all;a=a.split(p);for(g=
+z.call(arguments,1);b=a.shift();){if(c=d[b])for(e=c.tail;(c=c.next)!==e;)c.callback.apply(c.context||this,g);if(c=f){e=c.tail;for(b=[b].concat(g);(c=c.next)!==e;)c.callback.apply(c.context||this,b)}}return this}};k.bind=k.on;k.unbind=k.off;var o=g.Model=function(a,b){var c;a||(a={});b&&b.parse&&(a=this.parse(a));if(c=n(this,"defaults"))a=f.extend({},c,a);b&&b.collection&&(this.collection=b.collection);this.attributes={};this._escapedAttributes={};this.cid=f.uniqueId("c");this.changed={};this._silent=
+{};this._pending={};this.set(a,{silent:!0});this.changed={};this._silent={};this._pending={};this._previousAttributes=f.clone(this.attributes);this.initialize.apply(this,arguments)};f.extend(o.prototype,k,{changed:null,_silent:null,_pending:null,idAttribute:"id",initialize:function(){},toJSON:function(){return f.clone(this.attributes)},get:function(a){return this.attributes[a]},escape:function(a){var b;if(b=this._escapedAttributes[a])return b;b=this.get(a);return this._escapedAttributes[a]=f.escape(null==
+b?"":""+b)},has:function(a){return null!=this.get(a)},set:function(a,b,c){var d,e;f.isObject(a)||null==a?(d=a,c=b):(d={},d[a]=b);c||(c={});if(!d)return this;d instanceof o&&(d=d.attributes);if(c.unset)for(e in d)d[e]=void 0;if(!this._validate(d,c))return!1;this.idAttribute in d&&(this.id=d[this.idAttribute]);var b=c.changes={},h=this.attributes,g=this._escapedAttributes,j=this._previousAttributes||{};for(e in d){a=d[e];if(!f.isEqual(h[e],a)||c.unset&&f.has(h,e))delete g[e],(c.silent?this._silent:
+b)[e]=!0;c.unset?delete h[e]:h[e]=a;!f.isEqual(j[e],a)||f.has(h,e)!=f.has(j,e)?(this.changed[e]=a,c.silent||(this._pending[e]=!0)):(delete this.changed[e],delete this._pending[e])}c.silent||this.change(c);return this},unset:function(a,b){(b||(b={})).unset=!0;return this.set(a,null,b)},clear:function(a){(a||(a={})).unset=!0;return this.set(f.clone(this.attributes),a)},fetch:function(a){var a=a?f.clone(a):{},b=this,c=a.success;a.success=function(d,e,f){if(!b.set(b.parse(d,f),a))return!1;c&&c(b,d)};
+a.error=g.wrapError(a.error,b,a);return(this.sync||g.sync).call(this,"read",this,a)},save:function(a,b,c){var d,e;f.isObject(a)||null==a?(d=a,c=b):(d={},d[a]=b);c=c?f.clone(c):{};if(c.wait){if(!this._validate(d,c))return!1;e=f.clone(this.attributes)}a=f.extend({},c,{silent:!0});if(d&&!this.set(d,c.wait?a:c))return!1;var h=this,i=c.success;c.success=function(a,b,e){b=h.parse(a,e);if(c.wait){delete c.wait;b=f.extend(d||{},b)}if(!h.set(b,c))return false;i?i(h,a):h.trigger("sync",h,a,c)};c.error=g.wrapError(c.error,
+h,c);b=this.isNew()?"create":"update";b=(this.sync||g.sync).call(this,b,this,c);c.wait&&this.set(e,a);return b},destroy:function(a){var a=a?f.clone(a):{},b=this,c=a.success,d=function(){b.trigger("destroy",b,b.collection,a)};if(this.isNew())return d(),!1;a.success=function(e){a.wait&&d();c?c(b,e):b.trigger("sync",b,e,a)};a.error=g.wrapError(a.error,b,a);var e=(this.sync||g.sync).call(this,"delete",this,a);a.wait||d();return e},url:function(){var a=n(this,"urlRoot")||n(this.collection,"url")||t();
+return this.isNew()?a:a+("/"==a.charAt(a.length-1)?"":"/")+encodeURIComponent(this.id)},parse:function(a){return a},clone:function(){return new this.constructor(this.attributes)},isNew:function(){return null==this.id},change:function(a){a||(a={});var b=this._changing;this._changing=!0;for(var c in this._silent)this._pending[c]=!0;var d=f.extend({},a.changes,this._silent);this._silent={};for(c in d)this.trigger("change:"+c,this,this.get(c),a);if(b)return this;for(;!f.isEmpty(this._pending);){this._pending=
+{};this.trigger("change",this,a);for(c in this.changed)!this._pending[c]&&!this._silent[c]&&delete this.changed[c];this._previousAttributes=f.clone(this.attributes)}this._changing=!1;return this},hasChanged:function(a){return!arguments.length?!f.isEmpty(this.changed):f.has(this.changed,a)},changedAttributes:function(a){if(!a)return this.hasChanged()?f.clone(this.changed):!1;var b,c=!1,d=this._previousAttributes,e;for(e in a)if(!f.isEqual(d[e],b=a[e]))(c||(c={}))[e]=b;return c},previous:function(a){return!arguments.length||
+!this._previousAttributes?null:this._previousAttributes[a]},previousAttributes:function(){return f.clone(this._previousAttributes)},isValid:function(){return!this.validate(this.attributes)},_validate:function(a,b){if(b.silent||!this.validate)return!0;var a=f.extend({},this.attributes,a),c=this.validate(a,b);if(!c)return!0;b&&b.error?b.error(this,c,b):this.trigger("error",this,c,b);return!1}});var r=g.Collection=function(a,b){b||(b={});b.model&&(this.model=b.model);b.comparator&&(this.comparator=b.comparator);
+this._reset();this.initialize.apply(this,arguments);a&&this.reset(a,{silent:!0,parse:b.parse})};f.extend(r.prototype,k,{model:o,initialize:function(){},toJSON:function(a){return this.map(function(b){return b.toJSON(a)})},add:function(a,b){var c,d,e,g,i,j={},k={},l=[];b||(b={});a=f.isArray(a)?a.slice():[a];c=0;for(d=a.length;c<d;c++){if(!(e=a[c]=this._prepareModel(a[c],b)))throw Error("Can't add an invalid model to a collection");g=e.cid;i=e.id;j[g]||this._byCid[g]||null!=i&&(k[i]||this._byId[i])?
+l.push(c):j[g]=k[i]=e}for(c=l.length;c--;)a.splice(l[c],1);c=0;for(d=a.length;c<d;c++)(e=a[c]).on("all",this._onModelEvent,this),this._byCid[e.cid]=e,null!=e.id&&(this._byId[e.id]=e);this.length+=d;A.apply(this.models,[null!=b.at?b.at:this.models.length,0].concat(a));this.comparator&&this.sort({silent:!0});if(b.silent)return this;c=0;for(d=this.models.length;c<d;c++)if(j[(e=this.models[c]).cid])b.index=c,e.trigger("add",e,this,b);return this},remove:function(a,b){var c,d,e,g;b||(b={});a=f.isArray(a)?
+a.slice():[a];c=0;for(d=a.length;c<d;c++)if(g=this.getByCid(a[c])||this.get(a[c]))delete this._byId[g.id],delete this._byCid[g.cid],e=this.indexOf(g),this.models.splice(e,1),this.length--,b.silent||(b.index=e,g.trigger("remove",g,this,b)),this._removeReference(g);return this},push:function(a,b){a=this._prepareModel(a,b);this.add(a,b);return a},pop:function(a){var b=this.at(this.length-1);this.remove(b,a);return b},unshift:function(a,b){a=this._prepareModel(a,b);this.add(a,f.extend({at:0},b));return a},
+shift:function(a){var b=this.at(0);this.remove(b,a);return b},get:function(a){return null==a?void 0:this._byId[null!=a.id?a.id:a]},getByCid:function(a){return a&&this._byCid[a.cid||a]},at:function(a){return this.models[a]},where:function(a){return f.isEmpty(a)?[]:this.filter(function(b){for(var c in a)if(a[c]!==b.get(c))return!1;return!0})},sort:function(a){a||(a={});if(!this.comparator)throw Error("Cannot sort a set without a comparator");var b=f.bind(this.comparator,this);1==this.comparator.length?
+this.models=this.sortBy(b):this.models.sort(b);a.silent||this.trigger("reset",this,a);return this},pluck:function(a){return f.map(this.models,function(b){return b.get(a)})},reset:function(a,b){a||(a=[]);b||(b={});for(var c=0,d=this.models.length;c<d;c++)this._removeReference(this.models[c]);this._reset();this.add(a,f.extend({silent:!0},b));b.silent||this.trigger("reset",this,b);return this},fetch:function(a){a=a?f.clone(a):{};void 0===a.parse&&(a.parse=!0);var b=this,c=a.success;a.success=function(d,
+e,f){b[a.add?"add":"reset"](b.parse(d,f),a);c&&c(b,d)};a.error=g.wrapError(a.error,b,a);return(this.sync||g.sync).call(this,"read",this,a)},create:function(a,b){var c=this,b=b?f.clone(b):{},a=this._prepareModel(a,b);if(!a)return!1;b.wait||c.add(a,b);var d=b.success;b.success=function(e,f){b.wait&&c.add(e,b);d?d(e,f):e.trigger("sync",a,f,b)};a.save(null,b);return a},parse:function(a){return a},chain:function(){return f(this.models).chain()},_reset:function(){this.length=0;this.models=[];this._byId=
+{};this._byCid={}},_prepareModel:function(a,b){b||(b={});a instanceof o?a.collection||(a.collection=this):(b.collection=this,a=new this.model(a,b),a._validate(a.attributes,b)||(a=!1));return a},_removeReference:function(a){this==a.collection&&delete a.collection;a.off("all",this._onModelEvent,this)},_onModelEvent:function(a,b,c,d){("add"==a||"remove"==a)&&c!=this||("destroy"==a&&this.remove(b,d),b&&a==="change:"+b.idAttribute&&(delete this._byId[b.previous(b.idAttribute)],this._byId[b.id]=b),this.trigger.apply(this,
+arguments))}});f.each("forEach,each,map,reduce,reduceRight,find,detect,filter,select,reject,every,all,some,any,include,contains,invoke,max,min,sortBy,sortedIndex,toArray,size,first,initial,rest,last,without,indexOf,shuffle,lastIndexOf,isEmpty,groupBy".split(","),function(a){r.prototype[a]=function(){return f[a].apply(f,[this.models].concat(f.toArray(arguments)))}});var u=g.Router=function(a){a||(a={});a.routes&&(this.routes=a.routes);this._bindRoutes();this.initialize.apply(this,arguments)},B=/:\w+/g,
+C=/\*\w+/g,D=/[-[\]{}()+?.,\\^$|#\s]/g;f.extend(u.prototype,k,{initialize:function(){},route:function(a,b,c){g.history||(g.history=new m);f.isRegExp(a)||(a=this._routeToRegExp(a));c||(c=this[b]);g.history.route(a,f.bind(function(d){d=this._extractParameters(a,d);c&&c.apply(this,d);this.trigger.apply(this,["route:"+b].concat(d));g.history.trigger("route",this,b,d)},this));return this},navigate:function(a,b){g.history.navigate(a,b)},_bindRoutes:function(){if(this.routes){var a=[],b;for(b in this.routes)a.unshift([b,
+this.routes[b]]);b=0;for(var c=a.length;b<c;b++)this.route(a[b][0],a[b][1],this[a[b][1]])}},_routeToRegExp:function(a){a=a.replace(D,"\\$&").replace(B,"([^/]+)").replace(C,"(.*?)");return RegExp("^"+a+"$")},_extractParameters:function(a,b){return a.exec(b).slice(1)}});var m=g.History=function(){this.handlers=[];f.bindAll(this,"checkUrl")},s=/^[#\/]/,E=/msie [\w.]+/;m.started=!1;f.extend(m.prototype,k,{interval:50,getHash:function(a){return(a=(a?a.location:window.location).href.match(/#(.*)$/))?a[1]:
+""},getFragment:function(a,b){if(null==a)if(this._hasPushState||b){var a=window.location.pathname,c=window.location.search;c&&(a+=c)}else a=this.getHash();a.indexOf(this.options.root)||(a=a.substr(this.options.root.length));return a.replace(s,"")},start:function(a){if(m.started)throw Error("Backbone.history has already been started");m.started=!0;this.options=f.extend({},{root:"/"},this.options,a);this._wantsHashChange=!1!==this.options.hashChange;this._wantsPushState=!!this.options.pushState;this._hasPushState=
+!(!this.options.pushState||!window.history||!window.history.pushState);var a=this.getFragment(),b=document.documentMode;if(b=E.exec(navigator.userAgent.toLowerCase())&&(!b||7>=b))this.iframe=i('<iframe src="javascript:0" tabindex="-1" />').hide().appendTo("body")[0].contentWindow,this.navigate(a);this._hasPushState?i(window).bind("popstate",this.checkUrl):this._wantsHashChange&&"onhashchange"in window&&!b?i(window).bind("hashchange",this.checkUrl):this._wantsHashChange&&(this._checkUrlInterval=setInterval(this.checkUrl,
+this.interval));this.fragment=a;a=window.location;b=a.pathname==this.options.root;if(this._wantsHashChange&&this._wantsPushState&&!this._hasPushState&&!b)return this.fragment=this.getFragment(null,!0),window.location.replace(this.options.root+"#"+this.fragment),!0;this._wantsPushState&&this._hasPushState&&b&&a.hash&&(this.fragment=this.getHash().replace(s,""),window.history.replaceState({},document.title,a.protocol+"//"+a.host+this.options.root+this.fragment));if(!this.options.silent)return this.loadUrl()},
+stop:function(){i(window).unbind("popstate",this.checkUrl).unbind("hashchange",this.checkUrl);clearInterval(this._checkUrlInterval);m.started=!1},route:function(a,b){this.handlers.unshift({route:a,callback:b})},checkUrl:function(){var a=this.getFragment();a==this.fragment&&this.iframe&&(a=this.getFragment(this.getHash(this.iframe)));if(a==this.fragment)return!1;this.iframe&&this.navigate(a);this.loadUrl()||this.loadUrl(this.getHash())},loadUrl:function(a){var b=this.fragment=this.getFragment(a);return f.any(this.handlers,
+function(a){if(a.route.test(b))return a.callback(b),!0})},navigate:function(a,b){if(!m.started)return!1;if(!b||!0===b)b={trigger:b};var c=(a||"").replace(s,"");this.fragment!=c&&(this._hasPushState?(0!=c.indexOf(this.options.root)&&(c=this.options.root+c),this.fragment=c,window.history[b.replace?"replaceState":"pushState"]({},document.title,c)):this._wantsHashChange?(this.fragment=c,this._updateHash(window.location,c,b.replace),this.iframe&&c!=this.getFragment(this.getHash(this.iframe))&&(b.replace||
+this.iframe.document.open().close(),this._updateHash(this.iframe.location,c,b.replace))):window.location.assign(this.options.root+a),b.trigger&&this.loadUrl(a))},_updateHash:function(a,b,c){c?a.replace(a.toString().replace(/(javascript:|#).*$/,"")+"#"+b):a.hash=b}});var v=g.View=function(a){this.cid=f.uniqueId("view");this._configure(a||{});this._ensureElement();this.initialize.apply(this,arguments);this.delegateEvents()},F=/^(\S+)\s*(.*)$/,w="model,collection,el,id,attributes,className,tagName".split(",");
+f.extend(v.prototype,k,{tagName:"div",$:function(a){return this.$el.find(a)},initialize:function(){},render:function(){return this},remove:function(){this.$el.remove();return this},make:function(a,b,c){a=document.createElement(a);b&&i(a).attr(b);c&&i(a).html(c);return a},setElement:function(a,b){this.$el&&this.undelegateEvents();this.$el=a instanceof i?a:i(a);this.el=this.$el[0];!1!==b&&this.delegateEvents();return this},delegateEvents:function(a){if(a||(a=n(this,"events"))){this.undelegateEvents();
+for(var b in a){var c=a[b];f.isFunction(c)||(c=this[a[b]]);if(!c)throw Error('Method "'+a[b]+'" does not exist');var d=b.match(F),e=d[1],d=d[2],c=f.bind(c,this),e=e+(".delegateEvents"+this.cid);""===d?this.$el.bind(e,c):this.$el.delegate(d,e,c)}}},undelegateEvents:function(){this.$el.unbind(".delegateEvents"+this.cid)},_configure:function(a){this.options&&(a=f.extend({},this.options,a));for(var b=0,c=w.length;b<c;b++){var d=w[b];a[d]&&(this[d]=a[d])}this.options=a},_ensureElement:function(){if(this.el)this.setElement(this.el,
+!1);else{var a=n(this,"attributes")||{};this.id&&(a.id=this.id);this.className&&(a["class"]=this.className);this.setElement(this.make(this.tagName,a),!1)}}});o.extend=r.extend=u.extend=v.extend=function(a,b){var c=G(this,a,b);c.extend=this.extend;return c};var H={create:"POST",update:"PUT","delete":"DELETE",read:"GET"};g.sync=function(a,b,c){var d=H[a];c||(c={});var e={type:d,dataType:"json"};c.url||(e.url=n(b,"url")||t());if(!c.data&&b&&("create"==a||"update"==a))e.contentType="application/json",
+e.data=JSON.stringify(b.toJSON());g.emulateJSON&&(e.contentType="application/x-www-form-urlencoded",e.data=e.data?{model:e.data}:{});if(g.emulateHTTP&&("PUT"===d||"DELETE"===d))g.emulateJSON&&(e.data._method=d),e.type="POST",e.beforeSend=function(a){a.setRequestHeader("X-HTTP-Method-Override",d)};"GET"!==e.type&&!g.emulateJSON&&(e.processData=!1);return i.ajax(f.extend(e,c))};g.wrapError=function(a,b,c){return function(d,e){e=d===b?e:d;a?a(b,e,c):b.trigger("error",b,e,c)}};var x=function(){},G=function(a,
+b,c){var d;d=b&&b.hasOwnProperty("constructor")?b.constructor:function(){a.apply(this,arguments)};f.extend(d,a);x.prototype=a.prototype;d.prototype=new x;b&&f.extend(d.prototype,b);c&&f.extend(d,c);d.prototype.constructor=d;d.__super__=a.prototype;return d},n=function(a,b){return!a||!a[b]?null:f.isFunction(a[b])?a[b]():a[b]},t=function(){throw Error('A "url" property or function must be specified');}}).call(this);

docs/client-server/web/files/css

@@ -0,0 +1,16 @@
+/* latin */
+@font-face {
+  font-family: 'Droid Sans';
+  font-style: normal;
+  font-weight: 400;
+  src: local('Droid Sans'), local('DroidSans'), url(http://fonts.gstatic.com/s/droidsans/v5/s-BiyweUPV0v-yRb-cjciPk_vArhqVIZ0nv9q090hN8.woff2) format('woff2');
+  unicode-range: U+0000-00FF, U+0131, U+0152-0153, U+02C6, U+02DA, U+02DC, U+2000-206F, U+2074, U+20AC, U+2212, U+2215, U+E0FF, U+EFFD, U+F000;
+}
+/* latin */
+@font-face {
+  font-family: 'Droid Sans';
+  font-style: normal;
+  font-weight: 700;
+  src: local('Droid Sans Bold'), local('DroidSans-Bold'), url(http://fonts.gstatic.com/s/droidsans/v5/EFpQQyG9GqCrobXxL-KRMYWiMMZ7xLd792ULpGE4W_Y.woff2) format('woff2');
+  unicode-range: U+0000-00FF, U+0131, U+0152-0153, U+02C6, U+02DA, U+02DC, U+2000-206F, U+2074, U+20AC, U+2212, U+2215, U+E0FF, U+EFFD, U+F000;
+}

docs/client-server/web/files/jquery.ba-bbq.min.js

@@ -0,0 +1,18 @@
+/*
+ * jQuery BBQ: Back Button & Query Library - v1.2.1 - 2/17/2010
+ * http://benalman.com/projects/jquery-bbq-plugin/
+ * 
+ * Copyright (c) 2010 "Cowboy" Ben Alman
+ * Dual licensed under the MIT and GPL licenses.
+ * http://benalman.com/about/license/
+ */
+(function($,p){var i,m=Array.prototype.slice,r=decodeURIComponent,a=$.param,c,l,v,b=$.bbq=$.bbq||{},q,u,j,e=$.event.special,d="hashchange",A="querystring",D="fragment",y="elemUrlAttr",g="location",k="href",t="src",x=/^.*\?|#.*$/g,w=/^.*\#/,h,C={};function E(F){return typeof F==="string"}function B(G){var F=m.call(arguments,1);return function(){return G.apply(this,F.concat(m.call(arguments)))}}function n(F){return F.replace(/^[^#]*#?(.*)$/,"$1")}function o(F){return F.replace(/(?:^[^?#]*\?([^#]*).*$)?.*/,"$1")}function f(H,M,F,I,G){var O,L,K,N,J;if(I!==i){K=F.match(H?/^([^#]*)\#?(.*)$/:/^([^#?]*)\??([^#]*)(#?.*)/);J=K[3]||"";if(G===2&&E(I)){L=I.replace(H?w:x,"")}else{N=l(K[2]);I=E(I)?l[H?D:A](I):I;L=G===2?I:G===1?$.extend({},I,N):$.extend({},N,I);L=a(L);if(H){L=L.replace(h,r)}}O=K[1]+(H?"#":L||!K[1]?"?":"")+L+J}else{O=M(F!==i?F:p[g][k])}return O}a[A]=B(f,0,o);a[D]=c=B(f,1,n);c.noEscape=function(G){G=G||"";var F=$.map(G.split(""),encodeURIComponent);h=new RegExp(F.join("|"),"g")};c.noEscape(",/");$.deparam=l=function(I,F){var H={},G={"true":!0,"false":!1,"null":null};$.each(I.replace(/\+/g," ").split("&"),function(L,Q){var K=Q.split("="),P=r(K[0]),J,O=H,M=0,R=P.split("]["),N=R.length-1;if(/\[/.test(R[0])&&/\]$/.test(R[N])){R[N]=R[N].replace(/\]$/,"");R=R.shift().split("[").concat(R);N=R.length-1}else{N=0}if(K.length===2){J=r(K[1]);if(F){J=J&&!isNaN(J)?+J:J==="undefined"?i:G[J]!==i?G[J]:J}if(N){for(;M<=N;M++){P=R[M]===""?O.length:R[M];O=O[P]=M<N?O[P]||(R[M+1]&&isNaN(R[M+1])?{}:[]):J}}else{if($.isArray(H[P])){H[P].push(J)}else{if(H[P]!==i){H[P]=[H[P],J]}else{H[P]=J}}}}else{if(P){H[P]=F?i:""}}});return H};function z(H,F,G){if(F===i||typeof F==="boolean"){G=F;F=a[H?D:A]()}else{F=E(F)?F.replace(H?w:x,""):F}return l(F,G)}l[A]=B(z,0);l[D]=v=B(z,1);$[y]||($[y]=function(F){return $.extend(C,F)})({a:k,base:k,iframe:t,img:t,input:t,form:"action",link:k,script:t});j=$[y];function s(I,G,H,F){if(!E(H)&&typeof H!=="object"){F=H;H=G;G=i}return this.each(function(){var L=$(this),J=G||j()[(this.nodeName||"").toLowerCase()]||"",K=J&&L.attr(J)||"";L.attr(J,a[I](K,H,F))})}$.fn[A]=B(s,A);$.fn[D]=B(s,D);b.pushState=q=function(I,F){if(E(I)&&/^#/.test(I)&&F===i){F=2}var H=I!==i,G=c(p[g][k],H?I:{},H?F:2);p[g][k]=G+(/#/.test(G)?"":"#")};b.getState=u=function(F,G){return F===i||typeof F==="boolean"?v(F):v(G)[F]};b.removeState=function(F){var G={};if(F!==i){G=u();$.each($.isArray(F)?F:arguments,function(I,H){delete G[H]})}q(G,2)};e[d]=$.extend(e[d],{add:function(F){var H;function G(J){var I=J[D]=c();J.getState=function(K,L){return K===i||typeof K==="boolean"?l(I,K):l(I,L)[K]};H.apply(this,arguments)}if($.isFunction(F)){H=F;return G}else{H=F.handler;F.handler=G}}})})(jQuery,this);
+/*
+ * jQuery hashchange event - v1.2 - 2/11/2010
+ * http://benalman.com/projects/jquery-hashchange-plugin/
+ * 
+ * Copyright (c) 2010 "Cowboy" Ben Alman
+ * Dual licensed under the MIT and GPL licenses.
+ * http://benalman.com/about/license/
+ */
+(function($,i,b){var j,k=$.event.special,c="location",d="hashchange",l="href",f=$.browser,g=document.documentMode,h=f.msie&&(g===b||g<8),e="on"+d in i&&!h;function a(m){m=m||i[c][l];return m.replace(/^[^#]*#?(.*)$/,"$1")}$[d+"Delay"]=100;k[d]=$.extend(k[d],{setup:function(){if(e){return false}$(j.start)},teardown:function(){if(e){return false}$(j.stop)}});j=(function(){var m={},r,n,o,q;function p(){o=q=function(s){return s};if(h){n=$('<iframe src="javascript:0"/>').hide().insertAfter("body")[0].contentWindow;q=function(){return a(n.document[c][l])};o=function(u,s){if(u!==s){var t=n.document;t.open().close();t[c].hash="#"+u}};o(a())}}m.start=function(){if(r){return}var t=a();o||p();(function s(){var v=a(),u=q(t);if(v!==t){o(t=v,u);$(i).trigger(d)}else{if(u!==t){i[c][l]=i[c][l].replace(/#.*/,"")+"#"+u}}r=setTimeout(s,$[d+"Delay"])})()};m.stop=function(){if(!n){r&&clearTimeout(r);r=0}};return m})()})(jQuery,this);

docs/client-server/web/files/jquery.slideto.min.js

@@ -0,0 +1 @@
+(function(b){b.fn.slideto=function(a){a=b.extend({slide_duration:"slow",highlight_duration:3E3,highlight:true,highlight_color:"#FFFF99"},a);return this.each(function(){obj=b(this);b("body").animate({scrollTop:obj.offset().top},a.slide_duration,function(){a.highlight&&b.ui.version&&obj.effect("highlight",{color:a.highlight_color},a.highlight_duration)})})}})(jQuery);

docs/client-server/web/files/jquery.wiggle.min.js

@@ -0,0 +1,8 @@
+/*
+jQuery Wiggle
+Author: WonderGroup, Jordan Thomas
+URL: http://labs.wondergroup.com/demos/mini-ui/index.html
+License: MIT (http://en.wikipedia.org/wiki/MIT_License)
+*/
+jQuery.fn.wiggle=function(o){var d={speed:50,wiggles:3,travel:5,callback:null};var o=jQuery.extend(d,o);return this.each(function(){var cache=this;var wrap=jQuery(this).wrap('<div class="wiggle-wrap"></div>').css("position","relative");var calls=0;for(i=1;i<=o.wiggles;i++){jQuery(this).animate({left:"-="+o.travel},o.speed).animate({left:"+="+o.travel*2},o.speed*2).animate({left:"-="+o.travel},o.speed,function(){calls++;if(jQuery(cache).parent().hasClass('wiggle-wrap')){jQuery(cache).parent().replaceWith(cache);}
+if(calls==o.wiggles&&jQuery.isFunction(o.callback)){o.callback();}});}});};

docs/client-server/web/files/reset.css

@@ -0,0 +1,125 @@
+/* http://meyerweb.com/eric/tools/css/reset/ v2.0 | 20110126 */
+html,
+body,
+div,
+span,
+applet,
+object,
+iframe,
+h1,
+h2,
+h3,
+h4,
+h5,
+h6,
+p,
+blockquote,
+pre,
+a,
+abbr,
+acronym,
+address,
+big,
+cite,
+code,
+del,
+dfn,
+em,
+img,
+ins,
+kbd,
+q,
+s,
+samp,
+small,
+strike,
+strong,
+sub,
+sup,
+tt,
+var,
+b,
+u,
+i,
+center,
+dl,
+dt,
+dd,
+ol,
+ul,
+li,
+fieldset,
+form,
+label,
+legend,
+table,
+caption,
+tbody,
+tfoot,
+thead,
+tr,
+th,
+td,
+article,
+aside,
+canvas,
+details,
+embed,
+figure,
+figcaption,
+footer,
+header,
+hgroup,
+menu,
+nav,
+output,
+ruby,
+section,
+summary,
+time,
+mark,
+audio,
+video {
+  margin: 0;
+  padding: 0;
+  border: 0;
+  font-size: 100%;
+  font: inherit;
+  vertical-align: baseline;
+}
+/* HTML5 display-role reset for older browsers */
+article,
+aside,
+details,
+figcaption,
+figure,
+footer,
+header,
+hgroup,
+menu,
+nav,
+section {
+  display: block;
+}
+body {
+  line-height: 1;
+}
+ol,
+ul {
+  list-style: none;
+}
+blockquote,
+q {
+  quotes: none;
+}
+blockquote:before,
+blockquote:after,
+q:before,
+q:after {
+  content: '';
+  content: none;
+}
+table {
+  border-collapse: collapse;
+  border-spacing: 0;
+}

docs/client-server/web/files/swagger-oauth.js

@@ -0,0 +1,211 @@
+var appName;
+var popupMask;
+var popupDialog;
+var clientId;
+var realm;
+
+function handleLogin() {
+  var scopes = [];
+
+  if(window.swaggerUi.api.authSchemes 
+    && window.swaggerUi.api.authSchemes.oauth2
+    && window.swaggerUi.api.authSchemes.oauth2.scopes) {
+    scopes = window.swaggerUi.api.authSchemes.oauth2.scopes;
+  }
+
+  if(window.swaggerUi.api
+    && window.swaggerUi.api.info) {
+    appName = window.swaggerUi.api.info.title;
+  }
+
+  if(popupDialog.length > 0)
+    popupDialog = popupDialog.last();
+  else {
+    popupDialog = $(
+      [
+        '<div class="api-popup-dialog">',
+        '<div class="api-popup-title">Select OAuth2.0 Scopes</div>',
+        '<div class="api-popup-content">',
+          '<p>Scopes are used to grant an application different levels of access to data on behalf of the end user. Each API may declare one or more scopes.',
+            '<a href="#">Learn how to use</a>',
+          '</p>',
+          '<p><strong>' + appName + '</strong> API requires the following scopes. Select which ones you want to grant to Swagger UI.</p>',
+          '<ul class="api-popup-scopes">',
+          '</ul>',
+          '<p class="error-msg"></p>',
+          '<div class="api-popup-actions"><button class="api-popup-authbtn api-button green" type="button">Authorize</button><button class="api-popup-cancel api-button gray" type="button">Cancel</button></div>',
+        '</div>',
+        '</div>'].join(''));
+    $(document.body).append(popupDialog);
+
+    popup = popupDialog.find('ul.api-popup-scopes').empty();
+    for (i = 0; i < scopes.length; i ++) {
+      scope = scopes[i];
+      str = '<li><input type="checkbox" id="scope_' + i + '" scope="' + scope.scope + '"/>' + '<label for="scope_' + i + '">' + scope.scope;
+      if (scope.description) {
+        str += '<br/><span class="api-scope-desc">' + scope.description + '</span>';
+      }
+      str += '</label></li>';
+      popup.append(str);
+    }
+  }
+
+  var $win = $(window),
+    dw = $win.width(),
+    dh = $win.height(),
+    st = $win.scrollTop(),
+    dlgWd = popupDialog.outerWidth(),
+    dlgHt = popupDialog.outerHeight(),
+    top = (dh -dlgHt)/2 + st,
+    left = (dw - dlgWd)/2;
+
+  popupDialog.css({
+    top: (top < 0? 0 : top) + 'px',
+    left: (left < 0? 0 : left) + 'px'
+  });
+
+  popupDialog.find('button.api-popup-cancel').click(function() {
+    popupMask.hide();
+    popupDialog.hide();
+  });
+  popupDialog.find('button.api-popup-authbtn').click(function() {
+    popupMask.hide();
+    popupDialog.hide();
+
+    var authSchemes = window.swaggerUi.api.authSchemes;
+    var host = window.location;
+    var redirectUrl = host.protocol + '//' + host.host + "/o2c.html";
+    var url = null;
+
+    var p = window.swaggerUi.api.authSchemes;
+    for (var key in p) {
+      if (p.hasOwnProperty(key)) {
+        var o = p[key].grantTypes;
+        for(var t in o) {
+          if(o.hasOwnProperty(t) && t === 'implicit') {
+            var dets = o[t];
+            url = dets.loginEndpoint.url + "?response_type=token";
+            window.swaggerUi.tokenName = dets.tokenName;
+          }
+        }
+      }
+    }
+    var scopes = []
+    var o = $('.api-popup-scopes').find('input:checked');
+
+    for(k =0; k < o.length; k++) {
+      scopes.push($(o[k]).attr("scope"));
+    }
+
+    window.enabledScopes=scopes;
+
+    url += '&redirect_uri=' + encodeURIComponent(redirectUrl);
+    url += '&realm=' + encodeURIComponent(realm);
+    url += '&client_id=' + encodeURIComponent(clientId);
+    url += '&scope=' + encodeURIComponent(scopes);
+
+    window.open(url);
+  });
+
+  popupMask.show();
+  popupDialog.show();
+  return;
+}
+
+
+function handleLogout() {
+  for(key in window.authorizations.authz){
+    window.authorizations.remove(key)
+  }
+  window.enabledScopes = null;
+  $('.api-ic.ic-on').addClass('ic-off');
+  $('.api-ic.ic-on').removeClass('ic-on');
+
+  // set the info box
+  $('.api-ic.ic-warning').addClass('ic-error');
+  $('.api-ic.ic-warning').removeClass('ic-warning');
+}
+
+function initOAuth(opts) {
+  var o = (opts||{});
+  var errors = [];
+
+  appName = (o.appName||errors.push("missing appName"));
+  popupMask = (o.popupMask||$('#api-common-mask'));
+  popupDialog = (o.popupDialog||$('.api-popup-dialog'));
+  clientId = (o.clientId||errors.push("missing client id"));
+  realm = (o.realm||errors.push("missing realm"));
+
+  if(errors.length > 0){
+    log("auth unable initialize oauth: " + errors);
+    return;
+  }
+
+  $('pre code').each(function(i, e) {hljs.highlightBlock(e)});
+  $('.api-ic').click(function(s) {
+    if($(s.target).hasClass('ic-off'))
+      handleLogin();
+    else {
+      handleLogout();
+    }
+    false;
+  });
+}
+
+function onOAuthComplete(token) {
+  if(token) {
+    if(token.error) {
+      var checkbox = $('input[type=checkbox],.secured')
+      checkbox.each(function(pos){
+        checkbox[pos].checked = false;
+      });
+      alert(token.error);
+    }
+    else {
+      var b = token[window.swaggerUi.tokenName];
+      if(b){
+        // if all roles are satisfied
+        var o = null;
+        $.each($('.auth #api_information_panel'), function(k, v) {
+          var children = v;
+          if(children && children.childNodes) {
+            var requiredScopes = [];
+            $.each((children.childNodes), function (k1, v1){
+              var inner = v1.innerHTML;
+              if(inner)
+                requiredScopes.push(inner);
+            });
+            var diff = [];
+            for(var i=0; i < requiredScopes.length; i++) {
+              var s = requiredScopes[i];
+              if(window.enabledScopes && window.enabledScopes.indexOf(s) == -1) {
+                diff.push(s);
+              }
+            }
+            if(diff.length > 0){
+              o = v.parentNode;
+              $(o.parentNode).find('.api-ic.ic-on').addClass('ic-off');
+              $(o.parentNode).find('.api-ic.ic-on').removeClass('ic-on');
+
+              // sorry, not all scopes are satisfied
+              $(o).find('.api-ic').addClass('ic-warning');
+              $(o).find('.api-ic').removeClass('ic-error');
+            }
+            else {
+              o = v.parentNode;
+              $(o.parentNode).find('.api-ic.ic-off').addClass('ic-on');
+              $(o.parentNode).find('.api-ic.ic-off').removeClass('ic-off');
+
+              // all scopes are satisfied
+              $(o).find('.api-ic').addClass('ic-info');
+              $(o).find('.api-ic').removeClass('ic-warning');
+              $(o).find('.api-ic').removeClass('ic-error');          
+            }
+          }
+        });
+
+        window.authorizations.add("oauth2", new ApiKeyAuthorization("Authorization", "Bearer " + b, "header"));
+      }
+    }
+  }
+}

docs/client-server/web/files/underscore-min.js

@@ -0,0 +1,32 @@
+// Underscore.js 1.3.3
+// (c) 2009-2012 Jeremy Ashkenas, DocumentCloud Inc.
+// Underscore is freely distributable under the MIT license.
+// Portions of Underscore are inspired or borrowed from Prototype,
+// Oliver Steele's Functional, and John Resig's Micro-Templating.
+// For all details and documentation:
+// http://documentcloud.github.com/underscore
+(function(){function r(a,c,d){if(a===c)return 0!==a||1/a==1/c;if(null==a||null==c)return a===c;a._chain&&(a=a._wrapped);c._chain&&(c=c._wrapped);if(a.isEqual&&b.isFunction(a.isEqual))return a.isEqual(c);if(c.isEqual&&b.isFunction(c.isEqual))return c.isEqual(a);var e=l.call(a);if(e!=l.call(c))return!1;switch(e){case "[object String]":return a==""+c;case "[object Number]":return a!=+a?c!=+c:0==a?1/a==1/c:a==+c;case "[object Date]":case "[object Boolean]":return+a==+c;case "[object RegExp]":return a.source==
+c.source&&a.global==c.global&&a.multiline==c.multiline&&a.ignoreCase==c.ignoreCase}if("object"!=typeof a||"object"!=typeof c)return!1;for(var f=d.length;f--;)if(d[f]==a)return!0;d.push(a);var f=0,g=!0;if("[object Array]"==e){if(f=a.length,g=f==c.length)for(;f--&&(g=f in a==f in c&&r(a[f],c[f],d)););}else{if("constructor"in a!="constructor"in c||a.constructor!=c.constructor)return!1;for(var h in a)if(b.has(a,h)&&(f++,!(g=b.has(c,h)&&r(a[h],c[h],d))))break;if(g){for(h in c)if(b.has(c,h)&&!f--)break;
+g=!f}}d.pop();return g}var s=this,I=s._,o={},k=Array.prototype,p=Object.prototype,i=k.slice,J=k.unshift,l=p.toString,K=p.hasOwnProperty,y=k.forEach,z=k.map,A=k.reduce,B=k.reduceRight,C=k.filter,D=k.every,E=k.some,q=k.indexOf,F=k.lastIndexOf,p=Array.isArray,L=Object.keys,t=Function.prototype.bind,b=function(a){return new m(a)};"undefined"!==typeof exports?("undefined"!==typeof module&&module.exports&&(exports=module.exports=b),exports._=b):s._=b;b.VERSION="1.3.3";var j=b.each=b.forEach=function(a,
+c,d){if(a!=null)if(y&&a.forEach===y)a.forEach(c,d);else if(a.length===+a.length)for(var e=0,f=a.length;e<f;e++){if(e in a&&c.call(d,a[e],e,a)===o)break}else for(e in a)if(b.has(a,e)&&c.call(d,a[e],e,a)===o)break};b.map=b.collect=function(a,c,b){var e=[];if(a==null)return e;if(z&&a.map===z)return a.map(c,b);j(a,function(a,g,h){e[e.length]=c.call(b,a,g,h)});if(a.length===+a.length)e.length=a.length;return e};b.reduce=b.foldl=b.inject=function(a,c,d,e){var f=arguments.length>2;a==null&&(a=[]);if(A&&
+a.reduce===A){e&&(c=b.bind(c,e));return f?a.reduce(c,d):a.reduce(c)}j(a,function(a,b,i){if(f)d=c.call(e,d,a,b,i);else{d=a;f=true}});if(!f)throw new TypeError("Reduce of empty array with no initial value");return d};b.reduceRight=b.foldr=function(a,c,d,e){var f=arguments.length>2;a==null&&(a=[]);if(B&&a.reduceRight===B){e&&(c=b.bind(c,e));return f?a.reduceRight(c,d):a.reduceRight(c)}var g=b.toArray(a).reverse();e&&!f&&(c=b.bind(c,e));return f?b.reduce(g,c,d,e):b.reduce(g,c)};b.find=b.detect=function(a,
+c,b){var e;G(a,function(a,g,h){if(c.call(b,a,g,h)){e=a;return true}});return e};b.filter=b.select=function(a,c,b){var e=[];if(a==null)return e;if(C&&a.filter===C)return a.filter(c,b);j(a,function(a,g,h){c.call(b,a,g,h)&&(e[e.length]=a)});return e};b.reject=function(a,c,b){var e=[];if(a==null)return e;j(a,function(a,g,h){c.call(b,a,g,h)||(e[e.length]=a)});return e};b.every=b.all=function(a,c,b){var e=true;if(a==null)return e;if(D&&a.every===D)return a.every(c,b);j(a,function(a,g,h){if(!(e=e&&c.call(b,
+a,g,h)))return o});return!!e};var G=b.some=b.any=function(a,c,d){c||(c=b.identity);var e=false;if(a==null)return e;if(E&&a.some===E)return a.some(c,d);j(a,function(a,b,h){if(e||(e=c.call(d,a,b,h)))return o});return!!e};b.include=b.contains=function(a,c){var b=false;if(a==null)return b;if(q&&a.indexOf===q)return a.indexOf(c)!=-1;return b=G(a,function(a){return a===c})};b.invoke=function(a,c){var d=i.call(arguments,2);return b.map(a,function(a){return(b.isFunction(c)?c||a:a[c]).apply(a,d)})};b.pluck=
+function(a,c){return b.map(a,function(a){return a[c]})};b.max=function(a,c,d){if(!c&&b.isArray(a)&&a[0]===+a[0])return Math.max.apply(Math,a);if(!c&&b.isEmpty(a))return-Infinity;var e={computed:-Infinity};j(a,function(a,b,h){b=c?c.call(d,a,b,h):a;b>=e.computed&&(e={value:a,computed:b})});return e.value};b.min=function(a,c,d){if(!c&&b.isArray(a)&&a[0]===+a[0])return Math.min.apply(Math,a);if(!c&&b.isEmpty(a))return Infinity;var e={computed:Infinity};j(a,function(a,b,h){b=c?c.call(d,a,b,h):a;b<e.computed&&
+(e={value:a,computed:b})});return e.value};b.shuffle=function(a){var b=[],d;j(a,function(a,f){d=Math.floor(Math.random()*(f+1));b[f]=b[d];b[d]=a});return b};b.sortBy=function(a,c,d){var e=b.isFunction(c)?c:function(a){return a[c]};return b.pluck(b.map(a,function(a,b,c){return{value:a,criteria:e.call(d,a,b,c)}}).sort(function(a,b){var c=a.criteria,d=b.criteria;return c===void 0?1:d===void 0?-1:c<d?-1:c>d?1:0}),"value")};b.groupBy=function(a,c){var d={},e=b.isFunction(c)?c:function(a){return a[c]};
+j(a,function(a,b){var c=e(a,b);(d[c]||(d[c]=[])).push(a)});return d};b.sortedIndex=function(a,c,d){d||(d=b.identity);for(var e=0,f=a.length;e<f;){var g=e+f>>1;d(a[g])<d(c)?e=g+1:f=g}return e};b.toArray=function(a){return!a?[]:b.isArray(a)||b.isArguments(a)?i.call(a):a.toArray&&b.isFunction(a.toArray)?a.toArray():b.values(a)};b.size=function(a){return b.isArray(a)?a.length:b.keys(a).length};b.first=b.head=b.take=function(a,b,d){return b!=null&&!d?i.call(a,0,b):a[0]};b.initial=function(a,b,d){return i.call(a,
+0,a.length-(b==null||d?1:b))};b.last=function(a,b,d){return b!=null&&!d?i.call(a,Math.max(a.length-b,0)):a[a.length-1]};b.rest=b.tail=function(a,b,d){return i.call(a,b==null||d?1:b)};b.compact=function(a){return b.filter(a,function(a){return!!a})};b.flatten=function(a,c){return b.reduce(a,function(a,e){if(b.isArray(e))return a.concat(c?e:b.flatten(e));a[a.length]=e;return a},[])};b.without=function(a){return b.difference(a,i.call(arguments,1))};b.uniq=b.unique=function(a,c,d){var d=d?b.map(a,d):a,
+e=[];a.length<3&&(c=true);b.reduce(d,function(d,g,h){if(c?b.last(d)!==g||!d.length:!b.include(d,g)){d.push(g);e.push(a[h])}return d},[]);return e};b.union=function(){return b.uniq(b.flatten(arguments,true))};b.intersection=b.intersect=function(a){var c=i.call(arguments,1);return b.filter(b.uniq(a),function(a){return b.every(c,function(c){return b.indexOf(c,a)>=0})})};b.difference=function(a){var c=b.flatten(i.call(arguments,1),true);return b.filter(a,function(a){return!b.include(c,a)})};b.zip=function(){for(var a=
+i.call(arguments),c=b.max(b.pluck(a,"length")),d=Array(c),e=0;e<c;e++)d[e]=b.pluck(a,""+e);return d};b.indexOf=function(a,c,d){if(a==null)return-1;var e;if(d){d=b.sortedIndex(a,c);return a[d]===c?d:-1}if(q&&a.indexOf===q)return a.indexOf(c);d=0;for(e=a.length;d<e;d++)if(d in a&&a[d]===c)return d;return-1};b.lastIndexOf=function(a,b){if(a==null)return-1;if(F&&a.lastIndexOf===F)return a.lastIndexOf(b);for(var d=a.length;d--;)if(d in a&&a[d]===b)return d;return-1};b.range=function(a,b,d){if(arguments.length<=
+1){b=a||0;a=0}for(var d=arguments[2]||1,e=Math.max(Math.ceil((b-a)/d),0),f=0,g=Array(e);f<e;){g[f++]=a;a=a+d}return g};var H=function(){};b.bind=function(a,c){var d,e;if(a.bind===t&&t)return t.apply(a,i.call(arguments,1));if(!b.isFunction(a))throw new TypeError;e=i.call(arguments,2);return d=function(){if(!(this instanceof d))return a.apply(c,e.concat(i.call(arguments)));H.prototype=a.prototype;var b=new H,g=a.apply(b,e.concat(i.call(arguments)));return Object(g)===g?g:b}};b.bindAll=function(a){var c=
+i.call(arguments,1);c.length==0&&(c=b.functions(a));j(c,function(c){a[c]=b.bind(a[c],a)});return a};b.memoize=function(a,c){var d={};c||(c=b.identity);return function(){var e=c.apply(this,arguments);return b.has(d,e)?d[e]:d[e]=a.apply(this,arguments)}};b.delay=function(a,b){var d=i.call(arguments,2);return setTimeout(function(){return a.apply(null,d)},b)};b.defer=function(a){return b.delay.apply(b,[a,1].concat(i.call(arguments,1)))};b.throttle=function(a,c){var d,e,f,g,h,i,j=b.debounce(function(){h=
+g=false},c);return function(){d=this;e=arguments;f||(f=setTimeout(function(){f=null;h&&a.apply(d,e);j()},c));g?h=true:i=a.apply(d,e);j();g=true;return i}};b.debounce=function(a,b,d){var e;return function(){var f=this,g=arguments;d&&!e&&a.apply(f,g);clearTimeout(e);e=setTimeout(function(){e=null;d||a.apply(f,g)},b)}};b.once=function(a){var b=false,d;return function(){if(b)return d;b=true;return d=a.apply(this,arguments)}};b.wrap=function(a,b){return function(){var d=[a].concat(i.call(arguments,0));
+return b.apply(this,d)}};b.compose=function(){var a=arguments;return function(){for(var b=arguments,d=a.length-1;d>=0;d--)b=[a[d].apply(this,b)];return b[0]}};b.after=function(a,b){return a<=0?b():function(){if(--a<1)return b.apply(this,arguments)}};b.keys=L||function(a){if(a!==Object(a))throw new TypeError("Invalid object");var c=[],d;for(d in a)b.has(a,d)&&(c[c.length]=d);return c};b.values=function(a){return b.map(a,b.identity)};b.functions=b.methods=function(a){var c=[],d;for(d in a)b.isFunction(a[d])&&
+c.push(d);return c.sort()};b.extend=function(a){j(i.call(arguments,1),function(b){for(var d in b)a[d]=b[d]});return a};b.pick=function(a){var c={};j(b.flatten(i.call(arguments,1)),function(b){b in a&&(c[b]=a[b])});return c};b.defaults=function(a){j(i.call(arguments,1),function(b){for(var d in b)a[d]==null&&(a[d]=b[d])});return a};b.clone=function(a){return!b.isObject(a)?a:b.isArray(a)?a.slice():b.extend({},a)};b.tap=function(a,b){b(a);return a};b.isEqual=function(a,b){return r(a,b,[])};b.isEmpty=
+function(a){if(a==null)return true;if(b.isArray(a)||b.isString(a))return a.length===0;for(var c in a)if(b.has(a,c))return false;return true};b.isElement=function(a){return!!(a&&a.nodeType==1)};b.isArray=p||function(a){return l.call(a)=="[object Array]"};b.isObject=function(a){return a===Object(a)};b.isArguments=function(a){return l.call(a)=="[object Arguments]"};b.isArguments(arguments)||(b.isArguments=function(a){return!(!a||!b.has(a,"callee"))});b.isFunction=function(a){return l.call(a)=="[object Function]"};
+b.isString=function(a){return l.call(a)=="[object String]"};b.isNumber=function(a){return l.call(a)=="[object Number]"};b.isFinite=function(a){return b.isNumber(a)&&isFinite(a)};b.isNaN=function(a){return a!==a};b.isBoolean=function(a){return a===true||a===false||l.call(a)=="[object Boolean]"};b.isDate=function(a){return l.call(a)=="[object Date]"};b.isRegExp=function(a){return l.call(a)=="[object RegExp]"};b.isNull=function(a){return a===null};b.isUndefined=function(a){return a===void 0};b.has=function(a,
+b){return K.call(a,b)};b.noConflict=function(){s._=I;return this};b.identity=function(a){return a};b.times=function(a,b,d){for(var e=0;e<a;e++)b.call(d,e)};b.escape=function(a){return(""+a).replace(/&/g,"&amp;").replace(/</g,"&lt;").replace(/>/g,"&gt;").replace(/"/g,"&quot;").replace(/'/g,"&#x27;").replace(/\//g,"&#x2F;")};b.result=function(a,c){if(a==null)return null;var d=a[c];return b.isFunction(d)?d.call(a):d};b.mixin=function(a){j(b.functions(a),function(c){M(c,b[c]=a[c])})};var N=0;b.uniqueId=
+function(a){var b=N++;return a?a+b:b};b.templateSettings={evaluate:/<%([\s\S]+?)%>/g,interpolate:/<%=([\s\S]+?)%>/g,escape:/<%-([\s\S]+?)%>/g};var u=/.^/,n={"\\":"\\","'":"'",r:"\r",n:"\n",t:"\t",u2028:"\u2028",u2029:"\u2029"},v;for(v in n)n[n[v]]=v;var O=/\\|'|\r|\n|\t|\u2028|\u2029/g,P=/\\(\\|'|r|n|t|u2028|u2029)/g,w=function(a){return a.replace(P,function(a,b){return n[b]})};b.template=function(a,c,d){d=b.defaults(d||{},b.templateSettings);a="__p+='"+a.replace(O,function(a){return"\\"+n[a]}).replace(d.escape||
+u,function(a,b){return"'+\n_.escape("+w(b)+")+\n'"}).replace(d.interpolate||u,function(a,b){return"'+\n("+w(b)+")+\n'"}).replace(d.evaluate||u,function(a,b){return"';\n"+w(b)+"\n;__p+='"})+"';\n";d.variable||(a="with(obj||{}){\n"+a+"}\n");var a="var __p='';var print=function(){__p+=Array.prototype.join.call(arguments, '')};\n"+a+"return __p;\n",e=new Function(d.variable||"obj","_",a);if(c)return e(c,b);c=function(a){return e.call(this,a,b)};c.source="function("+(d.variable||"obj")+"){\n"+a+"}";return c};
+b.chain=function(a){return b(a).chain()};var m=function(a){this._wrapped=a};b.prototype=m.prototype;var x=function(a,c){return c?b(a).chain():a},M=function(a,c){m.prototype[a]=function(){var a=i.call(arguments);J.call(a,this._wrapped);return x(c.apply(b,a),this._chain)}};b.mixin(b);j("pop,push,reverse,shift,sort,splice,unshift".split(","),function(a){var b=k[a];m.prototype[a]=function(){var d=this._wrapped;b.apply(d,arguments);var e=d.length;(a=="shift"||a=="splice")&&e===0&&delete d[0];return x(d,
+this._chain)}});j(["concat","join","slice"],function(a){var b=k[a];m.prototype[a]=function(){return x(b.apply(this._wrapped,arguments),this._chain)}});m.prototype.chain=function(){this._chain=true;return this};m.prototype.value=function(){return this._wrapped}}).call(this);

docs/client-server/web/swagger.html

@@ -0,0 +1,78 @@
+<!DOCTYPE html>
+<html><head><meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
+  <title>Matrix Client-Server API Documentation</title>
+  <link href="./files/css" rel="stylesheet" type="text/css">
+  <link href="./files/reset.css" media="screen" rel="stylesheet" type="text/css">
+  <link href="./files/screen.css" media="screen" rel="stylesheet" type="text/css">
+  <link href="./files/reset.css" media="print" rel="stylesheet" type="text/css">
+  <link href="./files/screen.css" media="print" rel="stylesheet" type="text/css">
+  <script type="text/javascript" src="./files/shred.bundle.js"></script>
+  <script src="./files/jquery-1.8.0.min.js" type="text/javascript"></script>
+  <script src="./files/jquery.slideto.min.js" type="text/javascript"></script>
+  <script src="./files/jquery.wiggle.min.js" type="text/javascript"></script>
+  <script src="./files/jquery.ba-bbq.min.js" type="text/javascript"></script>
+  <script src="./files/handlebars-1.0.0.js" type="text/javascript"></script>
+  <script src="./files/underscore-min.js" type="text/javascript"></script>
+  <script src="./files/backbone-min.js" type="text/javascript"></script>
+  <script src="./files/swagger.js" type="text/javascript"></script>
+  <script src="./files/swagger-ui.js" type="text/javascript"></script>
+  <script src="./files/highlight.7.3.pack.js" type="text/javascript"></script>
+
+  <!-- enabling this will enable oauth2 implicit scope support -->
+  <script src="./files/swagger-oauth.js" type="text/javascript"></script>
+
+  <script type="text/javascript">
+    $(function () {
+      window.swaggerUi = new SwaggerUi({
+      url: "http://localhost:8000/swagger_matrix/api-docs",
+      dom_id: "swagger-ui-container",
+      supportedSubmitMethods: ['get', 'post', 'put', 'delete'],
+      onComplete: function(swaggerApi, swaggerUi){
+        log("Loaded SwaggerUI");
+
+        if(typeof initOAuth == "function") {
+          initOAuth({
+            clientId: "your-client-id",
+            realm: "your-realms",
+            appName: "your-app-name"
+          });
+        }
+        $('pre code').each(function(i, e) {
+          hljs.highlightBlock(e)
+        });
+      },
+      onFailure: function(data) {
+        log("Unable to Load SwaggerUI");
+      },
+      docExpansion: "none"
+    });
+
+    $('#input_apiKey').change(function() {
+      var key = $('#input_apiKey')[0].value;
+      log("key: " + key);
+      if(key && key.trim() != "") {
+        log("added key " + key);
+        window.authorizations.add("key", new ApiKeyAuthorization("access_token", key, "query"));
+      }
+    })
+    window.swaggerUi.load();
+  });
+  </script>
+</head>
+
+<body class="swagger-section">
+<div id="header">
+  <div class="swagger-ui-wrap">
+    <a id="logo" href="http://swagger.wordnik.com/">swagger</a>
+    <form id="api_selector">
+      <div class="input"><input placeholder="http://example.com/api" id="input_baseUrl" name="baseUrl" type="text"></div>
+      <div class="input"><input placeholder="access_token" id="input_apiKey" name="apiKey" type="text"></div>
+    </form>
+  </div>
+</div>
+
+<div id="message-bar" class="swagger-ui-wrap message-fail">Can't read from server.  It may not have the appropriate access-control-origin settings.</div>
+<div id="swagger-ui-container" class="swagger-ui-wrap"></div>
+
+
+</body></html>

docs/freenode.txt

@@ -0,0 +1 @@
+NCjcRSEG

docs/human-id-rules.rst

@@ -0,0 +1,79 @@
+This document outlines the format for human-readable IDs within matrix.
+
+Overview
+--------
+UTF-8 is quickly becoming the standard character encoding set on the web. As
+such, Matrix requires that all strings MUST be encoded as UTF-8. However,
+using Unicode as the character set for human-readable IDs is troublesome. There
+are many different characters which appear identical to each other, but would
+identify different users. In addition, there are non-printable characters which
+cannot be rendered by the end-user. This opens up a security vulnerability with
+phishing/spoofing of IDs, commonly known as a homograph attack.
+
+Web browers encountered this problem when International Domain Names were
+introduced. A variety of checks were put in place in order to protect users. If
+an address failed the check, the raw punycode would be displayed to disambiguate
+the address. Similar checks are performed by home servers in Matrix. However, 
+Matrix does not use punycode representations, and so does not show raw punycode 
+on a failed check. Instead, home servers must outright reject these misleading 
+IDs.
+
+Types of human-readable IDs
+---------------------------
+There are two main human-readable IDs in question:
+
+- Room aliases
+- User IDs
+ 
+Room aliases look like ``#localpart:domain``. These aliases point to opaque
+non human-readable room IDs. These pointers can change, so there is already an
+issue present with the same ID pointing to a different destination at a later
+date.
+
+User IDs look like ``@localpart:domain``. These represent actual end-users, and
+unlike room aliases, there is no layer of indirection. This presents a much
+greater concern with homograph attacks. 
+
+Checks
+------
+- Similar to web browsers.
+- blacklisted chars (e.g. non-printable characters)
+- mix of language sets from 'preferred' language not allowed. 
+- Language sets from CLDR dataset.
+- Treated in segments (localpart, domain)
+- Additional restrictions for ease of processing IDs.
+   - Room alias localparts MUST NOT have ``#`` or ``:``.
+   - User ID localparts MUST NOT have ``@`` or ``:``.
+
+Rejecting
+---------
+- Home servers MUST reject room aliases which do not pass the check, both on 
+  GETs and PUTs.
+- Home servers MUST reject user ID localparts which do not pass the check, both
+  on creation and on events.
+- Any home server whose domain does not pass this check, MUST use their punycode
+  domain name instead of the IDN, to prevent other home servers rejecting you.
+- Error code is ``M_FAILED_HUMAN_ID_CHECK``. (generic enough for both failing 
+  due to homograph attacks, and failing due to including ``:`` s, etc)
+- Error message MAY go into further information about which characters were
+  rejected and why.
+- Error message SHOULD contain a ``failed_keys`` key which contains an array
+  of strings which represent the keys which failed the check e.g::
+  
+    failed_keys: [ user_id, room_alias ]
+  
+Other considerations
+--------------------
+- Basic security: Informational key on the event attached by HS to say "unsafe 
+  ID". Problem: clients can just ignore it, and since it will appear only very
+  rarely, easy to forget when implementing clients.
+- Moderate security: Requires client handshake. Forces clients to implement
+  a check, else they cannot communicate with the misleading ID. However, this is
+  extra overhead in both client implementations and round-trips.
+- High security: Outright rejection of the ID at the point of creation / 
+  receiving event. Point of creation rejection is preferable to avoid the ID
+  entering the system in the first place. However, malicious HSes can just allow
+  the ID. Hence, other home servers must reject them if they see them in events.
+  Client never sees the problem ID, provided the HS is correctly implemented.
+- High security decided; client doesn't need to worry about it, no additional
+  protocol complexity aside from rejection of an event.

docs/server-server/security-threat-model.rst

@@ -1,141 +0,0 @@
-Overview
-========
-
-Scope
------
-
-This document considers threats specific to the server to server federation 
-synapse protocol.
-
-
-Attacker
---------
-
-It is assumed that the attacker can see and manipulate all network traffic 
-between any of the servers and may be in control of one or more homeservers 
-participating in the federation protocol.
-
-Threat Model
-============
-
-Denial of Service
------------------
-
-The attacker could attempt to prevent delivery of messages to or from the 
-victim in order to:
-
-    * Disrupt service or marketing campaign of a commercial competitor.
-    * Censor a discussion or censor a participant in a discussion.
-    * Perform general vandalism.
-
-Threat: Resource Exhaustion
-~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-An attacker could cause the victims server to exhaust a particular resource 
-(e.g. open TCP connections, CPU, memory, disk storage)
-
-Threat: Unrecoverable Consistency Violations
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-An attacker could send messages which created an unrecoverable "split-brain"
-state in the cluster such that the victim's servers could no longer dervive a
-consistent view of the chatroom state.
-
-Threat: Bad History
-~~~~~~~~~~~~~~~~~~~
-
-An attacker could convince the victim to accept invalid messages which the 
-victim would then include in their view of the chatroom history. Other servers
-in the chatroom would reject the invalid messages and potentially reject the
-victims messages as well since they depended on the invalid messages.
-
-Threat: Block Network Traffic
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-An attacker could try to firewall traffic between the victim's server and some
-or all of the other servers in the chatroom.
-
-Threat: High Volume of Messages
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-An attacker could send large volumes of messages to a chatroom with the victim
-making the chatroom unusable.
-
-Threat: Banning users without necessary authorisation
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-An attacker could attempt to ban a user from a chatroom with the necessary
-authorisation.
-
-Spoofing
---------
-
-An attacker could try to send a message claiming to be from the victim without 
-the victim having sent the message in order to:
-
-    * Impersonate the victim while performing illict activity.
-    * Obtain privileges of the victim.
-
-Threat: Altering Message Contents
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-An attacker could try to alter the contents of an existing message from the 
-victim.
-
-Threat: Fake Message "origin" Field
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-An attacker could try to send a new message purporting to be from the victim
-with a phony "origin" field.
-
-Spamming
---------
-
-The attacker could try to send a high volume of solicicted or unsolicted 
-messages to the victim in order to:
-    
-    * Find victims for scams.
-    * Market unwanted products.
-
-Threat: Unsoliticted Messages
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-An attacker could try to send messages to victims who do not wish to receive 
-them.
-
-Threat: Abusive Messages
-~~~~~~~~~~~~~~~~~~~~~~~~
-
-An attacker could send abusive or threatening messages to the victim
-
-Spying
-------
-
-The attacker could try to access message contents or metadata for messages sent
-by the victim or to the victim that were not intended to reach the attacker in
-order to:
-
-    * Gain sensitive personal or commercial information.
-    * Impersonate the victim using credentials contained in the messages.
-      (e.g. password reset messages)
-    * Discover who the victim was talking to and when.
-
-Threat: Disclosure during Transmission
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-An attacker could try to expose the message contents or metadata during 
-transmission between the servers.
-
-Threat: Disclosure to Servers Outside Chatroom
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-An attacker could try to convince servers within a chatroom to send messages to
-a server it controls that was not authorised to be within the chatroom.
-
-Threat: Disclosure to Servers Within Chatroom
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-An attacker could take control of a server within a chatroom to expose message
-contents or metadata for messages in that room.
-
-

docs/server-server/specification.rst

@@ -1,8 +1,8 @@
-============================
-Synapse Server-to-Server API
-============================
+===========================
+Matrix Server-to-Server API
+===========================
 
-A description of the protocol used to communicate between Synapse home servers;
+A description of the protocol used to communicate between Matrix home servers;
 also known as Federation.
 
 
@@ -10,7 +10,7 @@ Overview
 ========
 
 The server-server API is a mechanism by which two home servers can exchange
-Synapse event messages, both as a real-time push of current events, and as a
+Matrix event messages, both as a real-time push of current events, and as a
 historic fetching mechanism to synchronise past history for clients to view. It
 uses HTTP connections between each pair of servers involved as the underlying
 transport. Messages are exchanged between servers in real-time by active pushing
@@ -19,7 +19,7 @@ historic data for the purpose of back-filling scrollback buffers and the like
 can also be performed.
 
 
- { Synapse entities }                            { Synapse entities }
+  { Matrix clients }                              { Matrix clients }
      ^          |                                    ^          |
      |  events  |                                    |  events  |
      |          V                                    |          V
@@ -29,27 +29,53 @@ can also be performed.
  |                  |<--------( HTTP )-----------|                  |
  +------------------+                            +------------------+
 
+There are three main kinds of communication that occur between home servers:
 
-Transactions and PDUs
-=====================
+ * Queries
+   These are single request/response interactions between a given pair of
+   servers, initiated by one side sending an HTTP request to obtain some
+   information, and responded by the other. They are not persisted and contain
+   no long-term significant history. They simply request a snapshot state at the
+   instant the query is made.
 
-The communication between home servers is performed by a bidirectional exchange
-of messages. These messages are called Transactions, and are encoded as JSON
-objects with a dict as the top-level element, passed over HTTP. A Transaction is
-meaningful only to the pair of home servers that exchanged it; they are not
-globally-meaningful.
+ * EDUs - Ephemeral Data Units
+   These are notifications of events that are pushed from one home server to
+   another. They are not persisted and contain no long-term significant history,
+   nor does the receiving home server have to reply to them.
 
-Each transaction has an opaque ID and timestamp (UNIX epoch time in miliseconds)
-generated by its origin server, an origin and destination server name, a list of
-"previous IDs", and a list of PDUs - the actual message payload that the
-Transaction carries.
+ * PDUs - Persisted Data Units
+   These are notifications of events that are broadcast from one home server to
+   any others that are interested in the same "context" (namely, a Room ID).
+   They are persisted to long-term storage and form the record of history for
+   that context.
+
+Where Queries are presented directly across the HTTP connection as GET requests
+to specific URLs, EDUs and PDUs are further wrapped in an envelope called a
+Transaction, which is transferred from the origin to the destination home server
+using a PUT request.
+
+
+Transactions and EDUs/PDUs
+==========================
+
+The transfer of EDUs and PDUs between home servers is performed by an exchange
+of Transaction messages, which are encoded as JSON objects with a dict as the
+top-level element, passed over an HTTP PUT request. A Transaction is meaningful
+only to the pair of home servers that exchanged it; they are not globally-
+meaningful.
+
+Each transaction has an opaque ID and timestamp (UNIX epoch time in
+milliseconds) generated by its origin server, an origin and destination server
+name, a list of "previous IDs", and a list of PDUs - the actual message payload
+that the Transaction carries.
 
  {"transaction_id":"916d630ea616342b42e98a3be0b74113",
   "ts":1404835423000,
   "origin":"red",
   "destination":"blue",
   "prev_ids":["e1da392e61898be4d2009b9fecce5325"],
-  "pdus":[...]}
+  "pdus":[...],
+  "edus":[...]}
 
 The "previous IDs" field will contain a list of previous transaction IDs that
 the origin server has sent to this destination. Its purpose is to act as a
@@ -58,7 +84,9 @@ successfully received that Transaction, or ask for a retransmission if not.
 
 The "pdus" field of a transaction is a list, containing zero or more PDUs.[*]
 Each PDU is itself a dict containing a number of keys, the exact details of
-which will vary depending on the type of PDU.
+which will vary depending on the type of PDU. Similarly, the "edus" field is
+another list containing the EDUs. This key may be entirely absent if there are
+no EDUs to transfer.
 
 (* Normally the PDU list will be non-empty, but the server should cope with
 receiving an "empty" transaction, as this is useful for informing peers of other
@@ -86,7 +114,7 @@ field of a PDU refers to PDUs that any origin server has sent, rather than
 previous IDs that this origin has sent. This list may refer to other PDUs sent
 by the same origin as the current one, or other origins.
 
-Because of the distributed nature of participants in a Synapse conversation, it
+Because of the distributed nature of participants in a Matrix conversation, it
 is impossible to establish a globally-consistent total ordering on the events.
 However, by annotating each outbound PDU at its origin with IDs of other PDUs it
 has received, a partial ordering can be constructed allowing causallity
@@ -112,13 +140,22 @@ so on. This part needs refining. And writing in its own document as the details
 relate to the server/system as a whole, not specifically to server-server
 federation.]]
 
+EDUs, by comparison to PDUs, do not have an ID, a context, or a list of
+"previous" IDs. The only mandatory fields for these are the type, origin and
+destination home server names, and the actual nested content.
+
+ {"edu_type":"m.presence",
+  "origin":"blue",
+  "destination":"orange",
+  "content":...}
+
 
 Protocol URLs
 =============
 
 All these URLs are namespaced within a prefix of 
 
-  /matrix/federation/v1/...
+  /_matrix/federation/v1/...
 
 For active pushing of messages representing live activity "as it happens":
 
@@ -179,3 +216,16 @@ To stream events all the events:
   Retrieves all of the transactions later than any version given by the "v"
   arguments. [[TODO(paul): I'm not sure what the "origin" argument does because
   I think at some point in the code it's got swapped around.]]
+
+
+To make a query:
+
+  GET .../query/:query_type
+    Query args: as specified by the individual query types
+
+  Response: JSON encoding of a response object
+
+  Performs a single query request on the receiving home server. The Query Type
+  part of the path specifies the kind of query being made, and its query
+  arguments have a meaning specific to that kind of query. The response is a
+  JSON-encoded object whose meaning also depends on the kind of query.

experiments/cursesio.py

@@ -1,4 +1,4 @@
-# Copyright 2014 matrix.org
+# Copyright 2014 OpenMarket Ltd
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.

experiments/test_messaging.py

@@ -1,5 +1,5 @@
 # -*- coding: utf-8 -*-
-# Copyright 2014 matrix.org
+# Copyright 2014 OpenMarket Ltd
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.

graph/graph.py

@@ -1,4 +1,4 @@
-# Copyright 2014 matrix.org
+# Copyright 2014 OpenMarket Ltd
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
@@ -113,14 +113,14 @@ def make_graph(pdus, room, filename_prefix):
                 graph.add_edge(state_edge)
 
     graph.write('%s.dot' % filename_prefix, format='raw', prog='dot')
-    graph.write_png("%s.png" % filename_prefix, prog='dot')
+#    graph.write_png("%s.png" % filename_prefix, prog='dot')
     graph.write_svg("%s.svg" % filename_prefix, prog='dot')
 
 
 def get_pdus(host, room):
     transaction = json.loads(
         urllib2.urlopen(
-            "http://%s/matrix/federation/v1/context/%s/" % (host, room)
+            "http://%s/_matrix/federation/v1/context/%s/" % (host, room)
         ).read()
     )
 

jsfiddles/create_room_send_msg/demo.css

@@ -0,0 +1,17 @@
+.loggedin {
+    visibility: hidden;
+}
+
+p {
+    font-family: monospace;
+}
+
+table
+{
+    border-spacing:5px;
+}
+
+th,td
+{
+    padding:5px;
+}

jsfiddles/create_room_send_msg/demo.html

@@ -0,0 +1,30 @@
+<div>
+    <p>This room creation / message sending demo requires a home server to be running on http://localhost:8008</p>
+</div>
+<form class="loginForm">
+    <input type="text" id="userLogin" placeholder="Username"></input>
+    <input type="password" id="passwordLogin" placeholder="Password"></input>
+    <input type="button" class="login" value="Login"></input>
+</form>
+<div class="loggedin">
+    <form class="createRoomForm">
+        <input type="text" id="roomAlias" placeholder="Room alias (optional)"></input>
+        <input type="button" class="createRoom" value="Create Room"></input>
+    </form>
+    <form class="sendMessageForm">
+        <input type="text" id="roomId" placeholder="Room ID"></input>
+        <input type="text" id="messageBody" placeholder="Message body"></input>
+        <input type="button" class="sendMessage" value="Send Message"></input>
+    </form>
+    <table id="rooms">
+        <tbody>
+            <tr>
+                <th>Room ID</th>
+                <th>My state</th>
+                <th>Room Alias</th>
+                <th>Latest message</th>
+            </tr>
+        </tbody>
+    </table>
+</div>
+    

jsfiddles/create_room_send_msg/demo.js

@@ -0,0 +1,113 @@
+var accountInfo = {};
+
+var showLoggedIn = function(data) {
+    accountInfo = data;
+    getCurrentRoomList();
+    $(".loggedin").css({visibility: "visible"});
+};
+
+$('.login').live('click', function() {
+    var user = $("#userLogin").val();
+    var password = $("#passwordLogin").val();
+    $.ajax({
+        url: "http://localhost:8008/_matrix/client/api/v1/login",
+        type: "POST",
+        contentType: "application/json; charset=utf-8",
+        data: JSON.stringify({ user: user, password: password, type: "m.login.password" }),
+        dataType: "json",
+        success: function(data) {
+            showLoggedIn(data);
+        },
+        error: function(err) {
+            var errMsg = "To try this, you need a home server running!";
+            var errJson = $.parseJSON(err.responseText);
+            if (errJson) {
+                errMsg = JSON.stringify(errJson);   
+            }
+            alert(errMsg);  
+        }
+    }); 
+});
+
+var getCurrentRoomList = function() {
+    var url = "http://localhost:8008/_matrix/client/api/v1/initialSync?access_token=" + accountInfo.access_token + "&limit=1";
+    $.getJSON(url, function(data) {
+        var rooms = data.rooms;
+        for (var i=0; i<rooms.length; ++i) {
+            rooms[i].latest_message = rooms[i].messages.chunk[0].content.body;
+            addRoom(rooms[i]);   
+        }
+    }).fail(function(err) {
+        alert(JSON.stringify($.parseJSON(err.responseText)));
+    });
+};
+
+$('.createRoom').live('click', function() {
+    var roomAlias = $("#roomAlias").val();
+    var data = {};
+    if (roomAlias.length > 0) {
+        data.room_alias_name = roomAlias;   
+    }
+    $.ajax({
+        url: "http://localhost:8008/_matrix/client/api/v1/createRoom?access_token="+accountInfo.access_token,
+        type: "POST",
+        contentType: "application/json; charset=utf-8",
+        data: JSON.stringify(data),
+        dataType: "json",
+        success: function(data) {
+            data.membership = "join"; // you are automatically joined into every room you make.
+            data.latest_message = "";
+            addRoom(data);
+        },
+        error: function(err) {
+            alert(JSON.stringify($.parseJSON(err.responseText)));  
+        }
+    }); 
+});
+
+var addRoom = function(data) {
+    row = "<tr>" +
+        "<td>"+data.room_id+"</td>" +
+        "<td>"+data.membership+"</td>" +
+        "<td>"+data.room_alias+"</td>" +
+        "<td>"+data.latest_message+"</td>" +
+        "</tr>";
+    $("#rooms").append(row);
+};
+
+$('.sendMessage').live('click', function() {
+    var roomId = $("#roomId").val();
+    var body = $("#messageBody").val();
+    var msgId = $.now();
+    
+    if (roomId.length === 0 || body.length === 0) {
+        return;
+    }
+    
+    var url = "http://localhost:8008/_matrix/client/api/v1/rooms/$roomid/send/m.room.message?access_token=$token";
+    url = url.replace("$token", accountInfo.access_token);
+    url = url.replace("$roomid", encodeURIComponent(roomId));
+    
+    var data = {
+        msgtype: "m.text",
+        body: body
+    };
+    
+    $.ajax({
+        url: url,
+        type: "POST",
+        contentType: "application/json; charset=utf-8",
+        data: JSON.stringify(data),
+        dataType: "json",
+        success: function(data) {
+            $("#messageBody").val("");
+            // wipe the table and reload it. Using the event stream would be the best
+            // solution but that is out of scope of this fiddle.
+            $("#rooms").find("tr:gt(0)").remove();
+            getCurrentRoomList();
+        },
+        error: function(err) {
+            alert(JSON.stringify($.parseJSON(err.responseText)));  
+        }
+    }); 
+});

jsfiddles/event_stream/demo.css

@@ -0,0 +1,17 @@
+.loggedin {
+    visibility: hidden;
+}
+
+p {
+    font-family: monospace;
+}
+
+table
+{
+    border-spacing:5px;
+}
+
+th,td
+{
+    padding:5px;
+}

jsfiddles/event_stream/demo.html

@@ -0,0 +1,23 @@
+<div>
+    <p>This event stream demo requires a home server to be running on http://localhost:8008</p>
+</div>
+<form class="loginForm">
+    <input type="text" id="userLogin" placeholder="Username"></input>
+    <input type="password" id="passwordLogin" placeholder="Password"></input>
+    <input type="button" class="login" value="Login"></input>
+</form>
+<div class="loggedin">
+    <form class="sendMessageForm">
+        <input type="button" class="sendMessage" value="Send random message"></input>
+    </form>
+    <p id="streamErrorText"></p>
+    <table id="rooms">
+        <tbody>
+            <tr>
+                <th>Room ID</th>
+                <th>Latest message</th>
+            </tr>
+        </tbody>
+    </table>
+</div>
+    

jsfiddles/event_stream/demo.js

@@ -0,0 +1,145 @@
+var accountInfo = {};
+
+var eventStreamInfo = {
+    from: "END"
+};
+
+var roomInfo = [];
+
+var longpollEventStream = function() {
+    var url = "http://localhost:8008/_matrix/client/api/v1/events?access_token=$token&from=$from";
+    url = url.replace("$token", accountInfo.access_token);
+    url = url.replace("$from", eventStreamInfo.from);
+    
+    $.getJSON(url, function(data) {
+        eventStreamInfo.from = data.end;
+        
+        var hasNewLatestMessage = false;
+        for (var i=0; i<data.chunk.length; ++i) {
+            if (data.chunk[i].type === "m.room.message") {
+                for (var j=0; j<roomInfo.length; ++j) {
+                    if (roomInfo[j].room_id === data.chunk[i].room_id) {
+                        roomInfo[j].latest_message = data.chunk[i].content.body;
+                        hasNewLatestMessage = true;
+                    }
+                }
+            }
+        }
+        
+        if (hasNewLatestMessage) {
+           setRooms(roomInfo);
+        }
+        $("#streamErrorText").text("");
+        longpollEventStream();
+    }).fail(function(err) {
+        $("#streamErrorText").text("Event stream error: "+JSON.stringify($.parseJSON(err.responseText)));
+        setTimeout(longpollEventStream, 5000);
+    });
+};
+
+var showLoggedIn = function(data) {
+    accountInfo = data;
+    longpollEventStream();
+    getCurrentRoomList();
+    $(".loggedin").css({visibility: "visible"});
+};
+
+$('.login').live('click', function() {
+    var user = $("#userLogin").val();
+    var password = $("#passwordLogin").val();
+    $.ajax({
+        url: "http://localhost:8008/_matrix/client/api/v1/login",
+        type: "POST",
+        contentType: "application/json; charset=utf-8",
+        data: JSON.stringify({ user: user, password: password, type: "m.login.password" }),
+        dataType: "json",
+        success: function(data) {
+            $("#rooms").find("tr:gt(0)").remove();
+            showLoggedIn(data);
+        },
+        error: function(err) {
+            var errMsg = "To try this, you need a home server running!";
+            var errJson = $.parseJSON(err.responseText);
+            if (errJson) {
+                errMsg = JSON.stringify(errJson);   
+            }
+            alert(errMsg);  
+        }
+    }); 
+});
+
+var getCurrentRoomList = function() {
+    $("#roomId").val("");
+    var url = "http://localhost:8008/_matrix/client/api/v1/initialSync?access_token=" + accountInfo.access_token + "&limit=1";
+    $.getJSON(url, function(data) {
+        var rooms = data.rooms;
+        for (var i=0; i<rooms.length; ++i) {
+            if ("messages" in rooms[i]) {
+                rooms[i].latest_message = rooms[i].messages.chunk[0].content.body;   
+            }
+        }
+        roomInfo = rooms;
+        setRooms(roomInfo);  
+    }).fail(function(err) {
+        alert(JSON.stringify($.parseJSON(err.responseText)));
+    });
+};
+
+$('.sendMessage').live('click', function() {
+    if (roomInfo.length === 0) {
+        alert("There is no room to send a message to!");
+        return;
+    }
+    
+    var index = Math.floor(Math.random() * roomInfo.length);
+    
+    sendMessage(roomInfo[index].room_id);
+});
+
+var sendMessage = function(roomId) {
+    var body = "jsfiddle message @" + $.now();
+    
+    if (roomId.length === 0) {
+        return;
+    }
+    
+    var url = "http://localhost:8008/_matrix/client/api/v1/rooms/$roomid/send/m.room.message?access_token=$token";
+    url = url.replace("$token", accountInfo.access_token);
+    url = url.replace("$roomid", encodeURIComponent(roomId));
+    
+    var data = {
+        msgtype: "m.text",
+        body: body
+    };
+    
+    $.ajax({
+        url: url,
+        type: "POST",
+        contentType: "application/json; charset=utf-8",
+        data: JSON.stringify(data),
+        dataType: "json",
+        success: function(data) {
+            $("#messageBody").val("");
+        },
+        error: function(err) {
+            alert(JSON.stringify($.parseJSON(err.responseText)));  
+        }
+    }); 
+};
+
+var setRooms = function(roomList) {
+    // wipe existing entries
+    $("#rooms").find("tr:gt(0)").remove();
+    
+    var rows = "";
+    for (var i=0; i<roomList.length; ++i) {
+        row = "<tr>" +
+            "<td>"+roomList[i].room_id+"</td>" +
+            "<td>"+roomList[i].latest_message+"</td>" +
+            "</tr>";  
+        rows += row;
+    }
+    
+    $("#rooms").append(rows);
+};
+

jsfiddles/example_app/demo.css

@@ -0,0 +1,43 @@
+.roomListDashboard, .roomContents, .sendMessageForm {
+    visibility: hidden;
+}
+
+.roomList {
+    background-color: #909090;
+}
+
+.messageWrapper {
+    background-color: #EEEEEE;
+    height: 400px;
+    overflow: scroll;
+}
+
+.membersWrapper {
+    background-color: #EEEEEE;
+    height: 200px;
+    width: 50%;
+    overflow: scroll;
+}
+
+.textEntry {
+    width: 100%
+}
+
+p {
+    font-family: monospace;
+}
+
+table
+{
+    border-spacing:5px;
+}
+
+th,td
+{
+    padding:5px;
+}
+
+.roomList tr:not(:first-child):hover {
+    background-color: orange;
+    cursor: pointer;
+}

jsfiddles/example_app/demo.details

@@ -0,0 +1,7 @@
+ name: Example Matrix Client
+ description: Includes login, live event streaming, creating rooms, sending messages and viewing member lists.
+ authors:
+   - matrix.org
+ resources:
+   - http://matrix.org
+ normalize_css: no

jsfiddles/example_app/demo.html

@@ -0,0 +1,56 @@
+<div class="signUp">
+    <p>Matrix example application: Requires a local home server running at http://localhost:8008</p>
+    <form class="registrationForm">
+        <p>No account? Register:</p>
+        <input type="text" id="userReg" placeholder="Username"></input>
+        <input type="password" id="passwordReg" placeholder="Password"></input>
+        <input type="button" class="register" value="Register"></input>
+    </form>
+    <form class="loginForm">
+        <p>Got an account? Login:</p>
+        <input type="text" id="userLogin" placeholder="Username"></input>
+        <input type="password" id="passwordLogin" placeholder="Password"></input>
+        <input type="button" class="login" value="Login"></input>
+    </form>
+</div>
+
+<div class="roomListDashboard">
+    <form class="createRoomForm">
+        <input type="text" id="roomAlias" placeholder="Room alias"></input>
+        <input type="button" class="createRoom" value="Create Room"></input>
+    </form>
+    <table id="rooms" class="roomList">
+        <tbody>
+            <tr>
+                <th>Room</th>
+                <th>My state</th>
+                <th>Latest message</th>
+            </tr>
+        </tbody>
+    </table>
+</div>
+
+<div class="roomContents">
+    <p id="roomName">Select a room</p>
+    <div class="messageWrapper">
+        <table id="messages">
+            <tbody>
+            </tbody>
+        </table>
+    </div>
+    <form class="sendMessageForm">
+        <input type="text" class="textEntry" id="body" placeholder="Enter text here..." onkeydown="javascript:if (event.keyCode == 13) document.getElementById('sendMsg').focus()"></input>
+        <input type="button" class="sendMessage" id="sendMsg" value="Send"></input>
+    </form>
+</div>
+
+<div>
+    <p>Member list:</p>
+    <div class="membersWrapper">
+        <table id="members">
+            <tbody>
+            </tbody>
+        </table>
+    </div>
+</div>
+    

jsfiddles/example_app/demo.js

@@ -0,0 +1,327 @@
+var accountInfo = {};
+
+var eventStreamInfo = {
+    from: "END"
+};
+
+var roomInfo = [];
+var memberInfo = [];
+var viewingRoomId;
+
+// ************** Event Streaming **************
+var longpollEventStream = function() {
+    var url = "http://localhost:8008/_matrix/client/api/v1/events?access_token=$token&from=$from";
+    url = url.replace("$token", accountInfo.access_token);
+    url = url.replace("$from", eventStreamInfo.from);
+
+    $.getJSON(url, function(data) {
+        eventStreamInfo.from = data.end;
+        
+        var hasNewLatestMessage = false;
+        var updatedMemberList = false;
+        var i=0;
+        var j=0;
+        for (i=0; i<data.chunk.length; ++i) {
+            if (data.chunk[i].type === "m.room.message") {
+                console.log("Got new message: " + JSON.stringify(data.chunk[i]));
+                if (viewingRoomId === data.chunk[i].room_id) {
+                    addMessage(data.chunk[i]);
+                }
+                
+                for (j=0; j<roomInfo.length; ++j) {
+                    if (roomInfo[j].room_id === data.chunk[i].room_id) {
+                        roomInfo[j].latest_message = data.chunk[i].content.body;
+                        hasNewLatestMessage = true;
+                    }
+                }
+            }
+            else if (data.chunk[i].type === "m.room.member") {
+                if (viewingRoomId === data.chunk[i].room_id) {
+                    console.log("Got new member: " + JSON.stringify(data.chunk[i]));
+                    addMessage(data.chunk[i]);
+                    for (j=0; j<memberInfo.length; ++j) {
+                        if (memberInfo[j].state_key === data.chunk[i].state_key) {
+                            memberInfo[j] = data.chunk[i];
+                            updatedMemberList = true;
+                            break;
+                        }
+                    }
+                    if (!updatedMemberList) {
+                        memberInfo.push(data.chunk[i]);  
+                        updatedMemberList = true;
+                    }
+                }
+                if (data.chunk[i].state_key === accountInfo.user_id) {
+                    getCurrentRoomList(); // update our join/invite list
+                }
+            }
+            else {
+                console.log("Discarding: " + JSON.stringify(data.chunk[i]));
+            }
+        }
+        
+        if (hasNewLatestMessage) {
+           setRooms(roomInfo);
+        }
+        if (updatedMemberList) {
+            $("#members").empty();
+            for (i=0; i<memberInfo.length; ++i) { 
+                addMember(memberInfo[i]);
+            }
+        }
+        longpollEventStream();
+    }).fail(function(err) {
+        setTimeout(longpollEventStream, 5000);
+    });
+};
+
+// ************** Registration and Login **************
+var onLoggedIn = function(data) {
+    accountInfo = data;
+    longpollEventStream();
+    getCurrentRoomList();
+    $(".roomListDashboard").css({visibility: "visible"});
+    $(".roomContents").css({visibility: "visible"});
+    $(".signUp").css({display: "none"});
+};
+
+$('.login').live('click', function() {
+    var user = $("#userLogin").val();
+    var password = $("#passwordLogin").val();
+    $.ajax({
+        url: "http://localhost:8008/_matrix/client/api/v1/login",
+        type: "POST",
+        contentType: "application/json; charset=utf-8",
+        data: JSON.stringify({ user: user, password: password, type: "m.login.password" }),
+        dataType: "json",
+        success: function(data) {
+            onLoggedIn(data);
+        },
+        error: function(err) {
+            alert("Unable to login: is the home server running?");  
+        }
+    }); 
+});
+
+$('.register').live('click', function() {
+    var user = $("#userReg").val();
+    var password = $("#passwordReg").val();
+    $.ajax({
+        url: "http://localhost:8008/_matrix/client/api/v1/register",
+        type: "POST",
+        contentType: "application/json; charset=utf-8",
+        data: JSON.stringify({ user_id: user, password: password }),
+        dataType: "json",
+        success: function(data) {
+            onLoggedIn(data);
+        },
+        error: function(err) {
+            var msg = "Is the home server running?";
+            var errJson = $.parseJSON(err.responseText);
+            if (errJson !== null) {
+                msg = errJson.error;   
+            }
+            alert("Unable to register: "+msg);  
+        }
+    });
+});
+
+// ************** Creating a room ******************
+$('.createRoom').live('click', function() {
+    var roomAlias = $("#roomAlias").val();
+    var data = {};
+    if (roomAlias.length > 0) {
+        data.room_alias_name = roomAlias;   
+    }
+    $.ajax({
+        url: "http://localhost:8008/_matrix/client/api/v1/createRoom?access_token="+accountInfo.access_token,
+        type: "POST",
+        contentType: "application/json; charset=utf-8",
+        data: JSON.stringify(data),
+        dataType: "json",
+        success: function(response) {
+            $("#roomAlias").val("");
+            response.membership = "join"; // you are automatically joined into every room you make.
+            response.latest_message = "";
+            
+            roomInfo.push(response);
+            setRooms(roomInfo);
+        },
+        error: function(err) {
+            alert(JSON.stringify($.parseJSON(err.responseText)));  
+        }
+    }); 
+});
+
+// ************** Getting current state **************
+var getCurrentRoomList = function() {
+    var url = "http://localhost:8008/_matrix/client/api/v1/initialSync?access_token=" + accountInfo.access_token + "&limit=1";
+    $.getJSON(url, function(data) {
+        var rooms = data.rooms;
+        for (var i=0; i<rooms.length; ++i) {
+            if ("messages" in rooms[i]) {
+                rooms[i].latest_message = rooms[i].messages.chunk[0].content.body;   
+            }
+        }
+        roomInfo = rooms;
+        setRooms(roomInfo);  
+    }).fail(function(err) {
+        alert(JSON.stringify($.parseJSON(err.responseText)));
+    });
+};
+
+var loadRoomContent = function(roomId) {
+    console.log("loadRoomContent " + roomId);
+    viewingRoomId = roomId;
+    $("#roomName").text("Room: "+roomId);
+    $(".sendMessageForm").css({visibility: "visible"});
+    getMessages(roomId);
+    getMemberList(roomId);
+};
+
+var getMessages = function(roomId) {
+    $("#messages").empty();
+    var url = "http://localhost:8008/_matrix/client/api/v1/rooms/" + 
+              encodeURIComponent(roomId) + "/messages?access_token=" + accountInfo.access_token + "&from=END&dir=b&limit=10";
+    $.getJSON(url, function(data) {
+        for (var i=data.chunk.length-1; i>=0; --i) {
+            addMessage(data.chunk[i]);   
+        }
+    });
+};
+
+var getMemberList = function(roomId) {
+    $("#members").empty();
+    memberInfo = [];
+    var url = "http://localhost:8008/_matrix/client/api/v1/rooms/" + 
+              encodeURIComponent(roomId) + "/members?access_token=" + accountInfo.access_token;
+    $.getJSON(url, function(data) {
+        for (var i=0; i<data.chunk.length; ++i) {
+            memberInfo.push(data.chunk[i]);
+            addMember(data.chunk[i]);   
+        }
+    });
+};
+
+// ************** Sending messages **************
+$('.sendMessage').live('click', function() {
+    if (viewingRoomId === undefined) {
+        alert("There is no room to send a message to!");
+        return;
+    }
+    var body = $("#body").val();
+    sendMessage(viewingRoomId, body);
+});
+
+var sendMessage = function(roomId, body) {
+    var msgId = $.now();
+    
+    var url = "http://localhost:8008/_matrix/client/api/v1/rooms/$roomid/send/m.room.message?access_token=$token";
+    url = url.replace("$token", accountInfo.access_token);
+    url = url.replace("$roomid", encodeURIComponent(roomId));
+    
+    var data = {
+        msgtype: "m.text",
+        body: body
+    };
+    
+    $.ajax({
+        url: url,
+        type: "POST",
+        contentType: "application/json; charset=utf-8",
+        data: JSON.stringify(data),
+        dataType: "json",
+        success: function(data) {
+            $("#body").val("");
+        },
+        error: function(err) {
+            alert(JSON.stringify($.parseJSON(err.responseText)));  
+        }
+    });
+};
+
+// ************** Navigation and DOM manipulation **************
+var setRooms = function(roomList) {
+    // wipe existing entries
+    $("#rooms").find("tr:gt(0)").remove();
+    
+    var rows = "";
+    for (var i=0; i<roomList.length; ++i) {
+        row = "<tr>" +
+              "<td>"+roomList[i].room_id+"</td>" +
+              "<td>"+roomList[i].membership+"</td>" +
+              "<td>"+roomList[i].latest_message+"</td>" +
+              "</tr>";  
+        rows += row;
+    }
+    
+    $("#rooms").append(rows);
+    
+    $('#rooms').find("tr").click(function(){
+        var roomId = $(this).find('td:eq(0)').text();
+        var membership = $(this).find('td:eq(1)').text();
+        if (membership !== "join") {
+            console.log("Joining room " + roomId); 
+            var url = "http://localhost:8008/_matrix/client/api/v1/rooms/$roomid/join?access_token=$token";
+            url = url.replace("$token", accountInfo.access_token);
+            url = url.replace("$roomid", encodeURIComponent(roomId));
+            $.ajax({
+                url: url,
+                type: "POST",
+                contentType: "application/json; charset=utf-8",
+                data: JSON.stringify({membership: "join"}),
+                dataType: "json",
+                success: function(data) {
+                    loadRoomContent(roomId);
+                    getCurrentRoomList();
+                },
+                error: function(err) {
+                    alert(JSON.stringify($.parseJSON(err.responseText)));  
+                }
+            });
+        }
+        else {
+            loadRoomContent(roomId);
+        }
+    });
+};
+
+var addMessage = function(data) {
+
+    var msg = data.content.body;
+    if (data.type === "m.room.member") {
+        if (data.content.membership === undefined) {
+            return;
+        }
+        if (data.content.membership === "invite") {
+            msg = "<em>invited " + data.state_key + " to the room</em>";
+        }
+        else if (data.content.membership === "join") {
+            msg = "<em>joined the room</em>";
+        }
+        else if (data.content.membership === "leave") {
+            msg = "<em>left the room</em>";
+        }
+        else if (data.content.membership === "ban") {
+            msg = "<em>was banned from the room</em>";
+        }
+    }
+    if (msg === undefined) {
+        return;
+    }
+
+    var row = "<tr>" +
+              "<td>"+data.user_id+"</td>" +
+              "<td>"+msg+"</td>" +
+              "</tr>"; 
+    $("#messages").append(row);
+};
+
+var addMember = function(data) {
+    var row = "<tr>" +
+              "<td>"+data.state_key+"</td>" +
+              "<td>"+data.content.membership+"</td>" +
+              "</tr>"; 
+    $("#members").append(row);
+};
+

jsfiddles/register_login/demo.css

@@ -0,0 +1,7 @@
+.loggedin {
+    visibility: hidden;
+}
+
+p {
+    font-family: monospace;
+}

jsfiddles/register_login/demo.html

@@ -0,0 +1,20 @@
+<div>
+    <p>This registration/login demo requires a home server to be running on http://localhost:8008</p>
+</div>
+<form class="registrationForm">
+    <input type="text" id="user" placeholder="Username"></input>
+    <input type="password" id="password" placeholder="Password"></input>
+    <input type="button" class="register" value="Register"></input>
+</form>
+<form class="loginForm">
+    <input type="text" id="userLogin" placeholder="Username"></input>
+    <input type="password" id="passwordLogin" placeholder="Password"></input>
+    <input type="button" class="login" value="Login"></input>
+</form>
+<div class="loggedin">
+    <p id="welcomeText"></p>
+    <input type="button" class="testToken" value="Test token"></input>
+    <input type="button" class="logout" value="Logout"></input>
+    <p id="imSyncText"></p>
+</div>
+    

jsfiddles/register_login/demo.js

@@ -0,0 +1,79 @@
+var accountInfo = {};
+
+var showLoggedIn = function(data) {
+    accountInfo = data;
+    $(".loggedin").css({visibility: "visible"});
+    $("#welcomeText").text("Welcome " + accountInfo.user_id+". Your access token is:  " +
+                           accountInfo.access_token);    
+};
+
+$('.register').live('click', function() {
+    var user = $("#user").val();
+    var password = $("#password").val();
+    $.ajax({
+        url: "http://localhost:8008/_matrix/client/api/v1/register",
+        type: "POST",
+        contentType: "application/json; charset=utf-8",
+        data: JSON.stringify({ user_id: user, password: password }),
+        dataType: "json",
+        success: function(data) {
+            showLoggedIn(data);
+        },
+        error: function(err) {
+            var errMsg = "To try this, you need a home server running!";
+            var errJson = $.parseJSON(err.responseText);
+            if (errJson) {
+                errMsg = JSON.stringify(errJson);   
+            }
+            alert(errMsg);   
+        }
+    });
+});
+
+var login = function(user, password) {
+    $.ajax({
+        url: "http://localhost:8008/_matrix/client/api/v1/login",
+        type: "POST",
+        contentType: "application/json; charset=utf-8",
+        data: JSON.stringify({ user: user, password: password, type: "m.login.password" }),
+        dataType: "json",
+        success: function(data) {
+            showLoggedIn(data);
+        },
+        error: function(err) {
+            var errMsg = "To try this, you need a home server running!";
+            var errJson = $.parseJSON(err.responseText);
+            if (errJson) {
+                errMsg = JSON.stringify(errJson);   
+            }
+            alert(errMsg);  
+        }
+    });  
+};
+
+$('.login').live('click', function() {
+    var user = $("#userLogin").val();
+    var password = $("#passwordLogin").val();
+    $.getJSON("http://localhost:8008/_matrix/client/api/v1/login", function(data) {
+        if (data.flows[0].type !== "m.login.password") {
+            alert("I don't know how to login with this type: " + data.type);
+            return;
+        }
+        login(user, password);
+    });
+});
+
+$('.logout').live('click', function() {
+    accountInfo = {};
+    $("#imSyncText").text("");
+    $(".loggedin").css({visibility: "hidden"});
+});
+
+$('.testToken').live('click', function() {
+    var url = "http://localhost:8008/_matrix/client/api/v1/initialSync?access_token=" + accountInfo.access_token + "&limit=1";
+    $.getJSON(url, function(data) {
+         $("#imSyncText").text(JSON.stringify(data, undefined, 2));
+    }).fail(function(err) {
+        $("#imSyncText").text(JSON.stringify($.parseJSON(err.responseText)));
+    });
+});

jsfiddles/room_memberships/demo.css

@@ -0,0 +1,17 @@
+.loggedin {
+    visibility: hidden;
+}
+
+p {
+    font-family: monospace;
+}
+
+table
+{
+    border-spacing:5px;
+}
+
+th,td
+{
+    padding:5px;
+}

jsfiddles/room_memberships/demo.html

@@ -0,0 +1,37 @@
+<div>
+    <p>This room membership demo requires a home server to be running on http://localhost:8008</p>
+</div>
+<form class="loginForm">
+    <input type="text" id="userLogin" placeholder="Username"></input>
+    <input type="password" id="passwordLogin" placeholder="Password"></input>
+    <input type="button" class="login" value="Login"></input>
+</form>
+<div class="loggedin">
+    <form class="createRoomForm">
+        <input type="button" class="createRoom" value="Create Room"></input>
+    </form>
+    <form class="changeMembershipForm">
+        <input type="text" id="roomId" placeholder="Room ID"></input>
+        <input type="text" id="targetUser" placeholder="Target User ID"></input>
+        <select id="membership">
+            <option value="invite">invite</option>
+            <option value="join">join</option>
+            <option value="leave">leave</option>
+        </select>
+        <input type="button" class="changeMembership" value="Change Membership"></input>
+    </form>
+    <form class="joinAliasForm">
+        <input type="text" id="roomAlias" placeholder="Room Alias (#name:domain)"></input>
+        <input type="button" class="joinAlias" value="Join via Alias"></input>
+    </form>
+    <table id="rooms">
+        <tbody>
+            <tr>
+                <th>Room ID</th>
+                <th>My state</th>
+                <th>Room Alias</th>
+            </tr>
+        </tbody>
+    </table>
+</div>
+    

jsfiddles/room_memberships/demo.js

@@ -0,0 +1,141 @@
+var accountInfo = {};
+
+var showLoggedIn = function(data) {
+    accountInfo = data;
+    getCurrentRoomList();
+    $(".loggedin").css({visibility: "visible"});
+    $("#membership").change(function() {
+    if ($("#membership").val() === "invite") {
+        $("#targetUser").css({visibility: "visible"});
+    }
+    else {
+        $("#targetUser").css({visibility: "hidden"});
+    }
+});
+};
+
+$('.login').live('click', function() {
+    var user = $("#userLogin").val();
+    var password = $("#passwordLogin").val();
+    $.ajax({
+        url: "http://localhost:8008/_matrix/client/api/v1/login",
+        type: "POST",
+        contentType: "application/json; charset=utf-8",
+        data: JSON.stringify({ user: user, password: password, type: "m.login.password" }),
+        dataType: "json",
+        success: function(data) {
+            $("#rooms").find("tr:gt(0)").remove();
+            showLoggedIn(data);
+        },
+        error: function(err) {
+            var errMsg = "To try this, you need a home server running!";
+            var errJson = $.parseJSON(err.responseText);
+            if (errJson) {
+                errMsg = JSON.stringify(errJson);   
+            }
+            alert(errMsg); 
+        }
+    }); 
+});
+
+var getCurrentRoomList = function() {
+    $("#roomId").val("");
+    // wipe the table and reload it. Using the event stream would be the best
+    // solution but that is out of scope of this fiddle.
+    $("#rooms").find("tr:gt(0)").remove();
+    
+    var url = "http://localhost:8008/_matrix/client/api/v1/initialSync?access_token=" + accountInfo.access_token + "&limit=1";
+    $.getJSON(url, function(data) {
+        var rooms = data.rooms;
+        for (var i=0; i<rooms.length; ++i) {
+            addRoom(rooms[i]);   
+        }
+    }).fail(function(err) {
+        alert(JSON.stringify($.parseJSON(err.responseText)));
+    });
+};
+
+$('.createRoom').live('click', function() {
+    var data = {};
+    $.ajax({
+        url: "http://localhost:8008/_matrix/client/api/v1/createRoom?access_token="+accountInfo.access_token,
+        type: "POST",
+        contentType: "application/json; charset=utf-8",
+        data: JSON.stringify(data),
+        dataType: "json",
+        success: function(data) {
+            data.membership = "join"; // you are automatically joined into every room you make.
+            data.latest_message = "";
+            addRoom(data);
+        },
+        error: function(err) {
+            alert(JSON.stringify($.parseJSON(err.responseText)));  
+        }
+    }); 
+});
+
+var addRoom = function(data) {
+    row = "<tr>" +
+        "<td>"+data.room_id+"</td>" +
+        "<td>"+data.membership+"</td>" +
+        "<td>"+data.room_alias+"</td>" +
+        "</tr>";
+    $("#rooms").append(row);
+};
+
+$('.changeMembership').live('click', function() {
+    var roomId = $("#roomId").val();
+    var member = $("#targetUser").val();
+    var membership = $("#membership").val();
+    
+    if (roomId.length === 0) {
+        return;
+    }
+    
+    var url = "http://localhost:8008/_matrix/client/api/v1/rooms/$roomid/$membership?access_token=$token";
+    url = url.replace("$token", accountInfo.access_token);
+    url = url.replace("$roomid", encodeURIComponent(roomId));
+    url = url.replace("$membership", membership);
+    
+    var data = {};
+    
+    if (membership === "invite") {
+        data = {
+            user_id: member
+        };
+    }
+
+    $.ajax({
+        url: url,
+        type: "POST",
+        contentType: "application/json; charset=utf-8",
+        data: JSON.stringify(data),
+        dataType: "json",
+        success: function(data) {
+            getCurrentRoomList();
+        },
+        error: function(err) {
+            alert(JSON.stringify($.parseJSON(err.responseText)));  
+        }
+    }); 
+});
+
+$('.joinAlias').live('click', function() {
+    var roomAlias = $("#roomAlias").val();
+    var url = "http://localhost:8008/_matrix/client/api/v1/join/$roomalias?access_token=$token";
+    url = url.replace("$token", accountInfo.access_token);
+    url = url.replace("$roomalias", encodeURIComponent(roomAlias));
+    $.ajax({
+        url: url,
+        type: "POST",
+        contentType: "application/json; charset=utf-8",
+        data: JSON.stringify({}),
+        dataType: "json",
+        success: function(data) {
+            getCurrentRoomList();
+        },
+        error: function(err) {
+            alert(JSON.stringify($.parseJSON(err.responseText)));  
+        }
+    }); 
+});

nuke-room-from-db.sh

@@ -0,0 +1,24 @@
+#!/bin/bash
+
+## CAUTION:
+## This script will remove (hopefully) all trace of the given room ID from
+## your homeserver.db
+
+## Do not run it lightly.
+
+ROOMID="$1"
+
+sqlite3 homeserver.db <<EOF
+DELETE FROM context_depth WHERE context = '$ROOMID';
+DELETE FROM current_state WHERE context = '$ROOMID';
+DELETE FROM feedback WHERE room_id = '$ROOMID';
+DELETE FROM messages WHERE room_id = '$ROOMID';
+DELETE FROM pdu_backward_extremities WHERE context = '$ROOMID';
+DELETE FROM pdu_edges WHERE context = '$ROOMID';
+DELETE FROM pdu_forward_extremities WHERE context = '$ROOMID';
+DELETE FROM pdus WHERE context = '$ROOMID';
+DELETE FROM room_data WHERE room_id = '$ROOMID';
+DELETE FROM room_memberships WHERE room_id = '$ROOMID';
+DELETE FROM rooms WHERE room_id = '$ROOMID';
+DELETE FROM state_pdus WHERE context = '$ROOMID';
+EOF

scripts/basic.css

@@ -0,0 +1,510 @@
+/*
+ * basic.css
+ * ~~~~~~~~~
+ *
+ * Sphinx stylesheet -- basic theme.
+ *
+ * :copyright: Copyright 2007-2010 by the Sphinx team, see AUTHORS.
+ * :license: BSD, see LICENSE for details.
+ *
+ */
+
+/* -- main layout ----------------------------------------------------------- */
+
+div.clearer {
+    clear: both;
+}
+
+/* -- relbar ---------------------------------------------------------------- */
+
+div.related {
+    width: 100%;
+    font-size: 90%;
+}
+
+div.related h3 {
+    display: none;
+}
+
+div.related ul {
+    margin: 0;
+    padding: 0 0 0 10px;
+    list-style: none;
+}
+
+div.related li {
+    display: inline;
+}
+
+div.related li.right {
+    float: right;
+    margin-right: 5px;
+}
+
+/* -- sidebar --------------------------------------------------------------- */
+
+div.sphinxsidebarwrapper {
+    padding: 10px 5px 0 10px;
+}
+
+div.sphinxsidebar {
+    float: left;
+    width: 230px;
+    margin-left: -100%;
+    font-size: 90%;
+}
+
+div.sphinxsidebar ul {
+    list-style: none;
+}
+
+div.sphinxsidebar ul ul,
+div.sphinxsidebar ul.want-points {
+    margin-left: 20px;
+    list-style: square;
+}
+
+div.sphinxsidebar ul ul {
+    margin-top: 0;
+    margin-bottom: 0;
+}
+
+div.sphinxsidebar form {
+    margin-top: 10px;
+}
+
+div.sphinxsidebar input {
+    border: 1px solid #98dbcc;
+    font-family: sans-serif;
+    font-size: 1em;
+}
+
+img {
+    border: 0;
+}
+
+/* -- search page ----------------------------------------------------------- */
+
+ul.search {
+    margin: 10px 0 0 20px;
+    padding: 0;
+}
+
+ul.search li {
+    padding: 5px 0 5px 20px;
+    background-image: url(file.png);
+    background-repeat: no-repeat;
+    background-position: 0 7px;
+}
+
+ul.search li a {
+    font-weight: bold;
+}
+
+ul.search li div.context {
+    color: #888;
+    margin: 2px 0 0 30px;
+    text-align: left;
+}
+
+ul.keywordmatches li.goodmatch a {
+    font-weight: bold;
+}
+
+/* -- index page ------------------------------------------------------------ */
+
+table.contentstable {
+    width: 90%;
+}
+
+table.contentstable p.biglink {
+    line-height: 150%;
+}
+
+a.biglink {
+    font-size: 1.3em;
+}
+
+span.linkdescr {
+    font-style: italic;
+    padding-top: 5px;
+    font-size: 90%;
+}
+
+/* -- general index --------------------------------------------------------- */
+
+table.indextable {
+    width: 100%;
+}
+
+table.indextable td {
+    text-align: left;
+    vertical-align: top;
+}
+
+table.indextable dl, table.indextable dd {
+    margin-top: 0;
+    margin-bottom: 0;
+}
+
+table.indextable tr.pcap {
+    height: 10px;
+}
+
+table.indextable tr.cap {
+    margin-top: 10px;
+    background-color: #f2f2f2;
+}
+
+img.toggler {
+    margin-right: 3px;
+    margin-top: 3px;
+    cursor: pointer;
+}
+
+div.modindex-jumpbox {
+    border-top: 1px solid #ddd;
+    border-bottom: 1px solid #ddd;
+    margin: 1em 0 1em 0;
+    padding: 0.4em;
+}
+
+div.genindex-jumpbox {
+    border-top: 1px solid #ddd;
+    border-bottom: 1px solid #ddd;
+    margin: 1em 0 1em 0;
+    padding: 0.4em;
+}
+
+/* -- general body styles --------------------------------------------------- */
+
+a.headerlink {
+    visibility: hidden;
+}
+
+h1:hover > a.headerlink,
+h2:hover > a.headerlink,
+h3:hover > a.headerlink,
+h4:hover > a.headerlink,
+h5:hover > a.headerlink,
+h6:hover > a.headerlink,
+dt:hover > a.headerlink {
+    visibility: visible;
+}
+
+div.document p.caption {
+    text-align: inherit;
+}
+
+div.document td {
+    text-align: left;
+}
+
+.field-list ul {
+    padding-left: 1em;
+}
+
+.first {
+    margin-top: 0 !important;
+}
+
+p.rubric {
+    margin-top: 30px;
+    font-weight: bold;
+}
+
+.align-left {
+    text-align: left;
+}
+
+.align-center {
+    clear: both;
+    text-align: center;
+}
+
+.align-right {
+    text-align: right;
+}
+
+/* -- sidebars -------------------------------------------------------------- */
+
+div.sidebar {
+    margin: 0 0 0.5em 1em;
+    border: 1px solid #ddb;
+    padding: 7px 7px 0 7px;
+    background-color: #ffe;
+    width: 40%;
+    float: right;
+}
+
+p.sidebar-title {
+    font-weight: bold;
+}
+
+/* -- topics ---------------------------------------------------------------- */
+
+div.topic {
+    border: 1px solid #ccc;
+    padding: 7px 7px 0 7px;
+    margin: 10px 0 10px 0;
+}
+
+p.topic-title {
+    font-size: 1.1em;
+    font-weight: bold;
+    margin-top: 10px;
+}
+
+/* -- admonitions ----------------------------------------------------------- */
+
+div.admonition {
+    margin-top: 10px;
+    margin-bottom: 10px;
+    padding: 7px;
+}
+
+div.admonition dt {
+    font-weight: bold;
+}
+
+div.admonition dl {
+    margin-bottom: 0;
+}
+
+p.admonition-title {
+    margin: 0px 10px 5px 0px;
+    font-weight: bold;
+}
+
+div.document p.centered {
+    text-align: center;
+    margin-top: 25px;
+}
+
+/* -- tables ---------------------------------------------------------------- */
+
+table.docutils {
+    border: 0;
+    border-collapse: collapse;
+}
+
+table.docutils td, table.docutils th {
+    padding: 1px 8px 1px 5px;
+    border-top: 0;
+    border-left: 0;
+    border-right: 0;
+    border-bottom: 1px solid #aaa;
+}
+
+table.field-list td, table.field-list th {
+    border: 0 !important;
+}
+
+table.footnote td, table.footnote th {
+    border: 0 !important;
+}
+
+th {
+    text-align: left;
+    padding-right: 5px;
+}
+
+table.citation {
+    border-left: solid 1px gray;
+    margin-left: 1px;
+}
+
+table.citation td {
+    border-bottom: none;
+}
+
+/* -- other body styles ----------------------------------------------------- */
+
+ol.arabic {
+    list-style: decimal;
+}
+
+ol.loweralpha {
+    list-style: lower-alpha;
+}
+
+ol.upperalpha {
+    list-style: upper-alpha;
+}
+
+ol.lowerroman {
+    list-style: lower-roman;
+}
+
+ol.upperroman {
+    list-style: upper-roman;
+}
+
+dl {
+    margin-bottom: 15px;
+}
+
+dd p {
+    margin-top: 0px;
+}
+
+dd ul, dd table {
+    margin-bottom: 10px;
+}
+
+dd {
+    margin-top: 3px;
+    margin-bottom: 10px;
+    margin-left: 30px;
+}
+
+dt:target, .highlighted {
+    background-color: #fbe54e;
+}
+
+dl.glossary dt {
+    font-weight: bold;
+    font-size: 1.1em;
+}
+
+.field-list ul {
+    margin: 0;
+    padding-left: 1em;
+}
+
+.field-list p {
+    margin: 0;
+}
+
+.refcount {
+    color: #060;
+}
+
+.optional {
+    font-size: 1.3em;
+}
+
+.versionmodified {
+    font-style: italic;
+}
+
+.system-message {
+    background-color: #fda;
+    padding: 5px;
+    border: 3px solid red;
+}
+
+.footnote:target  {
+    background-color: #ffa
+}
+
+.line-block {
+    display: block;
+    margin-top: 1em;
+    margin-bottom: 1em;
+}
+
+.line-block .line-block {
+    margin-top: 0;
+    margin-bottom: 0;
+    margin-left: 1.5em;
+}
+
+.guilabel, .menuselection {
+    font-family: sans-serif;
+}
+
+.accelerator {
+    text-decoration: underline;
+}
+
+.classifier {
+    font-style: oblique;
+}
+
+/* -- code displays --------------------------------------------------------- */
+
+pre {
+    overflow: auto;
+}
+
+td.linenos pre {
+    padding: 5px 0px;
+    border: 0;
+    background-color: transparent;
+    color: #aaa;
+}
+
+table.highlighttable {
+    margin-left: 0.5em;
+}
+
+table.highlighttable td {
+    padding: 0 0.5em 0 0.5em;
+}
+
+tt.descname {
+    background-color: transparent;
+    font-weight: bold;
+    font-size: 1.2em;
+}
+
+tt.descclassname {
+    background-color: transparent;
+}
+
+tt.xref, a tt {
+    background-color: transparent;
+    font-weight: bold;
+}
+
+h1 tt, h2 tt, h3 tt, h4 tt, h5 tt, h6 tt {
+    background-color: transparent;
+}
+
+.viewcode-link {
+    float: right;
+}
+
+.viewcode-back {
+    float: right;
+    font-family: sans-serif;
+}
+
+div.viewcode-block:target {
+    margin: -1px -10px;
+    padding: 0 10px;
+}
+
+/* -- math display ---------------------------------------------------------- */
+
+img.math {
+    vertical-align: middle;
+}
+
+div.document div.math p {
+    text-align: center;
+}
+
+span.eqno {
+    float: right;
+}
+
+/* -- printout stylesheet --------------------------------------------------- */
+
+@media print {
+    div.document,
+    div.documentwrapper,
+    div.bodywrapper {
+        margin: 0 !important;
+        width: 100%;
+    }
+
+    div.sphinxsidebar,
+    div.related,
+    div.footer,
+    #top-link {
+        display: none;
+    }
+}
+

scripts/copyrighter.pl

@@ -1,5 +1,5 @@
 #!/usr/bin/perl -pi
-# Copyright 2014 matrix.org
+# Copyright 2014 OpenMarket Ltd
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
@@ -14,7 +14,7 @@
 # limitations under the License.
 
 $copyright = <<EOT;
-# Copyright 2014 matrix.org
+# Copyright 2014 OpenMarket Ltd
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.

scripts/gendoc.sh

@@ -0,0 +1,14 @@
+#!/bin/bash
+
+MATRIXDOTORG=$HOME/workspace/matrix.org
+
+rst2html-2.7.py --stylesheet=basic.css,nature.css ../docs/specification.rst > $MATRIXDOTORG/docs/spec/index.html
+rst2html-2.7.py --stylesheet=basic.css,nature.css ../docs/client-server/howto.rst > $MATRIXDOTORG/docs/howtos/client-server.html
+
+perl -pi -e 's#<head>#<head><link rel="stylesheet" href="/site.css">#' $MATRIXDOTORG/docs/spec/index.html $MATRIXDOTORG/docs/howtos/client-server.html
+
+perl -pi -e 's#<body>#<body><div id="header"><div id="headerContent">&nbsp;</div></div><div id="page"><div id="wrapper"><div style="text-align: center; padding: 40px;"><a href="/"><img src="/matrix.png" width="305" height="130" alt="[matrix]"/></a></div>#' $MATRIXDOTORG/docs/spec/index.html $MATRIXDOTORG/docs/howtos/client-server.html
+
+perl -pi -e 's#</body>#</div></div><div id="footer"><div id="footerContent">&copy 2014 Matrix.org</div></div></body>#' $MATRIXDOTORG/docs/spec/index.html $MATRIXDOTORG/docs/howtos/client-server.html
+
+scp -r $MATRIXDOTORG/docs matrix@ldc-prd-matrix-001:/sites/matrix

scripts/nature.css

@@ -0,0 +1,270 @@
+/*
+ * nature.css_t
+ * ~~~~~~~~~~~~
+ *
+ * Sphinx stylesheet -- nature theme.
+ *
+ * :copyright: Copyright 2007-2010 by the Sphinx team, see AUTHORS.
+ * :license: BSD, see LICENSE for details.
+ *
+ */
+ 
+/* -- page layout ----------------------------------------------------------- */
+ 
+body {
+    font-family: Arial, sans-serif;
+    font-size: 100%;
+    /*background-color: #111;*/
+    color: #555;
+    margin: 0;
+    padding: 0;
+}
+
+div.documentwrapper {
+    float: left;
+    width: 100%;
+}
+
+div.bodywrapper {
+    margin: 0 0 0 230px;
+}
+
+hr {
+    border: 1px solid #B1B4B6;
+}
+ 
+/*
+div.document {
+    background-color: #eee;
+}
+*/
+ 
+div.document {
+    background-color: #ffffff;
+    color: #3E4349;
+    padding: 0 30px 30px 30px;
+    font-size: 0.9em;
+}
+ 
+div.footer {
+    color: #555;
+    width: 100%;
+    padding: 13px 0;
+    text-align: center;
+    font-size: 75%;
+}
+ 
+div.footer a {
+    color: #444;
+    text-decoration: underline;
+}
+ 
+div.related {
+    background-color: #6BA81E;
+    line-height: 32px;
+    color: #fff;
+    text-shadow: 0px 1px 0 #444;
+    font-size: 0.9em;
+}
+ 
+div.related a {
+    color: #E2F3CC;
+}
+ 
+div.sphinxsidebar {
+    font-size: 0.75em;
+    line-height: 1.5em;
+}
+
+div.sphinxsidebarwrapper{
+    padding: 20px 0;
+}
+ 
+div.sphinxsidebar h3,
+div.sphinxsidebar h4 {
+    font-family: Arial, sans-serif;
+    color: #222;
+    font-size: 1.2em;
+    font-weight: normal;
+    margin: 0;
+    padding: 5px 10px;
+    background-color: #ddd;
+    text-shadow: 1px 1px 0 white
+}
+
+div.sphinxsidebar h4{
+    font-size: 1.1em;
+}
+ 
+div.sphinxsidebar h3 a {
+    color: #444;
+}
+ 
+ 
+div.sphinxsidebar p {
+    color: #888;
+    padding: 5px 20px;
+}
+ 
+div.sphinxsidebar p.topless {
+}
+ 
+div.sphinxsidebar ul {
+    margin: 10px 20px;
+    padding: 0;
+    color: #000;
+}
+ 
+div.sphinxsidebar a {
+    color: #444;
+}
+ 
+div.sphinxsidebar input {
+    border: 1px solid #ccc;
+    font-family: sans-serif;
+    font-size: 1em;
+}
+
+div.sphinxsidebar input[type=text]{
+    margin-left: 20px;
+}
+ 
+/* -- body styles ----------------------------------------------------------- */
+ 
+a {
+    color: #005B81;
+    text-decoration: none;
+}
+ 
+a:hover {
+    color: #E32E00;
+    text-decoration: underline;
+}
+ 
+div.document h1,
+div.document h2,
+div.document h3,
+div.document h4,
+div.document h5,
+div.document h6 {
+    font-family: Arial, sans-serif;
+    background-color: #BED4EB;
+    font-weight: normal;
+    color: #212224;
+    margin: 30px 0px 10px 0px;
+    padding: 5px 0 5px 10px;
+    text-shadow: 0px 1px 0 white
+}
+ 
+div.document h1 { border-top: 20px solid white; margin-top: 0; font-size: 200%; }
+div.document h2 { font-size: 150%; background-color: #C8D5E3; }
+div.document h3 { font-size: 120%; background-color: #D8DEE3; }
+div.document h4 { font-size: 110%; background-color: #D8DEE3; }
+div.document h5 { font-size: 100%; background-color: #D8DEE3; }
+div.document h6 { font-size: 100%; background-color: #D8DEE3; }
+ 
+a.headerlink {
+    color: #c60f0f;
+    font-size: 0.8em;
+    padding: 0 4px 0 4px;
+    text-decoration: none;
+}
+ 
+a.headerlink:hover {
+    background-color: #c60f0f;
+    color: white;
+}
+ 
+div.document p, div.document dd, div.document li {
+    line-height: 1.5em;
+}
+ 
+div.admonition p.admonition-title + p {
+    display: inline;
+}
+
+div.highlight{
+    background-color: white;
+}
+
+div.note {
+    background-color: #eee;
+    border: 1px solid #ccc;
+}
+ 
+div.seealso {
+    background-color: #ffc;
+    border: 1px solid #ff6;
+}
+ 
+div.topic {
+    background-color: #eee;
+}
+ 
+div.warning {
+    background-color: #ffe4e4;
+    border: 1px solid #f66;
+}
+ 
+p.admonition-title {
+    display: inline;
+}
+ 
+p.admonition-title:after {
+    content: ":";
+}
+ 
+pre {
+    padding: 10px;
+    background-color: White;
+    color: #222;
+    line-height: 1.2em;
+    border: 1px solid #C6C9CB;
+    font-size: 1.1em;
+    margin: 1.5em 0 1.5em 0;
+    -webkit-box-shadow: 1px 1px 1px #d8d8d8;
+    -moz-box-shadow: 1px 1px 1px #d8d8d8;
+}
+ 
+tt {
+    background-color: #ecf0f3;
+    color: #222;
+    /* padding: 1px 2px; */
+    font-size: 1.1em;
+    font-family: monospace;
+}
+
+.viewcode-back {
+    font-family: Arial, sans-serif;
+}
+
+div.viewcode-block:target {
+    background-color: #f4debf;
+    border-top: 1px solid #ac9;
+    border-bottom: 1px solid #ac9;
+}
+
+p {
+	margin: 0;
+}
+
+ul li dd {
+	margin-top: 0;
+}
+
+ul li dl {
+	margin-bottom: 0;
+}
+
+li dl dd {
+	margin-bottom: 0;
+}
+
+dd ul {
+	padding-left: 0;
+}
+
+li dd ul {
+	margin-bottom: 0;
+}
+

setup.py

@@ -1,4 +1,6 @@
-# Copyright 2014 matrix.org
+#!/usr/bin/env python
+
+# Copyright 2014 OpenMarket Ltd
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
@@ -25,13 +27,14 @@ def read(fname):
 
 setup(
     name="SynapseHomeServer",
-    version="0.1",
+    version="0.0.1",
     packages=find_packages(exclude=["tests"]),
     description="Reference Synapse Home Server",
     install_requires=[
         "syutil==0.0.1",
         "Twisted>=14.0.0",
         "service_identity>=1.0.0",
+        "pyyaml",
         "pyasn1",
         "pynacl",
         "daemonize",

synapse/__init__.py

@@ -1,5 +1,5 @@
 # -*- coding: utf-8 -*-
-# Copyright 2014 matrix.org
+# Copyright 2014 OpenMarket Ltd
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
@@ -15,3 +15,5 @@
 
 """ This is a reference implementation of a synapse home server.
 """
+
+__version__ = "0.3.3"

synapse/api/__init__.py

@@ -1,5 +1,5 @@
 # -*- coding: utf-8 -*-
-# Copyright 2014 matrix.org
+# Copyright 2014 OpenMarket Ltd
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.

synapse/api/auth.py

@@ -1,5 +1,5 @@
 # -*- coding: utf-8 -*-
-# Copyright 2014 matrix.org
+# Copyright 2014 OpenMarket Ltd
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
@@ -17,10 +17,10 @@
 
 from twisted.internet import defer
 
-from synapse.api.constants import Membership
-from synapse.api.errors import AuthError, StoreError
-from synapse.api.events.room import (RoomTopicEvent, RoomMemberEvent,
-                                     MessageEvent, FeedbackEvent)
+from synapse.api.constants import Membership, JoinRules
+from synapse.api.errors import AuthError, StoreError, Codes, SynapseError
+from synapse.api.events.room import RoomMemberEvent, RoomPowerLevelsEvent
+from synapse.util.logutils import log_function
 
 import logging
 
@@ -34,7 +34,7 @@ class Auth(object):
         self.store = hs.get_datastore()
 
     @defer.inlineCallbacks
-    def check(self, event, raises=False):
+    def check(self, event, snapshot, raises=False):
         """ Checks if this event is correctly authed.
 
         Returns:
@@ -44,15 +44,35 @@ class Auth(object):
             be raised only if raises=True.
         """
         try:
-            if event.type in [RoomTopicEvent.TYPE, MessageEvent.TYPE,
-                              FeedbackEvent.TYPE]:
-                yield self.check_joined_room(event.room_id, event.user_id)
+            if hasattr(event, "room_id"):
+                is_state = hasattr(event, "state_key")
+
+                if event.type == RoomMemberEvent.TYPE:
+                    yield self._can_replace_state(event)
+                    allowed = yield self.is_membership_change_allowed(event)
+                    defer.returnValue(allowed)
+                    return
+
+                self._check_joined_room(
+                    member=snapshot.membership_state,
+                    user_id=snapshot.user_id,
+                    room_id=snapshot.room_id,
+                )
+
+                if is_state:
+                    # TODO (erikj): This really only should be called for *new*
+                    # state
+                    yield self._can_add_state(event)
+                    yield self._can_replace_state(event)
+                else:
+                    yield self._can_send_event(event)
+
+                if event.type == RoomPowerLevelsEvent.TYPE:
+                    yield self._check_power_levels(event)
+
                 defer.returnValue(True)
-            elif event.type == RoomMemberEvent.TYPE:
-                allowed = yield self.is_membership_change_allowed(event)
-                defer.returnValue(allowed)
             else:
-                raise AuthError(500, "Unknown event type %s" % event.type)
+                raise AuthError(500, "Unknown event: %s" % event)
         except AuthError as e:
             logger.info("Event auth check failed on event %s with msg: %s",
                         event, e.msg)
@@ -67,16 +87,22 @@ class Auth(object):
                 room_id=room_id,
                 user_id=user_id
             )
-            if not member or member.membership != Membership.JOIN:
-                raise AuthError(403, "User %s not in room %s" %
-                                (user_id, room_id))
+            self._check_joined_room(member, user_id, room_id)
             defer.returnValue(member)
         except AttributeError:
             pass
         defer.returnValue(None)
 
+    def _check_joined_room(self, member, user_id, room_id):
+        if not member or member.membership != Membership.JOIN:
+            raise AuthError(403, "User %s not in room %s (%s)" % (
+                user_id, room_id, repr(member)
+            ))
+
     @defer.inlineCallbacks
     def is_membership_change_allowed(self, event):
+        target_user_id = event.state_key
+
         # does this room even exist
         room = yield self.store.get_room(event.room_id)
         if not room:
@@ -94,7 +120,7 @@ class Auth(object):
         # get info about the target
         try:
             target = yield self.store.get_room_member(
-                user_id=event.target_user_id,
+                user_id=target_user_id,
                 room_id=event.room_id)
         except:
             target = None
@@ -102,31 +128,74 @@ class Auth(object):
 
         membership = event.content["membership"]
 
+        join_rule = yield self.store.get_room_join_rule(event.room_id)
+        if not join_rule:
+            join_rule = JoinRules.INVITE
+
         if Membership.INVITE == membership:
+            # TODO (erikj): We should probably handle this more intelligently
+            # PRIVATE join rules.
+
             # Invites are valid iff caller is in the room and target isn't.
             if not caller_in_room:  # caller isn't joined
                 raise AuthError(403, "You are not in room %s." % event.room_id)
             elif target_in_room:  # the target is already in the room.
                 raise AuthError(403, "%s is already in the room." %
-                                     event.target_user_id)
+                                     target_user_id)
         elif Membership.JOIN == membership:
             # Joins are valid iff caller == target and they were:
             # invited: They are accepting the invitation
             # joined: It's a NOOP
-            if event.user_id != event.target_user_id:
+            if event.user_id != target_user_id:
                 raise AuthError(403, "Cannot force another user to join.")
-            elif room.is_public:
-                pass  # anyone can join public rooms.
-            elif (not caller or caller.membership not in
-                    [Membership.INVITE, Membership.JOIN]):
-                raise AuthError(403, "You are not invited to this room.")
+            elif join_rule == JoinRules.PUBLIC or room.is_public:
+                pass
+            elif join_rule == JoinRules.INVITE:
+                if (
+                    not caller or caller.membership not in
+                    [Membership.INVITE, Membership.JOIN]
+                ):
+                    raise AuthError(403, "You are not invited to this room.")
+            else:
+                # TODO (erikj): may_join list
+                # TODO (erikj): private rooms
+                raise AuthError(403, "You are not allowed to join this room")
         elif Membership.LEAVE == membership:
+            # TODO (erikj): Implement kicks.
+
             if not caller_in_room:  # trying to leave a room you aren't joined
                 raise AuthError(403, "You are not in room %s." % event.room_id)
-            elif event.target_user_id != event.user_id:
-                # trying to force another user to leave
-                raise AuthError(403, "Cannot force %s to leave." %
-                                event.target_user_id)
+            elif target_user_id != event.user_id:
+                user_level = yield self.store.get_power_level(
+                    event.room_id,
+                    event.user_id,
+                )
+                _, kick_level = yield self.store.get_ops_levels(event.room_id)
+
+                if kick_level:
+                    kick_level = int(kick_level)
+                else:
+                    kick_level = 50
+
+                if user_level < kick_level:
+                    raise AuthError(
+                        403, "You cannot kick user %s." % target_user_id
+                    )
+        elif Membership.BAN == membership:
+            user_level = yield self.store.get_power_level(
+                event.room_id,
+                event.user_id,
+            )
+
+            ban_level, _ = yield self.store.get_ops_levels(event.room_id)
+
+            if ban_level:
+                ban_level = int(ban_level)
+            else:
+                ban_level = 50  # FIXME (erikj): What should we do here?
+
+            if user_level < ban_level:
+                raise AuthError(403, "You don't have permission to ban")
         else:
             raise AuthError(500, "Unknown membership %s" % membership)
 
@@ -161,6 +230,191 @@ class Auth(object):
         """
         try:
             user_id = yield self.store.get_user_by_token(token=token)
+            if not user_id:
+                raise StoreError()
             defer.returnValue(self.hs.parse_userid(user_id))
         except StoreError:
-            raise AuthError(403, "Unrecognised access token.")
+            raise AuthError(403, "Unrecognised access token.",
+                            errcode=Codes.UNKNOWN_TOKEN)
+
+    @defer.inlineCallbacks
+    @log_function
+    def _can_send_event(self, event):
+        send_level = yield self.store.get_send_event_level(event.room_id)
+
+        if send_level:
+            send_level = int(send_level)
+        else:
+            send_level = 0
+
+        user_level = yield self.store.get_power_level(
+            event.room_id,
+            event.user_id,
+        )
+
+        if user_level:
+            user_level = int(user_level)
+        else:
+            user_level = 0
+
+        if user_level < send_level:
+            raise AuthError(
+                403, "You don't have permission to post to the room"
+            )
+
+        defer.returnValue(True)
+
+    @defer.inlineCallbacks
+    def _can_add_state(self, event):
+        add_level = yield self.store.get_add_state_level(event.room_id)
+
+        if not add_level:
+            defer.returnValue(True)
+
+        add_level = int(add_level)
+
+        user_level = yield self.store.get_power_level(
+            event.room_id,
+            event.user_id,
+        )
+
+        user_level = int(user_level)
+
+        if user_level < add_level:
+            raise AuthError(
+                403, "You don't have permission to add state to the room"
+            )
+
+        defer.returnValue(True)
+
+    @defer.inlineCallbacks
+    def _can_replace_state(self, event):
+        current_state = yield self.store.get_current_state(
+            event.room_id,
+            event.type,
+            event.state_key,
+        )
+
+        if current_state:
+            current_state = current_state[0]
+
+        user_level = yield self.store.get_power_level(
+            event.room_id,
+            event.user_id,
+        )
+
+        if user_level:
+            user_level = int(user_level)
+        else:
+            user_level = 0
+
+        logger.debug(
+            "Checking power level for %s, %s", event.user_id, user_level
+        )
+        if current_state and hasattr(current_state, "required_power_level"):
+            req = current_state.required_power_level
+
+            logger.debug("Checked power level for %s, %s", event.user_id, req)
+            if user_level < req:
+                raise AuthError(
+                    403,
+                    "You don't have permission to change that state"
+                )
+
+    @defer.inlineCallbacks
+    def _check_power_levels(self, event):
+        for k, v in event.content.items():
+            if k == "default":
+                continue
+
+            # FIXME (erikj): We don't want hsob_Ts in content.
+            if k == "hsob_ts":
+                continue
+
+            try:
+                self.hs.parse_userid(k)
+            except:
+                raise SynapseError(400, "Not a valid user_id: %s" % (k,))
+
+            try:
+                int(v)
+            except:
+                raise SynapseError(400, "Not a valid power level: %s" % (v,))
+
+        current_state = yield self.store.get_current_state(
+            event.room_id,
+            event.type,
+            event.state_key,
+        )
+
+        if not current_state:
+            return
+        else:
+            current_state = current_state[0]
+
+        user_level = yield self.store.get_power_level(
+            event.room_id,
+            event.user_id,
+        )
+
+        if user_level:
+            user_level = int(user_level)
+        else:
+            user_level = 0
+
+        old_list = current_state.content
+
+        # FIXME (erikj)
+        old_people = {k: v for k, v in old_list.items() if k.startswith("@")}
+        new_people = {
+            k: v for k, v in event.content.items()
+            if k.startswith("@")
+        }
+
+        removed = set(old_people.keys()) - set(new_people.keys())
+        added = set(old_people.keys()) - set(new_people.keys())
+        same = set(old_people.keys()) & set(new_people.keys())
+
+        for r in removed:
+            if int(old_list.content[r]) > user_level:
+                raise AuthError(
+                    403,
+                    "You don't have permission to remove user: %s" % (r, )
+                )
+
+        for n in added:
+            if int(event.content[n]) > user_level:
+                raise AuthError(
+                    403,
+                    "You don't have permission to add ops level greater "
+                    "than your own"
+                )
+
+        for s in same:
+            if int(event.content[s]) != int(old_list[s]):
+                if int(event.content[s]) > user_level:
+                    raise AuthError(
+                        403,
+                        "You don't have permission to add ops level greater "
+                        "than your own"
+                    )
+
+        if "default" in old_list:
+            old_default = int(old_list["default"])
+
+            if old_default > user_level:
+                raise AuthError(
+                    403,
+                    "You don't have permission to add ops level greater than "
+                    "your own"
+                )
+
+            if "default" in event.content:
+                new_default = int(event.content["default"])
+
+                if new_default > user_level:
+                    raise AuthError(
+                        403,
+                        "You don't have permission to add ops level greater "
+                        "than your own"
+                    )

synapse/api/constants.py

@@ -1,5 +1,5 @@
 # -*- coding: utf-8 -*-
-# Copyright 2014 matrix.org
+# Copyright 2014 OpenMarket Ltd
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
@@ -23,6 +23,8 @@ class Membership(object):
     JOIN = u"join"
     KNOCK = u"knock"
     LEAVE = u"leave"
+    BAN = u"ban"
+    LIST = (INVITE, JOIN, KNOCK, LEAVE, BAN)
 
 
 class Feedback(object):
@@ -30,8 +32,8 @@ class Feedback(object):
     """Represents the types of feedback a user can send in response to a
     message."""
 
-    DELIVERED = u"d"
-    READ = u"r"
+    DELIVERED = u"delivered"
+    READ = u"read"
     LIST = (DELIVERED, READ)
 
 
@@ -41,3 +43,19 @@ class PresenceState(object):
     UNAVAILABLE = u"unavailable"
     ONLINE = u"online"
     FREE_FOR_CHAT = u"free_for_chat"
+
+
+class JoinRules(object):
+    PUBLIC = u"public"
+    KNOCK = u"knock"
+    INVITE = u"invite"
+    PRIVATE = u"private"
+
+
+class LoginType(object):
+    PASSWORD = u"m.login.password"
+    OAUTH = u"m.login.oauth2"
+    EMAIL_CODE = u"m.login.email.code"
+    EMAIL_URL = u"m.login.email.url"
+    EMAIL_IDENTITY = u"m.login.email.identity"
+    RECAPTCHA = u"m.login.recaptcha"

synapse/api/errors.py

@@ -1,5 +1,5 @@
 # -*- coding: utf-8 -*-
-# Copyright 2014 matrix.org
+# Copyright 2014 OpenMarket Ltd
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
@@ -27,6 +27,10 @@ class Codes(object):
     BAD_PAGINATION = "M_BAD_PAGINATION"
     UNKNOWN = "M_UNKNOWN"
     NOT_FOUND = "M_NOT_FOUND"
+    UNKNOWN_TOKEN = "M_UNKNOWN_TOKEN"
+    LIMIT_EXCEEDED = "M_LIMIT_EXCEEDED"
+    CAPTCHA_NEEDED = "M_CAPTCHA_NEEDED"
+    CAPTCHA_INVALID = "M_CAPTCHA_INVALID"
 
 
 class CodeMessageException(Exception):
@@ -37,11 +41,15 @@ class CodeMessageException(Exception):
         super(CodeMessageException, self).__init__("%d: %s" % (code, msg))
         self.code = code
         self.msg = msg
+        self.response_code_message = None
+
+    def error_dict(self):
+        return cs_error(self.msg)
 
 
 class SynapseError(CodeMessageException):
     """A base error which can be caught for all synapse events."""
-    def __init__(self, code, msg, errcode=""):
+    def __init__(self, code, msg, errcode=Codes.UNKNOWN):
         """Constructs a synapse error.
 
         Args:
@@ -52,6 +60,11 @@ class SynapseError(CodeMessageException):
         super(SynapseError, self).__init__(code, msg)
         self.errcode = errcode
 
+    def error_dict(self):
+        return cs_error(
+            self.msg,
+            self.errcode,
+        )
 
 class RoomError(SynapseError):
     """An error raised when a room event fails."""
@@ -74,7 +87,10 @@ class AuthError(SynapseError):
 
 class EventStreamError(SynapseError):
     """An error raised when there a problem with the event stream."""
-    pass
+    def __init__(self, *args, **kwargs):
+        if "errcode" not in kwargs:
+            kwargs["errcode"] = Codes.BAD_PAGINATION
+        super(EventStreamError, self).__init__(*args, **kwargs)
 
 
 class LoginError(SynapseError):
@@ -87,13 +103,39 @@ class StoreError(SynapseError):
     pass
 
 
-def cs_exception(exception):
-    if isinstance(exception, SynapseError):
+class InvalidCaptchaError(SynapseError):
+    def __init__(self, code=400, msg="Invalid captcha.", error_url=None,
+                 errcode=Codes.CAPTCHA_INVALID):
+        super(InvalidCaptchaError, self).__init__(code, msg, errcode)
+        self.error_url = error_url
+
+    def error_dict(self):
         return cs_error(
-            exception.msg,
-            Codes.UNKNOWN if not exception.errcode else exception.errcode)
-    elif isinstance(exception, CodeMessageException):
-        return cs_error(exception.msg)
+            self.msg,
+            self.errcode,
+            error_url=self.error_url,
+        )
+
+class LimitExceededError(SynapseError):
+    """A client has sent too many requests and is being throttled.
+    """
+    def __init__(self, code=429, msg="Too Many Requests", retry_after_ms=None,
+                 errcode=Codes.LIMIT_EXCEEDED):
+        super(LimitExceededError, self).__init__(code, msg, errcode)
+        self.retry_after_ms = retry_after_ms
+        self.response_code_message = "Too Many Requests"
+
+    def error_dict(self):
+        return cs_error(
+            self.msg,
+            self.errcode,
+            retry_after_ms=self.retry_after_ms,
+        )
+
+
+def cs_exception(exception):
+    if isinstance(exception, CodeMessageException):
+        return exception.error_dict()
     else:
         logging.error("Unknown exception type: %s", type(exception))
 

synapse/api/events/__init__.py

@@ -1,5 +1,5 @@
 # -*- coding: utf-8 -*-
-# Copyright 2014 matrix.org
+# Copyright 2014 OpenMarket Ltd
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
@@ -17,6 +17,19 @@ from synapse.api.errors import SynapseError, Codes
 from synapse.util.jsonobject import JsonEncodedObject
 
 
+def serialize_event(hs, e):
+    # FIXME(erikj): To handle the case of presence events and the like
+    if not isinstance(e, SynapseEvent):
+        return e
+
+    d = e.get_dict()
+    if "age_ts" in d:
+        d["age"] = int(hs.get_clock().time_msec()) - d["age_ts"]
+        del d["age_ts"]
+
+    return d
+
+
 class SynapseEvent(JsonEncodedObject):
 
     """Base class for Synapse events. These are JSON objects which must abide
@@ -41,16 +54,21 @@ class SynapseEvent(JsonEncodedObject):
         "room_id",
         "user_id",  # sender/initiator
         "content",  # HTTP body, JSON
+        "state_key",
+        "required_power_level",
+        "age_ts",
+        "prev_content",
     ]
 
     internal_keys = [
         "is_state",
-        "state_key",
         "prev_events",
         "prev_state",
         "depth",
         "destinations",
         "origin",
+        "outlier",
+        "power_level",
     ]
 
     required_keys = [
@@ -138,7 +156,8 @@ class SynapseEvent(JsonEncodedObject):
                     return "Missing %s key" % key
 
                 if type(content[key]) != type(template[key]):
-                    return "Key %s is of the wrong type." % key
+                    return "Key %s is of the wrong type (got %s, want %s)" % (
+                        key, type(content[key]), type(template[key]))
 
                 if type(content[key]) == dict:
                     # we must go deeper
@@ -151,3 +170,11 @@ class SynapseEvent(JsonEncodedObject):
                         msg = self._check_json(entry, template[key][0])
                         if msg:
                             return msg
+
+
+class SynapseStateEvent(SynapseEvent):
+
+    def __init__(self, **kwargs):
+        if "state_key" not in kwargs:
+            kwargs["state_key"] = ""
+        super(SynapseStateEvent, self).__init__(**kwargs)

synapse/api/events/factory.py

@@ -1,5 +1,5 @@
 # -*- coding: utf-8 -*-
-# Copyright 2014 matrix.org
+# Copyright 2014 OpenMarket Ltd
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
@@ -15,7 +15,9 @@
 
 from synapse.api.events.room import (
     RoomTopicEvent, MessageEvent, RoomMemberEvent, FeedbackEvent,
-    InviteJoinEvent, RoomConfigEvent
+    InviteJoinEvent, RoomConfigEvent, RoomNameEvent, GenericEvent,
+    RoomPowerLevelsEvent, RoomJoinRulesEvent, RoomOpsPowerLevelsEvent,
+    RoomCreateEvent, RoomAddStateLevelEvent, RoomSendEventLevelEvent
 )
 
 from synapse.util.stringutils import random_string
@@ -25,27 +27,49 @@ class EventFactory(object):
 
     _event_classes = [
         RoomTopicEvent,
+        RoomNameEvent,
         MessageEvent,
         RoomMemberEvent,
         FeedbackEvent,
         InviteJoinEvent,
-        RoomConfigEvent
+        RoomConfigEvent,
+        RoomPowerLevelsEvent,
+        RoomJoinRulesEvent,
+        RoomCreateEvent,
+        RoomAddStateLevelEvent,
+        RoomSendEventLevelEvent,
+        RoomOpsPowerLevelsEvent,
     ]
 
-    def __init__(self):
+    def __init__(self, hs):
         self._event_list = {}  # dict of TYPE to event class
         for event_class in EventFactory._event_classes:
             self._event_list[event_class.TYPE] = event_class
 
+        self.clock = hs.get_clock()
+        self.hs = hs
+
     def create_event(self, etype=None, **kwargs):
         kwargs["type"] = etype
         if "event_id" not in kwargs:
-            kwargs["event_id"] = random_string(10)
+            kwargs["event_id"] = "%s@%s" % (
+                random_string(10), self.hs.hostname
+            )
 
-        try:
+        if "ts" not in kwargs:
+            kwargs["ts"] = int(self.clock.time_msec())
+
+        # The "age" key is a delta timestamp that should be converted into an
+        # absolute timestamp the minute we see it.
+        if "age" in kwargs:
+            kwargs["age_ts"] = int(self.clock.time_msec()) - int(kwargs["age"])
+            del kwargs["age"]
+        elif "age_ts" not in kwargs:
+            kwargs["age_ts"] = int(self.clock.time_msec())
+
+        if etype in self._event_list:
             handler = self._event_list[etype]
-        except KeyError:  # unknown event type
-            # TODO allow custom event types.
-            raise NotImplementedError("Unknown etype=%s" % etype)
+        else:
+            handler = GenericEvent
 
         return handler(**kwargs)

synapse/api/events/room.py

@@ -1,5 +1,5 @@
 # -*- coding: utf-8 -*-
-# Copyright 2014 matrix.org
+# Copyright 2014 OpenMarket Ltd
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
@@ -13,31 +13,63 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-from . import SynapseEvent
+from synapse.api.constants import Feedback, Membership
+from synapse.api.errors import SynapseError
+from . import SynapseEvent, SynapseStateEvent
+
+
+class GenericEvent(SynapseEvent):
+    def get_content_template(self):
+        return {}
 
 
 class RoomTopicEvent(SynapseEvent):
     TYPE = "m.room.topic"
 
+    internal_keys = SynapseEvent.internal_keys + [
+        "topic",
+    ]
+
     def __init__(self, **kwargs):
         kwargs["state_key"] = ""
+        if "topic" in kwargs["content"]:
+            kwargs["topic"] = kwargs["content"]["topic"]
         super(RoomTopicEvent, self).__init__(**kwargs)
 
     def get_content_template(self):
         return {"topic": u"string"}
 
 
+class RoomNameEvent(SynapseEvent):
+    TYPE = "m.room.name"
+
+    internal_keys = SynapseEvent.internal_keys + [
+        "name",
+    ]
+
+    def __init__(self, **kwargs):
+        kwargs["state_key"] = ""
+        if "name" in kwargs["content"]:
+            kwargs["name"] = kwargs["content"]["name"]
+        super(RoomNameEvent, self).__init__(**kwargs)
+
+    def get_content_template(self):
+        return {"name": u"string"}
+
+
 class RoomMemberEvent(SynapseEvent):
     TYPE = "m.room.member"
 
     valid_keys = SynapseEvent.valid_keys + [
-        "target_user_id",  # target
+        # target is the state_key
         "membership",  # action
     ]
 
     def __init__(self, **kwargs):
-        if "target_user_id" in kwargs:
-            kwargs["state_key"] = kwargs["target_user_id"]
+        if "membership" not in kwargs:
+            kwargs["membership"] = kwargs.get("content", {}).get("membership")
+        if not kwargs["membership"] in Membership.LIST:
+            raise SynapseError(400, "Bad membership value.")
         super(RoomMemberEvent, self).__init__(**kwargs)
 
     def get_content_template(self):
@@ -61,24 +93,25 @@ class MessageEvent(SynapseEvent):
 class FeedbackEvent(SynapseEvent):
     TYPE = "m.room.message.feedback"
 
-    valid_keys = SynapseEvent.valid_keys + [
-        "msg_id",  # the message ID being acknowledged
-        "msg_sender_id",  # person who is sending the feedback is 'user_id'
-        "feedback_type",  # the type of feedback (delivery, read, etc)
-    ]
+    valid_keys = SynapseEvent.valid_keys
 
     def __init__(self, **kwargs):
         super(FeedbackEvent, self).__init__(**kwargs)
+        if not kwargs["content"]["type"] in Feedback.LIST:
+            raise SynapseError(400, "Bad feedback value.")
 
     def get_content_template(self):
-        return {}
+        return {
+            "type": u"string",
+            "target_event_id": u"string"
+        }
 
 
 class InviteJoinEvent(SynapseEvent):
     TYPE = "m.room.invite_join"
 
     valid_keys = SynapseEvent.valid_keys + [
-        "target_user_id",
+        # target_user_id is the state_key
         "target_host",
     ]
 
@@ -98,3 +131,52 @@ class RoomConfigEvent(SynapseEvent):
 
     def get_content_template(self):
         return {}
+
+
+class RoomCreateEvent(SynapseStateEvent):
+    TYPE = "m.room.create"
+
+    def get_content_template(self):
+        return {}
+
+
+class RoomJoinRulesEvent(SynapseStateEvent):
+    TYPE = "m.room.join_rules"
+
+    def get_content_template(self):
+        return {}
+
+
+class RoomPowerLevelsEvent(SynapseStateEvent):
+    TYPE = "m.room.power_levels"
+
+    def get_content_template(self):
+        return {}
+
+
+class RoomAddStateLevelEvent(SynapseStateEvent):
+    TYPE = "m.room.add_state_level"
+
+    def get_content_template(self):
+        return {}
+
+
+class RoomSendEventLevelEvent(SynapseStateEvent):
+    TYPE = "m.room.send_event_level"
+
+    def get_content_template(self):
+        return {}
+
+
+class RoomOpsPowerLevelsEvent(SynapseStateEvent):
+    TYPE = "m.room.ops_levels"
+
+    def get_content_template(self):
+        return {}
+
+
+class RoomAliasesEvent(SynapseStateEvent):
+    TYPE = "m.room.aliases"
+
+    def get_content_template(self):
+        return {}

synapse/api/notifier.py

@@ -1,191 +0,0 @@
-# -*- coding: utf-8 -*-
-# Copyright 2014 matrix.org
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-from synapse.api.constants import Membership
-from synapse.api.events.room import RoomMemberEvent
-
-from twisted.internet import defer
-from twisted.internet import reactor
-
-import logging
-
-logger = logging.getLogger(__name__)
-
-
-class Notifier(object):
-
-    def __init__(self, hs):
-        self.store = hs.get_datastore()
-        self.hs = hs
-        self.stored_event_listeners = {}
-
-    @defer.inlineCallbacks
-    def on_new_room_event(self, event, store_id):
-        """Called when there is a new room event which may potentially be sent
-        down listening users' event streams.
-
-        This function looks for interested *users* who may want to be notified
-        for this event. This is different to users requesting from the event
-        stream which looks for interested *events* for this user.
-
-        Args:
-            event (SynapseEvent): The new event, which must have a room_id
-            store_id (int): The ID of this event after it was stored with the
-            data store.
-        '"""
-        member_list = yield self.store.get_room_members(room_id=event.room_id,
-                                                        membership="join")
-        if not member_list:
-            member_list = []
-
-        member_list = [u.user_id for u in member_list]
-
-        # invites MUST prod the person being invited, who won't be in the room.
-        if (event.type == RoomMemberEvent.TYPE and
-                event.content["membership"] == Membership.INVITE):
-            member_list.append(event.target_user_id)
-
-        for user_id in member_list:
-            if user_id in self.stored_event_listeners:
-                self._notify_and_callback(
-                    user_id=user_id,
-                    event_data=event.get_dict(),
-                    stream_type=event.type,
-                    store_id=store_id)
-
-    def on_new_user_event(self, user_id, event_data, stream_type, store_id):
-        if user_id in self.stored_event_listeners:
-            self._notify_and_callback(
-                user_id=user_id,
-                event_data=event_data,
-                stream_type=stream_type,
-                store_id=store_id
-            )
-
-    def _notify_and_callback(self, user_id, event_data, stream_type, store_id):
-        logger.debug(
-            "Notifying %s of a new event.",
-            user_id
-        )
-
-        stream_ids = list(self.stored_event_listeners[user_id])
-        for stream_id in stream_ids:
-            self._notify_and_callback_stream(user_id, stream_id, event_data,
-                                             stream_type, store_id)
-
-        if not self.stored_event_listeners[user_id]:
-            del self.stored_event_listeners[user_id]
-
-    def _notify_and_callback_stream(self, user_id, stream_id, event_data,
-                                    stream_type, store_id):
-
-        event_listener = self.stored_event_listeners[user_id].pop(stream_id)
-        return_event_object = {
-            k: event_listener[k] for k in ["start", "chunk", "end"]
-        }
-
-        # work out the new end token
-        token = event_listener["start"]
-        end = self._next_token(stream_type, store_id, token)
-        return_event_object["end"] = end
-
-        # add the event to the chunk
-        chunk = event_listener["chunk"]
-        chunk.append(event_data)
-
-        # callback the defer. We know this can't have been resolved before as
-        # we always remove the event_listener from the map before resolving.
-        event_listener["defer"].callback(return_event_object)
-
-    def _next_token(self, stream_type, store_id, current_token):
-        stream_handler = self.hs.get_handlers().event_stream_handler
-        return stream_handler.get_event_stream_token(
-            stream_type,
-            store_id,
-            current_token
-        )
-
-    def store_events_for(self, user_id=None, stream_id=None, from_tok=None):
-        """Store all incoming events for this user. This should be paired with
-        get_events_for to return chunked data.
-
-        Args:
-            user_id (str): The user to monitor incoming events for.
-            stream (object): The stream that is receiving events
-            from_tok (str): The token to monitor incoming events from.
-        """
-        event_listener = {
-            "start": from_tok,
-            "chunk": [],
-            "end": from_tok,
-            "defer": defer.Deferred(),
-        }
-
-        if user_id not in self.stored_event_listeners:
-            self.stored_event_listeners[user_id] = {stream_id: event_listener}
-        else:
-            self.stored_event_listeners[user_id][stream_id] = event_listener
-
-    def purge_events_for(self, user_id=None, stream_id=None):
-        """Purges any stored events for this user.
-
-        Args:
-            user_id (str): The user to purge stored events for.
-        """
-        try:
-            del self.stored_event_listeners[user_id][stream_id]
-            if not self.stored_event_listeners[user_id]:
-                del self.stored_event_listeners[user_id]
-        except KeyError:
-            pass
-
-    def get_events_for(self, user_id=None, stream_id=None, timeout=0):
-        """Retrieve stored events for this user, waiting if necessary.
-
-        It is advisable to wrap this call in a maybeDeferred.
-
-        Args:
-            user_id (str): The user to get events for.
-            timeout (int): The time in seconds to wait before giving up.
-        Returns:
-            A Deferred or a dict containing the chunk data, depending on if
-            there was data to return yet. The Deferred callback may be None if
-            there were no events before the timeout expired.
-        """
-        logger.debug("%s is listening for events.", user_id)
-
-        try:
-            streams = self.stored_event_listeners[user_id][stream_id]["chunk"]
-            if streams:
-                logger.debug("%s returning existing chunk.", user_id)
-                return streams
-        except KeyError:
-            return None
-
-        reactor.callLater(
-            (timeout / 1000.0), self._timeout, user_id, stream_id
-        )
-        return self.stored_event_listeners[user_id][stream_id]["defer"]
-
-    def _timeout(self, user_id, stream_id):
-        try:
-            # We remove the event_listener from the map so that we can't
-            # resolve the deferred twice.
-            event_listeners = self.stored_event_listeners[user_id]
-            event_listener = event_listeners.pop(stream_id)
-            event_listener["defer"].callback(None)
-            logger.debug("%s event listening timed out.", user_id)
-        except KeyError:
-            pass

synapse/api/ratelimiting.py

@@ -0,0 +1,79 @@
+# Copyright 2014 OpenMarket Ltd
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import collections
+
+
+class Ratelimiter(object):
+    """
+    Ratelimit message sending by user.
+    """
+
+    def __init__(self):
+        self.message_counts = collections.OrderedDict()
+
+    def send_message(self, user_id, time_now_s, msg_rate_hz, burst_count):
+        """Can the user send a message?
+        Args:
+            user_id: The user sending a message.
+            time_now_s: The time now.
+            msg_rate_hz: The long term number of messages a user can send in a
+                second.
+            burst_count: How many messages the user can send before being
+                limited.
+        Returns:
+            A pair of a bool indicating if they can send a message now and a
+                time in seconds of when they can next send a message.
+        """
+        self.prune_message_counts(time_now_s)
+        message_count, time_start, _ignored = self.message_counts.pop(
+            user_id, (0., time_now_s, None),
+        )
+        time_delta = time_now_s - time_start
+        sent_count = message_count - time_delta * msg_rate_hz
+        if sent_count < 0:
+            allowed = True
+            time_start = time_now_s
+            message_count = 1.
+        elif sent_count > burst_count - 1.:
+            allowed = False
+        else:
+            allowed = True
+            message_count += 1
+
+        self.message_counts[user_id] = (
+            message_count, time_start, msg_rate_hz
+        )
+
+        if msg_rate_hz > 0:
+            time_allowed = (
+                time_start + (message_count - burst_count + 1) / msg_rate_hz
+            )
+            if time_allowed < time_now_s:
+                time_allowed = time_now_s
+        else:
+            time_allowed = -1
+
+        return allowed, time_allowed
+
+    def prune_message_counts(self, time_now_s):
+        for user_id in self.message_counts.keys():
+            message_count, time_start, msg_rate_hz = (
+                self.message_counts[user_id]
+            )
+            time_delta = time_now_s - time_start
+            if message_count - time_delta * msg_rate_hz > 0:
+                break
+            else:
+                del self.message_counts[user_id]

synapse/api/streams/__init__.py

@@ -1,97 +0,0 @@
-# -*- coding: utf-8 -*-
-# Copyright 2014 matrix.org
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-from synapse.api.errors import SynapseError
-
-
-class PaginationConfig(object):
-
-    """A configuration object which stores pagination parameters."""
-
-    def __init__(self, from_tok=None, to_tok=None, limit=0):
-        self.from_tok = from_tok
-        self.to_tok = to_tok
-        self.limit = limit
-
-    @classmethod
-    def from_request(cls, request, raise_invalid_params=True):
-        params = {
-            "from_tok": PaginationStream.TOK_START,
-            "to_tok": PaginationStream.TOK_END,
-            "limit": 0
-        }
-
-        query_param_mappings = [  # 3-tuple of qp_key, attribute, rules
-            ("from", "from_tok", lambda x: type(x) == str),
-            ("to", "to_tok", lambda x: type(x) == str),
-            ("limit", "limit", lambda x: x.isdigit())
-        ]
-
-        for qp, attr, is_valid in query_param_mappings:
-            if qp in request.args:
-                if is_valid(request.args[qp][0]):
-                    params[attr] = request.args[qp][0]
-                elif raise_invalid_params:
-                    raise SynapseError(400, "%s parameter is invalid." % qp)
-
-        return PaginationConfig(**params)
-
-
-class PaginationStream(object):
-
-    """ An interface for streaming data as chunks. """
-
-    TOK_START = "START"
-    TOK_END = "END"
-
-    def get_chunk(self, config=None):
-        """ Return the next chunk in the stream.
-
-        Args:
-            config (PaginationConfig): The config to aid which chunk to get.
-        Returns:
-            A dict containing the new start token "start", the new end token
-            "end" and the data "chunk" as a list.
-        """
-        raise NotImplementedError()
-
-
-class StreamData(object):
-
-    """ An interface for obtaining streaming data from a table. """
-
-    def __init__(self, hs):
-        self.hs = hs
-        self.store = hs.get_datastore()
-
-    def get_rows(self, user_id, from_pkey, to_pkey, limit):
-        """ Get event stream data between the specified pkeys.
-
-        Args:
-            user_id : The user's ID
-            from_pkey : The starting pkey.
-            to_pkey : The end pkey. May be -1 to mean "latest".
-            limit: The max number of results to return.
-        Returns:
-            A tuple containing the list of event stream data and the last pkey.
-        """
-        raise NotImplementedError()
-
-    def max_token(self):
-        """ Get the latest currently-valid token.
-
-        Returns:
-            The latest token."""
-        raise NotImplementedError()

synapse/api/streams/event.py

@@ -1,248 +0,0 @@
-# -*- coding: utf-8 -*-
-# Copyright 2014 matrix.org
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-"""This module contains classes for streaming from the event stream: /events.
-"""
-from twisted.internet import defer
-
-from synapse.api.errors import EventStreamError
-from synapse.api.events.room import (
-    RoomMemberEvent, MessageEvent, FeedbackEvent, RoomTopicEvent
-)
-from synapse.api.streams import PaginationStream, StreamData
-
-import logging
-
-logger = logging.getLogger(__name__)
-
-
-class MessagesStreamData(StreamData):
-    EVENT_TYPE = MessageEvent.TYPE
-
-    def __init__(self, hs, room_id=None, feedback=False):
-        super(MessagesStreamData, self).__init__(hs)
-        self.room_id = room_id
-        self.with_feedback = feedback
-
-    @defer.inlineCallbacks
-    def get_rows(self, user_id, from_key, to_key, limit):
-        (data, latest_ver) = yield self.store.get_message_stream(
-            user_id=user_id,
-            from_key=from_key,
-            to_key=to_key,
-            limit=limit,
-            room_id=self.room_id,
-            with_feedback=self.with_feedback
-        )
-        defer.returnValue((data, latest_ver))
-
-    @defer.inlineCallbacks
-    def max_token(self):
-        val = yield self.store.get_max_message_id()
-        defer.returnValue(val)
-
-
-class RoomMemberStreamData(StreamData):
-    EVENT_TYPE = RoomMemberEvent.TYPE
-
-    @defer.inlineCallbacks
-    def get_rows(self, user_id, from_key, to_key, limit):
-        (data, latest_ver) = yield self.store.get_room_member_stream(
-            user_id=user_id,
-            from_key=from_key,
-            to_key=to_key
-        )
-
-        defer.returnValue((data, latest_ver))
-
-    @defer.inlineCallbacks
-    def max_token(self):
-        val = yield self.store.get_max_room_member_id()
-        defer.returnValue(val)
-
-
-class FeedbackStreamData(StreamData):
-    EVENT_TYPE = FeedbackEvent.TYPE
-
-    def __init__(self, hs, room_id=None):
-        super(FeedbackStreamData, self).__init__(hs)
-        self.room_id = room_id
-
-    @defer.inlineCallbacks
-    def get_rows(self, user_id, from_key, to_key, limit):
-        (data, latest_ver) = yield self.store.get_feedback_stream(
-            user_id=user_id,
-            from_key=from_key,
-            to_key=to_key,
-            limit=limit,
-            room_id=self.room_id
-        )
-        defer.returnValue((data, latest_ver))
-
-    @defer.inlineCallbacks
-    def max_token(self):
-        val = yield self.store.get_max_feedback_id()
-        defer.returnValue(val)
-
-
-class RoomDataStreamData(StreamData):
-    EVENT_TYPE = RoomTopicEvent.TYPE  # TODO need multiple event types
-
-    def __init__(self, hs, room_id=None):
-        super(RoomDataStreamData, self).__init__(hs)
-        self.room_id = room_id
-
-    @defer.inlineCallbacks
-    def get_rows(self, user_id, from_key, to_key, limit):
-        (data, latest_ver) = yield self.store.get_room_data_stream(
-            user_id=user_id,
-            from_key=from_key,
-            to_key=to_key,
-            limit=limit,
-            room_id=self.room_id
-        )
-        defer.returnValue((data, latest_ver))
-
-    @defer.inlineCallbacks
-    def max_token(self):
-        val = yield self.store.get_max_room_data_id()
-        defer.returnValue(val)
-
-
-class EventStream(PaginationStream):
-
-    SEPARATOR = '_'
-
-    def __init__(self, user_id, stream_data_list):
-        super(EventStream, self).__init__()
-        self.user_id = user_id
-        self.stream_data = stream_data_list
-
-    @defer.inlineCallbacks
-    def fix_tokens(self, pagination_config):
-        pagination_config.from_tok = yield self.fix_token(
-            pagination_config.from_tok)
-        pagination_config.to_tok = yield self.fix_token(
-            pagination_config.to_tok)
-        defer.returnValue(pagination_config)
-
-    @defer.inlineCallbacks
-    def fix_token(self, token):
-        """Fixes unknown values in a token to known values.
-
-        Args:
-            token (str): The token to fix up.
-        Returns:
-            The fixed-up token, which may == token.
-        """
-        # replace TOK_START and TOK_END with 0_0_0 or -1_-1_-1 depending.
-        replacements = [
-            (PaginationStream.TOK_START, "0"),
-            (PaginationStream.TOK_END, "-1")
-        ]
-        for magic_token, key in replacements:
-            if magic_token == token:
-                token = EventStream.SEPARATOR.join(
-                    [key] * len(self.stream_data)
-                )
-
-        # replace -1 values with an actual pkey
-        token_segments = self._split_token(token)
-        for i, tok in enumerate(token_segments):
-            if tok == -1:
-                # add 1 to the max token because results are EXCLUSIVE from the
-                # latest version.
-                token_segments[i] = 1 + (yield self.stream_data[i].max_token())
-        defer.returnValue(EventStream.SEPARATOR.join(
-            str(x) for x in token_segments
-        ))
-
-    @defer.inlineCallbacks
-    def get_chunk(self, config=None):
-        # no support for limit on >1 streams, makes no sense.
-        if config.limit and len(self.stream_data) > 1:
-            raise EventStreamError(
-                400, "Limit not supported on multiplexed streams."
-            )
-
-        (chunk_data, next_tok) = yield self._get_chunk_data(config.from_tok,
-                                                            config.to_tok,
-                                                            config.limit)
-
-        defer.returnValue({
-            "chunk": chunk_data,
-            "start": config.from_tok,
-            "end": next_tok
-        })
-
-    @defer.inlineCallbacks
-    def _get_chunk_data(self, from_tok, to_tok, limit):
-        """ Get event data between the two tokens.
-
-        Tokens are SEPARATOR separated values representing pkey values of
-        certain tables, and the position determines the StreamData invoked
-        according to the STREAM_DATA list.
-
-        The magic value '-1' can be used to get the latest value.
-
-        Args:
-            from_tok - The token to start from.
-            to_tok - The token to end at. Must have values > from_tok or be -1.
-        Returns:
-            A list of event data.
-        Raises:
-            EventStreamError if something went wrong.
-        """
-        # sanity check
-        if (from_tok.count(EventStream.SEPARATOR) !=
-                to_tok.count(EventStream.SEPARATOR) or
-                (from_tok.count(EventStream.SEPARATOR) + 1) !=
-                len(self.stream_data)):
-            raise EventStreamError(400, "Token lengths don't match.")
-
-        chunk = []
-        next_ver = []
-        for i, (from_pkey, to_pkey) in enumerate(zip(
-            self._split_token(from_tok),
-            self._split_token(to_tok)
-        )):
-            if from_pkey == to_pkey:
-                # tokens are the same, we have nothing to do.
-                next_ver.append(str(to_pkey))
-                continue
-
-            (event_chunk, max_pkey) = yield self.stream_data[i].get_rows(
-                self.user_id, from_pkey, to_pkey, limit
-            )
-
-            chunk += event_chunk
-            next_ver.append(str(max_pkey))
-
-        defer.returnValue((chunk, EventStream.SEPARATOR.join(next_ver)))
-
-    def _split_token(self, token):
-        """Splits the given token into a list of pkeys.
-
-        Args:
-            token (str): The token with SEPARATOR values.
-        Returns:
-            A list of ints.
-        """
-        segments = token.split(EventStream.SEPARATOR)
-        try:
-            int_segments = [int(x) for x in segments]
-        except ValueError:
-            raise EventStreamError(400, "Bad token: %s" % token)
-        return int_segments

synapse/api/urls.py

@@ -0,0 +1,21 @@
+# -*- coding: utf-8 -*-
+# Copyright 2014 OpenMarket Ltd
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Contains the URL paths to prefix various aspects of the server with. """
+
+CLIENT_PREFIX = "/_matrix/client/api/v1"
+FEDERATION_PREFIX = "/_matrix/federation/v1"
+WEB_CLIENT_PREFIX = "/_matrix/client"
+CONTENT_REPO_PREFIX = "/_matrix/content"

synapse/app/__init__.py

@@ -1,5 +1,5 @@
 # -*- coding: utf-8 -*-
-# Copyright 2014 matrix.org
+# Copyright 2014 OpenMarket Ltd
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.

synapse/app/homeserver.py

@@ -1,6 +1,6 @@
 #!/usr/bin/env python
 # -*- coding: utf-8 -*-
-# Copyright 2014 matrix.org
+# Copyright 2014 OpenMarket Ltd
 #
 # Licensed under the Apache License, Version 2.0 (the "License");
 # you may not use this file except in compliance with the License.
@@ -14,151 +14,227 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-from synapse.storage import read_schema
+from synapse.storage import prepare_database
 
 from synapse.server import HomeServer
 
 from twisted.internet import reactor
 from twisted.enterprise import adbapi
-from twisted.python.log import PythonLoggingObserver
-from synapse.http.server import TwistedHttpServer
+from twisted.web.resource import Resource
+from twisted.web.static import File
+from twisted.web.server import Site
+from synapse.http.server import JsonResource, RootRedirect
+from synapse.http.content_repository import ContentRepoResource
 from synapse.http.client import TwistedHttpClient
+from synapse.api.urls import (
+    CLIENT_PREFIX, FEDERATION_PREFIX, WEB_CLIENT_PREFIX, CONTENT_REPO_PREFIX
+)
+from synapse.config.homeserver import HomeServerConfig
+from synapse.crypto import context_factory
 
 from daemonize import Daemonize
+import twisted.manhole.telnet
 
-import argparse
 import logging
-import logging.config
+import os
+import re
+import sys
 import sqlite3
 
 logger = logging.getLogger(__name__)
 
 
 class SynapseHomeServer(HomeServer):
-    def build_http_server(self):
-        return TwistedHttpServer()
 
     def build_http_client(self):
-        return TwistedHttpClient()
+        return TwistedHttpClient(self)
 
-    def build_db_pool(self):
-        """ Set up all the dbs. Since all the *.sql have IF NOT EXISTS, so we
-        don't have to worry about overwriting existing content.
-        """
-        logging.info("Preparing database: %s...", self.db_name)
-        pool = adbapi.ConnectionPool(
-            'sqlite3', self.db_name, check_same_thread=False,
-            cp_min=1, cp_max=1)
+    def build_resource_for_client(self):
+        return JsonResource()
 
-        schemas = [
-            "transactions",
-            "pdu",
-            "users",
-            "profiles",
-            "presence",
-            "im",
-            "room_aliases",
-        ]
+    def build_resource_for_federation(self):
+        return JsonResource()
 
-        for sql_loc in schemas:
-            sql_script = read_schema(sql_loc)
+    def build_resource_for_web_client(self):
+        return File("webclient")  # TODO configurable?
 
-            with sqlite3.connect(self.db_name) as db_conn:
-                c = db_conn.cursor()
-                c.executescript(sql_script)
-                c.close()
-                db_conn.commit()
-
-        logging.info("Database prepared in %s.", self.db_name)
-
-        return pool
-
-
-def setup_logging(verbosity=0, filename=None, config_path=None):
-    """ Sets up logging with verbosity levels.
-
-    Args:
-        verbosity: The verbosity level.
-        filename: Log to the given file rather than to the console.
-        config_path: Path to a python logging config file.
-    """
-
-    if config_path is None:
-        log_format = (
-            '%(asctime)s - %(name)s - %(lineno)d - %(levelname)s - %(message)s'
+    def build_resource_for_content_repo(self):
+        return ContentRepoResource(
+            self, self.upload_dir, self.auth, self.content_addr
         )
 
-        level = logging.INFO
-        if verbosity:
-            level = logging.DEBUG
+    def build_db_pool(self):
+        return adbapi.ConnectionPool(
+            "sqlite3", self.get_db_name(),
+            check_same_thread=False,
+            cp_min=1,
+            cp_max=1
+        )
 
-        # FIXME: we need a logging.WARN for a -q quiet option
+    def create_resource_tree(self, web_client, redirect_root_to_web_client):
+        """Create the resource tree for this Home Server.
 
-        logging.basicConfig(level=level, filename=filename, format=log_format)
-    else:
-        logging.config.fileConfig(config_path)
+        This in unduly complicated because Twisted does not support putting
+        child resources more than 1 level deep at a time.
 
-    observer = PythonLoggingObserver()
-    observer.start()
+        Args:
+            web_client (bool): True to enable the web client.
+            redirect_root_to_web_client (bool): True to redirect '/' to the
+            location of the web client. This does nothing if web_client is not
+            True.
+        """
+        # list containing (path_str, Resource) e.g:
+        # [ ("/aaa/bbb/cc", Resource1), ("/aaa/dummy", Resource2) ]
+        desired_tree = [
+            (CLIENT_PREFIX, self.get_resource_for_client()),
+            (FEDERATION_PREFIX, self.get_resource_for_federation()),
+            (CONTENT_REPO_PREFIX, self.get_resource_for_content_repo())
+        ]
+        if web_client:
+            logger.info("Adding the web client.")
+            desired_tree.append((WEB_CLIENT_PREFIX,
+                                self.get_resource_for_web_client()))
 
+        if web_client and redirect_root_to_web_client:
+            self.root_resource = RootRedirect(WEB_CLIENT_PREFIX)
+        else:
+            self.root_resource = Resource()
 
-def run():
-    reactor.run()
+        # ideally we'd just use getChild and putChild but getChild doesn't work
+        # unless you give it a Request object IN ADDITION to the name :/ So
+        # instead, we'll store a copy of this mapping so we can actually add
+        # extra resources to existing nodes. See self._resource_id for the key.
+        resource_mappings = {}
+        for (full_path, resource) in desired_tree:
+            logging.info("Attaching %s to path %s", resource, full_path)
+            last_resource = self.root_resource
+            for path_seg in full_path.split('/')[1:-1]:
+                if not path_seg in last_resource.listNames():
+                    # resource doesn't exist, so make a "dummy resource"
+                    child_resource = Resource()
+                    last_resource.putChild(path_seg, child_resource)
+                    res_id = self._resource_id(last_resource, path_seg)
+                    resource_mappings[res_id] = child_resource
+                    last_resource = child_resource
+                else:
+                    # we have an existing Resource, use that instead.
+                    res_id = self._resource_id(last_resource, path_seg)
+                    last_resource = resource_mappings[res_id]
+
+            # ===========================
+            # now attach the actual desired resource
+            last_path_seg = full_path.split('/')[-1]
+
+            # if there is already a resource here, thieve its children and
+            # replace it
+            res_id = self._resource_id(last_resource, last_path_seg)
+            if res_id in resource_mappings:
+                # there is a dummy resource at this path already, which needs
+                # to be replaced with the desired resource.
+                existing_dummy_resource = resource_mappings[res_id]
+                for child_name in existing_dummy_resource.listNames():
+                    child_res_id = self._resource_id(existing_dummy_resource,
+                                                     child_name)
+                    child_resource = resource_mappings[child_res_id]
+                    # steal the children
+                    resource.putChild(child_name, child_resource)
+
+            # finally, insert the desired resource in the right place
+            last_resource.putChild(last_path_seg, resource)
+            res_id = self._resource_id(last_resource, last_path_seg)
+            resource_mappings[res_id] = resource
+
+        return self.root_resource
+
+    def _resource_id(self, resource, path_seg):
+        """Construct an arbitrary resource ID so you can retrieve the mapping
+        later.
+
+        If you want to represent resource A putChild resource B with path C,
+        the mapping should looks like _resource_id(A,C) = B.
+
+        Args:
+            resource (Resource): The *parent* Resource
+            path_seg (str): The name of the child Resource to be attached.
+        Returns:
+            str: A unique string which can be a key to the child Resource.
+        """
+        return "%s-%s" % (resource, path_seg)
+
+    def start_listening(self, secure_port, unsecure_port):
+        if secure_port is not None:
+            reactor.listenSSL(
+                secure_port, Site(self.root_resource), self.tls_context_factory
+            )
+            logger.info("Synapse now listening on port %d", secure_port)
+        if unsecure_port is not None:
+            reactor.listenTCP(
+                unsecure_port, Site(self.root_resource)
+            )
+            logger.info("Synapse now listening on port %d", unsecure_port)
 
 
 def setup():
-    parser = argparse.ArgumentParser()
-    parser.add_argument("-p", "--port", dest="port", type=int, default=8080,
-                        help="The port to listen on.")
-    parser.add_argument("-d", "--database", dest="db", default="homeserver.db",
-                        help="The database name.")
-    parser.add_argument("-H", "--host", dest="host", default="localhost",
-                        help="The hostname of the server.")
-    parser.add_argument('-v', '--verbose', dest="verbose", action='count',
-                        help="The verbosity level.")
-    parser.add_argument('-f', '--log-file', dest="log_file", default=None,
-                        help="File to log to.")
-    parser.add_argument('--log-config', dest="log_config", default=None,
-                        help="Python logging config")
-    parser.add_argument('-D', '--daemonize', action='store_true',
-                        default=False, help="Daemonize the home server")
-    parser.add_argument('--pid-file', dest="pid", help="When running as a "
-                        "daemon, the file to store the pid in",
-                        default="hs.pid")
-    parser.add_argument("-w", "--webclient", dest="webclient",
-                        action="store_true", help="Host the web client.")
-    args = parser.parse_args()
-
-    verbosity = int(args.verbose) if args.verbose else None
-
-    setup_logging(
-        verbosity=verbosity,
-        filename=args.log_file,
-        config_path=args.log_config,
+    config = HomeServerConfig.load_config(
+        "Synapse Homeserver",
+        sys.argv[1:],
+        generate_section="Homeserver"
     )
 
-    logger.info("Server hostname: %s", args.host)
+    config.setup_logging()
+
+    logger.info("Server hostname: %s", config.server_name)
+
+    if re.search(":[0-9]+$", config.server_name):
+        domain_with_port = config.server_name
+    else:
+        domain_with_port = "%s:%s" % (config.server_name, config.bind_port)
+
+    tls_context_factory = context_factory.ServerContextFactory(config)
 
     hs = SynapseHomeServer(
-        args.host,
-        db_name=args.db
+        config.server_name,
+        domain_with_port=domain_with_port,
+        upload_dir=os.path.abspath("uploads"),
+        db_name=config.database_path,
+        tls_context_factory=tls_context_factory,
+        config=config,
+        content_addr=config.content_addr,
     )
 
-    # This object doesn't need to be saved because it's set as the handler for
-    # the replication layer
-    hs.get_federation()
-
     hs.register_servlets()
 
-    hs.get_http_server().start_listening(args.port)
+    hs.create_resource_tree(
+        web_client=config.webclient,
+        redirect_root_to_web_client=True,
+    )
 
-    hs.build_db_pool()
+    db_name = hs.get_db_name()
 
-    if args.daemonize:
+    logging.info("Preparing database: %s...", db_name)
+
+    with sqlite3.connect(db_name) as db_conn:
+        prepare_database(db_conn)
+
+    logging.info("Database prepared in %s.", db_name)
+
+    hs.get_db_pool()
+
+    if config.manhole:
+        f = twisted.manhole.telnet.ShellFactory()
+        f.username = "matrix"
+        f.password = "rabbithole"
+        f.namespace['hs'] = hs
+        reactor.listenTCP(config.manhole, f, interface='127.0.0.1')
+
+    hs.start_listening(config.bind_port, config.unsecure_port)
+
+    if config.daemonize:
+        print config.pid_file
         daemon = Daemonize(
             app="synapse-homeserver",
-            pid=args.pid,
-            action=run,
+            pid=config.pid_file,
+            action=reactor.run,
             auto_close_fds=False,
             verbose=True,
             logger=logger,
@@ -166,7 +242,7 @@ def setup():
 
         daemon.start()
     else:
-        run()
+        reactor.run()
 
 
 if __name__ == '__main__':

synapse/config/__init__.py

@@ -0,0 +1,14 @@
+# -*- coding: utf-8 -*-
+# Copyright 2014 OpenMarket Ltd
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.

synapse/config/_base.py

@@ -0,0 +1,131 @@
+# -*- coding: utf-8 -*-
+# Copyright 2014 OpenMarket Ltd
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import argparse
+import sys
+import os
+import yaml
+
+
+class ConfigError(Exception):
+    pass
+
+
+class Config(object):
+    def __init__(self, args):
+        pass
+
+    @staticmethod
+    def abspath(file_path):
+        return os.path.abspath(file_path) if file_path else file_path
+
+    @classmethod
+    def check_file(cls, file_path, config_name):
+        if file_path is None:
+            raise ConfigError(
+                "Missing config for %s."
+                " Try running again with --generate-config"
+                % (config_name,)
+            )
+        if not os.path.exists(file_path):
+            raise ConfigError(
+                "File % config for %s doesn't exist."
+                " Try running again with --generate-config"
+                % (config_name,)
+            )
+        return cls.abspath(file_path)
+
+    @classmethod
+    def read_file(cls, file_path, config_name):
+        cls.check_file(file_path, config_name)
+        with open(file_path) as file_stream:
+            return file_stream.read()
+
+    @staticmethod
+    def read_config_file(file_path):
+        with open(file_path) as file_stream:
+            return yaml.load(file_stream)
+
+    @classmethod
+    def add_arguments(cls, parser):
+        pass
+
+    @classmethod
+    def generate_config(cls, args, config_dir_path):
+        pass
+
+    @classmethod
+    def load_config(cls, description, argv, generate_section=None):
+        config_parser = argparse.ArgumentParser(add_help=False)
+        config_parser.add_argument(
+            "-c", "--config-path",
+            metavar="CONFIG_FILE",
+            help="Specify config file"
+        )
+        config_parser.add_argument(
+            "--generate-config",
+            action="store_true",
+            help="Generate config file"
+        )
+        config_args, remaining_args = config_parser.parse_known_args(argv)
+
+        if config_args.generate_config:
+            if not config_args.config_path:
+                config_parser.error(
+                    "Must specify where to generate the config file"
+                )
+            config_dir_path = os.path.dirname(config_args.config_path)
+            if os.path.exists(config_args.config_path):
+                defaults = cls.read_config_file(config_args.config_path)
+            else:
+                defaults = {}
+        else:
+            if config_args.config_path:
+                defaults = cls.read_config_file(config_args.config_path)
+            else:
+                defaults = {}
+
+        parser = argparse.ArgumentParser(
+            parents=[config_parser],
+            description=description,
+            formatter_class=argparse.RawDescriptionHelpFormatter,
+        )
+        cls.add_arguments(parser)
+        parser.set_defaults(**defaults)
+
+        args = parser.parse_args(remaining_args)
+
+        if config_args.generate_config:
+            config_dir_path = os.path.dirname(config_args.config_path)
+            config_dir_path = os.path.abspath(config_dir_path)
+            if not os.path.exists(config_dir_path):
+                os.makedirs(config_dir_path)
+            cls.generate_config(args, config_dir_path)
+            config = {}
+            for key, value in vars(args).items():
+                if (key not in set(["config_path", "generate_config"])
+                    and value is not None):
+                    config[key] = value
+            with open(config_args.config_path, "w") as config_file:
+                # TODO(paul) it would be lovely if we wrote out vim- and emacs-
+                #   style mode markers into the file, to hint to people that
+                #   this is a YAML file.
+                yaml.dump(config, config_file, default_flow_style=False)
+            sys.exit(0)
+
+        return cls(args)
+
+
+

synapse/config/captcha.py

@@ -0,0 +1,46 @@
+# Copyright 2014 OpenMarket Ltd
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from ._base import Config
+
+
+class CaptchaConfig(Config):
+
+    def __init__(self, args):
+        super(CaptchaConfig, self).__init__(args)
+        self.recaptcha_private_key = args.recaptcha_private_key
+        self.enable_registration_captcha = args.enable_registration_captcha
+        self.captcha_ip_origin_is_x_forwarded = (
+            args.captcha_ip_origin_is_x_forwarded
+        )
+
+    @classmethod
+    def add_arguments(cls, parser):
+        super(CaptchaConfig, cls).add_arguments(parser)
+        group = parser.add_argument_group("recaptcha")
+        group.add_argument(
+            "--recaptcha-private-key", type=str, default="YOUR_PRIVATE_KEY",
+            help="The matching private key for the web client's public key."
+        )
+        group.add_argument(
+            "--enable-registration-captcha", type=bool, default=False,
+            help="Enables ReCaptcha checks when registering, preventing signup"
+            + " unless a captcha is answered. Requires a valid ReCaptcha "
+            + "public/private key."
+        )
+        group.add_argument(
+            "--captcha_ip_origin_is_x_forwarded", type=bool, default=False,
+            help="When checking captchas, use the X-Forwarded-For (XFF) header"
+            + " as the client IP and not the actual client IP."
+        )