summaryrefslogtreecommitdiffstats
path: root/testing/web-platform/tests/tools/third_party/websockets/docs/topics/deployment.rst
diff options
context:
space:
mode:
Diffstat (limited to 'testing/web-platform/tests/tools/third_party/websockets/docs/topics/deployment.rst')
-rw-r--r--testing/web-platform/tests/tools/third_party/websockets/docs/topics/deployment.rst181
1 files changed, 181 insertions, 0 deletions
diff --git a/testing/web-platform/tests/tools/third_party/websockets/docs/topics/deployment.rst b/testing/web-platform/tests/tools/third_party/websockets/docs/topics/deployment.rst
new file mode 100644
index 0000000000..2a1fe9a785
--- /dev/null
+++ b/testing/web-platform/tests/tools/third_party/websockets/docs/topics/deployment.rst
@@ -0,0 +1,181 @@
+Deployment
+==========
+
+.. currentmodule:: websockets
+
+When you deploy your websockets server to production, at a high level, your
+architecture will almost certainly look like the following diagram:
+
+.. image:: deployment.svg
+
+The basic unit for scaling a websockets server is "one server process". Each
+blue box in the diagram represents one server process.
+
+There's more variation in routing. While the routing layer is shown as one big
+box, it is likely to involve several subsystems.
+
+When you design a deployment, your should consider two questions:
+
+1. How will I run the appropriate number of server processes?
+2. How will I route incoming connections to these processes?
+
+These questions are strongly related. There's a wide range of acceptable
+answers, depending on your goals and your constraints.
+
+You can find a few concrete examples in the :ref:`deployment how-to guides
+<deployment-howto>`.
+
+Running server processes
+------------------------
+
+How many processes do I need?
+.............................
+
+Typically, one server process will manage a few hundreds or thousands
+connections, depending on the frequency of messages and the amount of work
+they require.
+
+CPU and memory usage increase with the number of connections to the server.
+
+Often CPU is the limiting factor. If a server process goes to 100% CPU, then
+you reached the limit. How much headroom you want to keep is up to you.
+
+Once you know how many connections a server process can manage and how many
+connections you need to handle, you can calculate how many processes to run.
+
+You can also automate this calculation by configuring an autoscaler to keep
+CPU usage or connection count within acceptable limits.
+
+Don't scale with threads. Threads doesn't make sense for a server built with
+:mod:`asyncio`.
+
+How do I run processes?
+.......................
+
+Most solutions for running multiple instances of a server process fall into
+one of these three buckets:
+
+1. Running N processes on a platform:
+
+ * a Kubernetes Deployment
+
+ * its equivalent on a Platform as a Service provider
+
+2. Running N servers:
+
+ * an AWS Auto Scaling group, a GCP Managed instance group, etc.
+
+ * a fixed set of long-lived servers
+
+3. Running N processes on a server:
+
+ * preferably via a process manager or supervisor
+
+Option 1 is easiest of you have access to such a platform.
+
+Option 2 almost always combines with option 3.
+
+How do I start a process?
+.........................
+
+Run a Python program that invokes :func:`~server.serve`. That's it.
+
+Don't run an ASGI server such as Uvicorn, Hypercorn, or Daphne. They're
+alternatives to websockets, not complements.
+
+Don't run a WSGI server such as Gunicorn, Waitress, or mod_wsgi. They aren't
+designed to run WebSocket applications.
+
+Applications servers handle network connections and expose a Python API. You
+don't need one because websockets handles network connections directly.
+
+How do I stop a process?
+........................
+
+Process managers send the SIGTERM signal to terminate processes. Catch this
+signal and exit the server to ensure a graceful shutdown.
+
+Here's an example:
+
+.. literalinclude:: ../../example/faq/shutdown_server.py
+ :emphasize-lines: 12-15,18
+
+When exiting the context manager, :func:`~server.serve` closes all connections
+with code 1001 (going away). As a consequence:
+
+* If the connection handler is awaiting
+ :meth:`~server.WebSocketServerProtocol.recv`, it receives a
+ :exc:`~exceptions.ConnectionClosedOK` exception. It can catch the exception
+ and clean up before exiting.
+
+* Otherwise, it should be waiting on
+ :meth:`~server.WebSocketServerProtocol.wait_closed`, so it can receive the
+ :exc:`~exceptions.ConnectionClosedOK` exception and exit.
+
+This example is easily adapted to handle other signals.
+
+If you override the default signal handler for SIGINT, which raises
+:exc:`KeyboardInterrupt`, be aware that you won't be able to interrupt a
+program with Ctrl-C anymore when it's stuck in a loop.
+
+Routing connections
+-------------------
+
+What does routing involve?
+..........................
+
+Since the routing layer is directly exposed to the Internet, it should provide
+appropriate protection against threats ranging from Internet background noise
+to targeted attacks.
+
+You should always secure WebSocket connections with TLS. Since the routing
+layer carries the public domain name, it should terminate TLS connections.
+
+Finally, it must route connections to the server processes, balancing new
+connections across them.
+
+How do I route connections?
+...........................
+
+Here are typical solutions for load balancing, matched to ways of running
+processes:
+
+1. If you're running on a platform, it comes with a routing layer:
+
+ * a Kubernetes Ingress and Service
+
+ * a service mesh: Istio, Consul, Linkerd, etc.
+
+ * the routing mesh of a Platform as a Service
+
+2. If you're running N servers, you may load balance with:
+
+ * a cloud load balancer: AWS Elastic Load Balancing, GCP Cloud Load
+ Balancing, etc.
+
+ * A software load balancer: HAProxy, NGINX, etc.
+
+3. If you're running N processes on a server, you may load balance with:
+
+ * A software load balancer: HAProxy, NGINX, etc.
+
+ * The operating system — all processes listen on the same port
+
+You may trust the load balancer to handle encryption and to provide security.
+You may add another layer in front of the load balancer for these purposes.
+
+There are many possibilities. Don't add layers that you don't need, though.
+
+How do I implement a health check?
+..................................
+
+Load balancers need a way to check whether server processes are up and running
+to avoid routing connections to a non-functional backend.
+
+websockets provide minimal support for responding to HTTP requests with the
+:meth:`~server.WebSocketServerProtocol.process_request` hook.
+
+Here's an example:
+
+.. literalinclude:: ../../example/faq/health_check_server.py
+ :emphasize-lines: 7-9,18