diff options
Diffstat (limited to '')
| -rw-r--r-- | netwerk/docs/http/http3.md | 154 | ||||
| -rw-r--r-- | netwerk/docs/http/lifecycle.rst | 220 | ||||
| -rw-r--r-- | netwerk/docs/http/logging.rst | 341 | ||||
| -rw-r--r-- | netwerk/docs/http_server_for_testing.rst | 482 |
4 files changed, 1197 insertions, 0 deletions
diff --git a/netwerk/docs/http/http3.md b/netwerk/docs/http/http3.md new file mode 100644 index 0000000000..0f5e241569 --- /dev/null +++ b/netwerk/docs/http/http3.md @@ -0,0 +1,154 @@ +# Http3Session and Streams + +The HTTP/3 and QUIC protocol are implemented in the neqo library. Http3Session, Http3Steam, and Http3WebTransportStream are added to integrate the library into the existing necko code. + +The following classes are necessary: +- HttpConnectionUDP - this is the object that is registered in nsHttpConnectionMgr and it is also used as an async listener to socket events (it implements nsIUDPSocketSyncListener) +- nsUDPSocket - represent a UDP socket and implements nsASocketHandler. nsSocketTransportService manages UDP and TCP sockets and calls the corresponding nsASocketHandler when the socket encounters an error or has data to be read, etc. +- NeqoHttp3Conn is a c++ object that maps to the rust object Http3Client. +- Http3Session manages NeqoHttp3Conn/Http3Client and provides bridge between the rust implementation and necko legacy code, i.e. HttpConnectionUDP and nsHttpTransaction. +- Http3Streams are used to map reading and writing from/into a nsHttpTransaction onto the NeqoHttp3Conn/Http3Client API (e.g. nsHttpTransaction::OnWriteSegment will call Http3Client::read_data). NeqoHttp3Conn is only accessed by Http3Sesson and NeqoHttp3Conn functions are exposed through Http3Session where needed. + +```{mermaid} +graph TD + A[HttpConnectionMgr] --> B[HttpConnectionUDP] + B --> C[nsUDPSocket] + C --> B + D[nsSocketTransportService] --> C + B --> E[NeqoHttp3Conn] + B --> F[Http3Stream] + F -->|row| B + F --> G[nsHttpTransport] + G --> B + B --> G +``` + +## Interactions with Sockets and Driving Neqo + +As described in [this docs](https://github.com/mozilla/neqo/blob/main/neqo-http3/src/lib.rs), neqo does not create a socket, it produces, i.e. encodes, data that should be sent as a payload in a UDP packet and consumes data received on the UDP socket. Therefore the necko is responsible for creating a socket and reading and writing data from/into the socket. Necko uses nsUDPSocket and nsSocketTransportService for this. +The UDP socket is constantly polled for reading. It is not polled for writing, we let QUIC control to not overload the network path and buffers. + +When the UDP socket has an available packet, nsSocketTransportService will return from the polling function and call nsUDPSocket::OnSocketReady, which calls HttpConnectionUDP::OnPacketReceived, HttpConnectionUDP::RecvData and further Http3Session::RecvData. For writing data +HttpConnectionUDP::SendData is called which calls Http3Session::SendData. + +Neqo needs an external timer. The timer is managed by Http3Session. When the timer expires HttpConnectionUDP::OnQuicTimeoutExpired is executed that calls Http3Session::ProcessOutputAndEvents. + +HttpConnectionUDP::RecvData, HttpConnectionUDP::SendData or HttpConnectionUDP::OnQuicTimeoutExpired must be on the stack when we interact with neqo. The reason is that they are responsible for proper clean-up in case of an error. For example, if there is a slow reader that is ready to read, it will call Http3Session::TransactionHasDataToRecv to be registered in a list and HttpConnectionUDP::ForceRecv will be called that will call the same function chain as in the case a new packet is received, i.e. HttpConnectionUDP::RecvData and further Http3Session::RecvData. The other example is when a new HTTP transaction is added to the session, the transaction needs to send data. The transaction will be registered in a list and HttpConnectionUDP::ResumeSend will be called which further calls HttpConnectionUDP::SendData. + +Http3Session holds a reference to a ConnectionHandler object which is a wrapper object around HttpConnectionUDP. The destructor of ConnectionHandler calls nsHttpHandler::ReclaimConnection which is responsible for removing the connection from nsHttpConnectionMgr. +HttpConnectionUDP::RecvData, HttpConnectionUDP::SendData or HttpConnectionUDP::OnQuicTimeoutExpired call HttpConnectionUDP::CloseTransaction which will cause Http3Session to remove the reference to the ConnectionHandler object. The ConnectionHandler object will be destroyed and nsHttpHandler::ReclaimConnection will be called. +This behavior is historical and it is also used for HTTP/2 and older versions. In the case of the older versions, nsHttpHandler::ReclaimConnection may actually reuse a connection instead of removing it from nsHttpConnectionMgr. + +Three main neqo functions responsible for driving neqo are process_input, process_output, and next_event. They are called by: +- Http3Session::ProcessInput, +- Http3Session::ProcesOutput and, +- Http3Session::ProcessEvents. + +**ProcessInput** +In this function we take data from the UDP socket and call NeqoHttp3Conn::ProcessInput that maps to Http3Client::process_input. The packets are read from the socket until the socket buffer is empty. + +**ProcessEvents** +This function process all available neqo events. It returns earlier only in case of a fatal error. +It calls NeqoHttp3Conn::GetEvent which maps to Http3Client::next_event. +The events and their handling will be explained below. + +**ProcessOutput** +The function is called when necko has performed some action on neqo, e.g. new HTTP transaction is added, certificate verification is done, etc., or when the timer expires. In both cases, necko wants to check if neqo has data to send or change its state. This function calls NeqoHttp3Conn::ProcessOutput that maps to Http3Client::process_output. NeqoHttp3Conn::ProcessOutput may return a packet that is sent on the socket or a callback timeout. In the Http3Session::ProcessOutput function, NeqoHttp3Conn::ProcessOutput is called repeatedly and packets are sent until a callback timer is returned or a fatal error happens. + +**Http3Session::RecvData** performs the following steps: +- ProcessSlowConsumers - explained below. +- ProcessInput - process new packets. +- ProcessEvents - look if there are new events +- ProcessOutput - look if we have new packets to send after packets arrive(e.g. sending ack) or due to event processing (e.g. a stream has been canceled). + +**Http3Session::SendData** performed the following steps: +- Process (HTTP and WebTransport) streams that have data to write. +- ProcessOutput - look if there are new packets to be sent after streams have supplied data to neqo. + + +**Http3Session::ProcessOutputAndEvents** performed the following steps: +- ProcessOutput - after a timeout most probably neqo will have data to retransmit or it will send a ping +- ProcessEvents - look if the state of the connection has changed, i.e. the connection timed out + + +## HTTP and WebTransport Streams Reading Data + +The following diagram shows how data are read from an HTTP stream. The diagram for a WebTransport stream will be added later. + +```{mermaid} +flowchart TD + A1[nsUDPSocket::OnSocketReady] --> |HttpConnectionUDP::OnPacketReceived| C[HttpConnectionUDP] + A[HttpConnectionUDP::ResumeRecv calls] --> C + B[HttpConnectionUDPForceIO] --> |HttpConnectionUDP::RecvData| C + C -->|1. Http3Session::RecvData| D[Http3Session] + D --> |2. Http3Stream::WriteSegments|E[Http3Stream] + E --> |3. nsHttpTransaction::WriteSegmentsAgain| F[nsHttpTransaction] + F --> |4. nsPipeOutputStream::WriteSegments| G["nsPipeOutputStream"] + G --> |5. nsHttpTransaction::WritePiipeSegnemt| F + F --> |6. Http3Stream::OnWriteSegment| E + E --> |"7. Return response headers or call Http3Session::ReadResponseData"|D + D --> |8. NeqoHttp3Conn::ReadResponseDataReadResponseData| H[NeqoHHttp3Conn] + +``` + +When there is a new packet carrying a stream data arriving on a QUIC connection nsUDPSocket::OnSocketReady will be called which will call Http3Session::RecvData. Http3Session::RecvData and ProcessInput will read the new packet from the socket and give it to neqo for processing. In the next step, ProcessEvent will be called which will have a DataReadable event and Http3Stream::WriteSegments will be called. +Http3Stream::WriteSegments calls nsHttpTransaction::WriteSegmentsAgain repeatedly until all data are read from the QUIC stream or until the pipe cannot accept more data. The latter can happen when listeners of an HTTP transaction or WebTransport stream are slow and are not able to read all data available on an HTTP3/WebTransport stream fast enough. + +When the pipe cannot accept more data nsHttpTransaction will call nsPipeOutputStream::AsyncWait and wait for the nsHttpTransaction::OnOutputStreamReady callback. When nsHttpTransaction::OnOutputStreamReady is called, Http3Stream/Session::TransactionHasDataToRecv is is executed with the following actions: +- the corresponding stream to a list(mSlowConsumersReadyForRead) and +- nsHttpConnection::ResumeRecv is called (i.e. it forces the same code path as when a socket has data to receive so that errors can be properly handled as explained previously). + +These streams will be processed in ProcessSlowConsumers which is called by Http3Session::RecvData. + +## HTTP and WebTransport Streams Writing Data + +The following diagram shows how data are sent from an HTTP stream. The diagram for a WebTransport stream will be added later. + +```{mermaid} +flowchart TD + A[HttpConnectionUDP::ResumeSend calls] --> C[HttpConnectionUDP] + B[HttpConnectionUDPForceIO] --> |HttpConnectionUDP::SendData| C + C -->|1. Http3Session::SendData| D[Http3Session] + D --> |2. Http3Stream::ReadSegments|E[Http3Stream] + E --> |3. nsHttpTransaction::ReadSegmentsAgain| F[nsHttpTransaction] + F --> |4. nsPipeInputStream::ReadSegments| G["nsPipeInputStream(Request stream)"] + G --> |5. nsHttpTransaction::ReadRequestSegment| F + F --> |6. Http3Stream::OnReadSegment| E + E --> |7. Http3Session::TryActivating/SendRequestBody|D + D --> |8. NeqoHttp3Conn::Fetch/SendRequestBody| H[NeqoHHttp3Conn] +``` + +When a nsHttpTransaction has been newly added to a transaction or when nsHttpTransaction has more data to write Http3Session::StreamReadyToWrite is called (in the latter case through Http3Session::TransactionHasDataToWrite) which performs the following actions: +- add the corresponding stream to a list(mReadyForWrite) and +- call HttpConnectionUDP::ResumeSend + +The Http3Session::SendData function iterates through mReadyForWrite and calls Http3Stream::ReadSegments for each stream. + +## Neqo Events + +For **HeaderReady** and **DataReadable** the Http3Stream::WriteSegments function of the corresponding stream is called. The code path shown in the flowchart above will call the nssHttpTransaction served by the stream to take headers and data. + +**DataWritable** means that a stream could not accept more data earlier and that flow control now allows sending more data. Http3Sesson will mark the stream as writable(by calling Http3Session::StreamReadyToWrite) to verify if a stream wants to write more data. + +**Reset** and **StopSending** events will be propagated to the stream and the stream will be closed. + +**RequestsCreatable** events are posted when a QUIC connection could not accept new streams due to the flow control in the past and the stream flow control is increased and the streams are creatable again. Http3Session::ProcessPendingProcessPending will trigger the activation of the queued streams. + +**AuthenticationNeeded** and **EchFallbackAuthenticationNeeded** are posted when a certificate verification is needed. + + +**ZeroRttRejected** is posted when zero RTT data was rejected. + +**ResumptionToken** is posted when a new resumption token is available. + +**ConnectionConnected**, **GoawayReceived**, **ConnectionClosing** and **ConnectionClosed** expose change in the connection state. Difference between **ConnectionClosing** and **ConnectionClosed** that after **ConnectionClosed** the connection can be immediately closed and after **ConnectionClosing** we will keep the connection object for a short time until **ConnectionClosed** event is received. During this period we will retransmit the closing frame if they are lost. + +### WebTransport Events + +**Negotiated** - WebTransport is negotiated only after the HTTP/3 settings frame has been received from the server. At that point **Negotiated** event is posted to inform the application. + +The **Session** event is posted when a WebTransport session is successfully negotiated. + +The **SessionClosed** event is posted when a connection is closed gracefully or abruptly. + +The **NewStream** is posted when a new stream has been opened by the peer. diff --git a/netwerk/docs/http/lifecycle.rst b/netwerk/docs/http/lifecycle.rst new file mode 100644 index 0000000000..09a326d4d6 --- /dev/null +++ b/netwerk/docs/http/lifecycle.rst @@ -0,0 +1,220 @@ +The Lifecycle of a HTTP Request +=============================== + + +HTTP requests in Firefox go through several steps. Each piece of the request message and response message become available at certain points. Extracting that information is a challenge, though. + +What is Available When +---------------------- + ++-----------------------+---------------------------------------------------+---------------------------------------+------------------------+-------------------------------+ +| Data | When it's available | Sample JS code | Interfaces | Test code | ++=======================+===================================================+=======================================+========================+===============================+ +| HTTP request method | *http-on-modify-request* observer notification | channel.requestMethod | nsIHttpChannel_ | | ++-----------------------+---------------------------------------------------+---------------------------------------+------------------------+-------------------------------+ +| HTTP request URI | *http-on-modify-request* observer notification | channel.URI | nsIChannel_ | | ++-----------------------+---------------------------------------------------+---------------------------------------+------------------------+-------------------------------+ +| HTTP request headers | *http-on-modify-request* observer notification | channel.visitRequestHeaders(visitor) | nsIHttpChannel_ | | ++-----------------------+---------------------------------------------------+---------------------------------------+------------------------+-------------------------------+ +| HTTP request body | *http-on-modify-request* observer notification | channel.uploadStream | nsIUploadChannel_ | | ++-----------------------+---------------------------------------------------+---------------------------------------+------------------------+-------------------------------+ +|| HTTP response status || *http-on-examine-response* observer notification || channel.responseStatus || nsIHttpChannel_ || test_basic_functionality.js_ | +|| || || channel.responseStatusText || || | ++-----------------------+---------------------------------------------------+---------------------------------------+------------------------+-------------------------------+ +| HTTP response headers | *http-on-examine-response* observer notification | channel.visitResponseHeaders(visitor) | nsIHttpChannel_ | | ++-----------------------+---------------------------------------------------+---------------------------------------+------------------------+-------------------------------+ +|| HTTP response body || *onStopRequest* via stream listener tee || See below || nsITraceableChannel_ || test_traceable_channel.js_ | +|| || || || nsIStreamListenerTee_ || | +|| || || || nsIPipe_ || | ++-----------------------+---------------------------------------------------+---------------------------------------+------------------------+-------------------------------+ + +The Request: http-on-modify-request +----------------------------------- + +Firefox fires a "http-on-modify-request" observer notification before sending the HTTP request, and this blocks the sending of the request until all observers exit. This is generally the point at which you can modify the HTTP request headers (hence the name). + +Attaching a listener for a request is pretty simple:: + + const obs = { + QueryInterface: ChromeUtils.generateQI(["nsIObserver"]), + + observe: function(channel, topic, data) { + if (!(channel instanceof Ci.nsIHttpChannel)) + return; + + // process the channel's data + } + } + + Services.obs.addObserver(observer, "http-on-modify-request", false); + +See nsIObserverService_ for the details. + +The request method and URI are immediately available at this time. Request headers are trivially easy to get:: + + /** + * HTTP header visitor. + */ + class HeaderVisitor { + #targetObject; + + constructor(targetObject) { + this.#targetObject = targetObject; + } + + // nsIHttpHeaderVisitor + visitHeader(header, value) { + this.#targetObject[header] = value; + } + + QueryInterface = ChromeUtils.generateQI(["nsIHttpHeaderVisitor"]); + } + + // ... + const requestHeaders = {}; + const visitor = new HeaderVisitor(requestHeaders); + channel.visitRequestHeaders(visitor); + +This is also the time to set request headers, if you need to. The method for that on the nsIHttpChannel_ interface is `channel.setRequestHeader(header, value);` + +Most HTTP requests don't have a body, as they are GET requests. POST requests often have them, though. As the nsIUploadChannel_ documentation indicates, the body of most HTTP requests is available via a seekable stream (nsISeekableStream_). So you can simply capture the body stream and its current position, to revisit it later. network-helper.js_ has code to read the request body. + +The Response: http-on-examine-response +-------------------------------------- + +Firefox fires a "http-on-examine-response" observer notification after parsing the HTTP response status and headers, but **before** reading the response body. Attaching a listener for this phase is also very easy:: + + Services.obs.addObserver(observer, "http-on-examine-response", false); + +If you use the same observer for "http-on-modify-request" and "http-on-examine-response", make sure you check the topic argument before interacting with the channel. + +The response status is available via the *responseStatus* and *responseStatusText* properties. The response headers are available via the *visitResponseHeaders* method, and requires the same interface. + +The Response body: onStopRequest, stream listener tee +----------------------------------------------------- + +During the "http-on-examine-response" notification, the response body is *not* available. You can, however, use a stream listener tee to *copy* the stream so that the original stream data goes on, and you have a separate input stream you can read from with the same data. + +Here's some sample code to illustrate what you need:: + + const Pipe = Components.Constructor( + "@mozilla.org/pipe;1", + "nsIPipe", + "init" + ); + const StreamListenerTee = Components.Constructor( + "@mozilla.org/network/stream-listener-tee;1", + "nsIStreamListenerTee" + ); + const ScriptableStream = Components.Constructor( + "@mozilla.org/scriptableinputstream;1", + "nsIScriptableInputStream", + "init" + ); + + const obs = { + QueryInterface: ChromeUtils.generateQI(["nsIObserver", "nsIRequestObserver"]), + + /** @typedef {WeakMap<nsIHttpChannel, nsIPipe>} */ + requestToTeePipe: new WeakMap, + + // nsIObserver + observe: function(channel, topic, data) { + if (!(channel instanceof Ci.nsIHttpChannel)) + return; + + /* Create input and output streams to take the new data. + The 0xffffffff argument is the segment count. + It has to be this high because you don't know how much data is coming in the response body. + + As for why these are blocking streams: I believe this is because there's no actual need to make them non-blocking. + The stream processing happens during onStopRequest(), so we have all the data then and the operation can be synchronous. + But I could be very wrong on this. + */ + const pipe = new Pipe(false, false, 0, 0xffffffff); + + // Install the stream listener tee to intercept the HTTP body. + const tee = new StreamListenerTee; + const originalListener = channel.setNewListener(tee); + tee.init(originalListener, pipe.outputStream, this); + + this.requestToTeePipe.set(channel, pipe); + } + + // nsIRequestObserver + onStartRequest: function() { + // do nothing + } + + // nsIRequestObserver + onStopRequest: function(channel, statusCode) { + const pipe = this.requestToTeePipe.get(channel); + + // No more data coming in anyway. + pipe.outputStream.close(); + this.requestToTeePipe.delete(channel); + + let length = 0; + try { + length = pipe.inputStream.available(); + } + catch (e) { + if (e.result === Components.results.NS_BASE_STREAM_CLOSED) + throw e; + } + + let responseBody = ""; + if (length) { + // C++ code doesn't need the scriptable input stream. + const sin = new ScriptableStream(pipe.inputStream); + responseBody = sin.read(length); + sin.close(); + } + + void(responseBody); // do something with the body + } + } + +test_traceable_channel.js_ does essentially this. + +Character Encodings and Compression +----------------------------------- + +Canceling Requests +------------------ + +HTTP Activity Distributor Notes +------------------------------- + +URIContentLoader Notes +---------------------- + +Order of Operations +------------------- + +1. The HTTP channel is constructed. +2. The "http-on-modify-request" observer service notification fires. +3. If the request has been canceled, exit at this step. +4. The HTTP channel's request is submitted to the server. Time passes. +5. The HTTP channel's response comes in from the server. +6. The HTTP channel parses the response status and headers. +7. The "http-on-examine-response" observer service notification fires. + +Useful Code Samples and References +---------------------------------- + +- nsIHttpProtocolHandler_ defines a lot of observer topics, and has a lot of details. + +.. _nsIHttpChannel: https://searchfox.org/mozilla-central/source/netwerk/protocol/http/nsIHttpChannel.idl +.. _nsIChannel: https://searchfox.org/mozilla-central/source/netwerk/base/nsIChannel.idl +.. _nsIUploadChannel: https://searchfox.org/mozilla-central/source/netwerk/base/nsIUploadChannel.idl +.. _nsITraceableChannel: https://searchfox.org/mozilla-central/source/netwerk/base/nsITraceableChannel.idl +.. _nsISeekableStream: https://searchfox.org/mozilla-central/source/xpcom/io/nsISeekableStream.idl +.. _nsIObserverService: https://searchfox.org/mozilla-central/source/xpcom/ds/nsIObserverService.idl +.. _nsIHttpProtocolHandler: https://searchfox.org/mozilla-central/source/netwerk/protocol/http/nsIHttpProtocolHandler.idl +.. _nsIStreamListenerTee: https://searchfox.org/mozilla-central/source/netwerk/base/nsIStreamListenerTee.idl +.. _nsIPipe: https://searchfox.org/mozilla-central/source/xpcom/io/nsIPipe.idl + +.. _test_basic_functionality.js: https://searchfox.org/mozilla-central/source/netwerk/test/httpserver/test/test_basic_functionality.js +.. _test_traceable_channel.js: https://searchfox.org/mozilla-central/source/netwerk/test/unit/test_traceable_channel.js +.. _network-helper.js: https://searchfox.org/mozilla-central/source/devtools/shared/webconsole/network-helper.js diff --git a/netwerk/docs/http/logging.rst b/netwerk/docs/http/logging.rst new file mode 100644 index 0000000000..cee5a3e40d --- /dev/null +++ b/netwerk/docs/http/logging.rst @@ -0,0 +1,341 @@ +HTTP Logging +============ + + +Sometimes, while debugging your Web app (or client-side code using +Necko), it can be useful to log HTTP traffic. This saves a log of HTTP-related +information from your browser run into a file that you can examine (or +upload to Bugzilla if a developer has asked you for a log). + +.. note:: + + **Note:** The `Web + Console <https://developer.mozilla.org/en-US/docs/Tools/Web_Console>`__ + also offers the ability to peek at HTTP transactions within Firefox. + HTTP logging generally provides more detailed logging. + +.. _using-about-networking: + +Using about:logging +------------------- + +This is the best and easiest way to do HTTP logging. At any point +during while your browser is running, you can turn logging on and off. + +.. note:: + + **Note:** Before Firefox 108 the logging UI used to be located at `about:networking#logging` + +This allows you to capture only the "interesting" part of the browser's +behavior (i.e. your bug), which makes the HTTP log much smaller and +easier to analyze. + +#. Launch the browser and get it into whatever state you need to be in + just before your bug occurs. +#. Open a new tab and type in "about:logging" into the URL bar. +#. Adjust the location of the log file if you don't like the default +#. Adjust the list of modules that you want to log: this list has the + exact same format as the MOZ_LOG environment variable (see below). + Generally the default list is OK, unless a Mozilla developer has told + you to modify it. + + * For cookie issues, use presets ``Cookies`` + * For WebSocket issues, use presets ``WebSockets`` + * For HTTP/3 or QUIC issues, use presets ``HTTP/3`` + * For other networking issues, use presets ``Networking`` + +#. Click on Start Logging. +#. Reproduce the bug (i.e. go to the web site that is broken for you and + make the bug happen in the browser) +#. Make a note of the value of "Current Log File". +#. Click on Stop Logging. +#. Go to the folder containing the specified log file, and gather all + the log files. You will see several files that look like: + log.txt-main.1806.moz_log, log.txt-child.1954.moz_log, + log.txt-child.1970.moz_log, etc. This is because Firefox now uses + multiple processes, and each process gets its own log file. +#. For many bugs, the "log.txt-main.moz_log" file is the only thing you need to + upload as a file attachment to your Bugzilla bug (this is assuming + you're logging to help a mozilla developer). Other bugs may require + all the logs to be uploaded--ask the developer if you're not sure. +#. Pat yourself on the back--a job well done! Thanks for helping us + debug Firefox. + +.. note:: + + **Note:** The log may include sensitive data such as URLs and cookies. + To protect your privacy, we kindly request you to send the log file or + the profiler link directly and confidentially to necko@mozilla.com. + +Logging HTTP activity by manually setting environment variables +--------------------------------------------------------------- + +Sometimes the about:logging approach won't work, for instance if your +bug occurs during startup, or you're running on mobile, etc. In that +case you can set environment variables \*before\* you launch Firefox. +Note that this approach winds up logging the whole browser history, so +files can get rather large (they compress well :) + +Setting environment variables differs by operating system. Don't let the +scary-looking command line stuff frighten you off; it's not hard at all! + +Windows +~~~~~~~ + +#. If Firefox is already running, exit out of it. + +#. Open a command prompt by holding down the Windows key and pressing "R". + +#. Type CMD and press enter, a new Command Prompt window with a black + background will appear. + +#. | Copy and paste the following lines one at a time into the Command + Prompt window. Press the enter key after each one.: + | **For 64-bit Windows:** + + :: + + set MOZ_LOG=timestamp,rotate:200,nsHttp:5,cache2:5,nsSocketTransport:5,nsHostResolver:5 + set MOZ_LOG_FILE=%TEMP%\log.txt + "c:\Program Files\Mozilla Firefox\firefox.exe" + + **For 32-bit Windows:** + + :: + + set MOZ_LOG=timestamp,rotate:200,nsHttp:5,cache2:5,nsSocketTransport:5,nsHostResolver:5 + set MOZ_LOG_FILE=%TEMP%\log.txt + "c:\Program Files (x86)\Mozilla Firefox\firefox.exe" + + (These instructions assume that you installed Firefox to the default + location, and that drive C: is your Windows startup disk. Make the + appropriate adjustments if those aren't the case.) + +#. Reproduce whatever problem it is that you're having. + +#. Once you've reproduced the problem, exit Firefox and look for the + generated log files in your temporary directory. You can type + "%TEMP%" directly into the Windows Explorer location bar to get there + quickly. + +Linux +~~~~~ + +This section offers information on how to capture HTTP logs for Firefox +running on Linux. + +#. Quit out of Firefox if it's running. + +#. Open a new shell. The commands listed here assume a bash-compatible + shell. + +#. Copy and paste the following commands into the shell one at a time. + Make sure to hit enter after each line. + + :: + + export MOZ_LOG=timestamp,rotate:200,nsHttp:5,cache2:5,nsSocketTransport:5,nsHostResolver:5 + export MOZ_LOG_FILE=/tmp/log.txt + cd /path/to/firefox + ./firefox + +#. Reproduce the problem you're debugging. + +#. When the problem has been reproduced, exit Firefox and look for the + generated log files, which you can find at ``/tmp/log.txt``. + +macOS +~~~~~ + +These instructions show how to log HTTP traffic in Firefox on macOS. + +#. Quit Firefox is if it's currently running, by using the Quit option + in the File menu. Keep in mind that simply closing all windows does + **not** quit Firefox on macOS (this is standard practice for Mac + applications). + +#. Run the Terminal application, which is located in the Utilities + subfolder in your startup disk's Applications folder. + +#. Copy and paste the following commands into the Terminal window, + hitting the return key after each line. + + :: + + export MOZ_LOG=timestamp,rotate:200,nsHttp:5,cache2:5,nsSocketTransport:5,nsHostResolver:5 + export MOZ_LOG_FILE=~/Desktop/log.txt + cd /Applications/Firefox.app/Contents/MacOS + ./firefox + + (The instructions assume that you've installed Firefox directly into + your startup disk's Applications folder. If you've put it elsewhere, + change the path used on the third line appropriately.) + +#. Reproduce whatever problem you're trying to debug. + +#. Quit Firefox and look for the generated ``log.txt`` log files on your + desktop. + +.. note:: + + **Note:** The generated log file uses Unix-style line endings. Older + editors may have problems with this, but if you're using an even + reasonably modern Mac OS X application to view the log, you won't + have any problems. + +Start logging using command line arguments +------------------------------------------ + +Since Firefox 61 it's possible to start logging in a bit simpler way +than setting environment variables: using command line arguments. Here +is an example for the **Windows** platform, on other platforms we accept +the same form of the arguments: + +#. If Firefox is already running, exit out of it. + +#. Open a command prompt. On `Windows + XP <https://commandwindows.com/runline.htm>`__, you can find the + "Run..." command in the Start menu's "All Programs" submenu. On `all + newer versions of + Windows <http://www.xp-vista.com/other/where-is-run-in-windows-vista>`__, + you can hold down the Windows key and press "R". + +#. | Copy and paste the following line into the "Run" command window and + then press enter: + | **For 32-bit Windows:** + + :: + + "c:\Program Files (x86)\Mozilla Firefox\firefox.exe" -MOZ_LOG=timestamp,rotate:200,nsHttp:5,cache2:5,nsSocketTransport:5,nsHostResolver:5 -MOZ_LOG_FILE=%TEMP%\log.txt + + **For 64-bit Windows:** + + :: + + "c:\Program Files\Mozilla Firefox\firefox.exe" -MOZ_LOG=timestamp,rotate:200,nsHttp:5,cache2:5,nsSocketTransport:5,nsHostResolver:5 -MOZ_LOG_FILE=%TEMP%\log.txt + + (These instructions assume that you installed Firefox to the default + location, and that drive C: is your Windows startup disk. Make the + appropriate adjustments if those aren't the case.) + +#. Reproduce whatever problem it is that you're having. + +#. Once you've reproduced the problem, exit Firefox and look for the + generated log files in your temporary directory. You can type + "%TEMP%" directly into the Windows Explorer location bar to get there + quickly. + +Advanced techniques +------------------- + +You can adjust some of the settings listed above to change what HTTP +information get logged. + +Limiting the size of the logged data +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +By default there is no limit to the size of log file(s), and they +capture the logging throughout the time Firefox runs, from start to +finish. These files can get quite large (gigabytes)! So we have added +a 'rotate:SIZE_IN_MB' option to MOZ_LOG (we use it in the examples +above). If you are using Firefox >= 51, setting this option saves only +the last N megabytes of logging data, which helps keep them manageable +in size. (Unknown modules are ignored, so it's OK to use 'rotate' in +your environment even if you're running Firefox <= 50: it will do +nothing). + +This is accomplished by splitting the log into up to 4 separate files +(their filenames have a numbered extension, .0, .1, .2, .3) The logging +back end cycles the files it writes to, while ensuring that the sum of +these files’ sizes will never go over the specified limit. + +Note 1: **the file with the largest number is not guaranteed to be the +last file written!** We don’t move the files, we only cycle. Using the +rotate module automatically adds timestamps to the log, so it’s always +easy to recognize which file keeps the most recent data. + +Note 2: **rotate doesn’t support append**. When you specify rotate, on +every start all the files (including any previous non-rotated log file) +are deleted to avoid any mixture of information. The ``append`` module +specified is then ignored. + +Use 'sync' if your browser crashes or hangs +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +By default, HTTP logging buffers messages and only periodically writes +them to disk (this is more efficient and also makes logging less likely +to interfere with race conditions, etc). However, if you are seeing +your browser crash (or hang) you should add ",sync" to the list of +logging modules in your MOZ_LOG environment variable. This will cause +each log message to be immediately written (and fflush()'d), which is +likely to give us more information about your crash. + +Turning on QUIC logging +~~~~~~~~~~~~~~~~~~~~~~~ + +This can be done by setting `MOZ_LOG` to +`timestamp,rotate:200,nsHttp:5,neqo_http3::*:5,neqo_transport::*:5`. + +Logging only HTTP request and response headers +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +There are two ways to do this: + +#. Replace MOZ_LOG\ ``=nsHttp:5`` with MOZ_LOG\ ``=nsHttp:3`` in the + commands above. +#. There's a handy extension for Firefox called `HTTP Header + Live <https://addons.mozilla.org/firefox/addon/3829>`__ that you can + use to capture just the HTTP request and response headers. This is a + useful tool when you want to peek at HTTP traffic. + +Turning off logging of socket-level transactions +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +If you're not interested in socket-level log information, either because +it's not relevant to your bug or because you're debugging something that +includes a lot of noise that's hard to parse through, you can do that. +Simply remove the text ``nsSocketTransport:5`` from the commands above. + +Turning off DNS query logging +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +You can turn off logging of host resolving (that is, DNS queries) by +removing the text ``nsHostResolver:5`` from the commands above. + +Enable Logging for try server runs +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +You can enable logging on try by passing the `env` argument via `mach try`. +For example: + +.. note:: + + ``./mach try fuzzy --env "MOZ_LOG=nsHttp:5,SSLTokensCache:5"`` + +How to enable QUIC logging +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +The steps to enable QUIC logging (`QLOG <https://datatracker.ietf.org/doc/draft-ietf-quic-qlog-main-schema/>`__) are: + +#. Go to ``about:config``, search for ``network.http.http3.enable_qlog`` and set it to true. +#. Restart Firefox. +#. QLOG files will be saved in the ``qlog_$PID`` directory located within your system's temporary directory. +#. To visualize the QLOG data, visit https://qvis.quictools.info/. You can upload the QLOG files there to see the visual representation of the flows. + +See also +-------- + +- There are similar options available to debug mailnews protocols. + See `this + document <https://www-archive.mozilla.org/quality/mailnews/mail-troubleshoot.html>`__ for + more info about mailnews troubleshooting. +- On the Windows platform, nightly Firefox builds have FTP logging + built-in (don't ask why this is only the case for Windows!). To + enable FTP logging, just set ``MOZ_LOG=nsFtp:5`` (in older versions + of Mozilla, you need to use ``nsFTPProtocol`` instead of ``nsFtp``). +- When Mozilla's built-in logging capabilities aren't good enough, and + you need a full-fledged packet tracing tool, two free products are + `Wireshark <https://www.wireshark.org/>`__ + and `ngrep <https://github.com/jpr5/ngrep/>`__. They are available + for Windows and most flavors of UNIX (including Linux and Mac OS + X), are rock solid, and offer enough features to help uncover any + Mozilla networking problem. diff --git a/netwerk/docs/http_server_for_testing.rst b/netwerk/docs/http_server_for_testing.rst new file mode 100644 index 0000000000..49224e38ad --- /dev/null +++ b/netwerk/docs/http_server_for_testing.rst @@ -0,0 +1,482 @@ +HTTP Server for Unit Tests +========================== + +This page describes the JavaScript implementation of an +HTTP server located in ``netwerk/test/httpserver/``. + +Server Functionality +~~~~~~~~~~~~~~~~~~~~ + +Here are some of the things you can do with the server: + +- map a directory of files onto an HTTP path on the server, for an + arbitrary number of such directories (including nested directories) +- define custom error handlers for HTTP error codes +- serve a given file for requests for a specific path, optionally with + custom headers and status +- define custom "CGI" handlers for specific paths using a + JavaScript-based API to create the response (headers and actual + content) +- run multiple servers at once on different ports (8080, 8081, 8082, + and so on.) + +This functionality should be more than enough for you to use it with any +test which requires HTTP-provided behavior. + +Where You Can Use It +~~~~~~~~~~~~~~~~~~~~ + +The server is written primarily for use from ``xpcshell``-based +tests, and it can be used as an inline script or as an XPCOM component. The +Mochitest framework also uses it to serve its tests, and +`reftests <https://searchfox.org/mozilla-central/source/layout/tools/reftest/README.txt>`__ +can optionally use it when their behavior is dependent upon specific +HTTP header values. + +Ways You Might Use It +~~~~~~~~~~~~~~~~~~~~~ + +- application update testing +- cross-"server" security tests +- cross-domain security tests, in combination with the right proxy + settings (for example, using `Proxy + AutoConfig <https://en.wikipedia.org/wiki/Proxy_auto-config>`__) +- tests where the behavior is dependent on the values of HTTP headers + (for example, Content-Type) +- anything which requires use of files not stored locally +- open-id : the users could provide their own open id server (they only + need it when they're using their browser) +- micro-blogging : users could host their own micro blog based on + standards like RSS/Atom +- rest APIs : web application could interact with REST or SOAP APIs for + many purposes like : file/data storage, social sharing and so on +- download testing + +Using the Server +~~~~~~~~~~~~~~~~ + +The best and first place you should look for documentation is +``netwerk/test/httpserver/nsIHttpServer.idl``. It's extremely +comprehensive and detailed, and it should be enough to figure out how to +make the server do what you want. I also suggest taking a look at the +less-comprehensive server +`README <https://searchfox.org/mozilla-central/source/netwerk/test/httpserver/README>`__, +although the IDL should usually be sufficient. + +Running the Server +^^^^^^^^^^^^^^^^^^ + +From test suites, the server should be importable as a testing-only JS +module: + +.. code:: JavaScript + + ChromeUtils.import("resource://testing-common/httpd.js"); + +Once you've done that, you can create a new server as follows: + +.. code:: JavaScript + + let server = new HttpServer(); // Or nsHttpServer() if you don't use ChromeUtils.import. + + server.registerDirectory("/", nsILocalFileForBasePath); + + server.start(-1); // uses a random available port, allows us to run tests concurrently + const SERVER_PORT = server.identity.primaryPort; // you can use this further on + + // and when the tests are done, most likely from a callback... + server.stop(function() { /* continue execution here */ }); + +You can also pass in a numeric port argument to the ``start()`` method, +but we strongly suggest you don't do it. Using a dynamic port allow us +to run your test in parallel with other tests which reduces wait times +and makes everybody happy. If you really have to use a hardcoded port, +you will have to annotate your test in the xpcshell manifest file with +``run-sequentially = REASON``. +However, this should only be used as the last possible option. + +.. note:: + + You **must** make sure to stop the server (the last line above) + before your test completes. Failure to do so will result in the + "XPConnect is being called on a scope without a Components property" + assertion, which will cause your test to fail in debug builds, and + you'll make people running tests grumbly because you've broken the + tests. + +Debugging Errors +^^^^^^^^^^^^^^^^ + +The server's default error pages don't give much information, partly +because the error-dispatch mechanism doesn't currently accommodate doing +so and partly because exposing errors in a real server could make it +easier to exploit them. If you don't know why the server is acting a +particular way, edit +`httpd.js <https://searchfox.org/mozilla-central/source/netwerk/test/httpserver/httpd.js>`__ +and change the value of ``DEBUG`` to ``true``. This will cause the +server to print information about the processing of requests (and errors +encountered doing so) to the console, and it's usually not difficult to +determine why problems exist from that output. ``DEBUG`` is ``false`` by +default because the information printed with it set to ``true`` +unnecessarily obscures tinderbox output. + +Header Modification for Files +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The server supports modifying the headers of the files (not request +handlers) it serves. To modify the headers for a file, create a sibling +file with the first file's name followed by ``^headers^``. Here's an +example of how such a file might look: + +.. code:: text + + HTTP 404 I want a cool HTTP description! + Content-Type: text/plain + +The status line is optional; all other lines specify HTTP headers in the +standard HTTP format. Any line ending style is accepted, and the file +may optionally end with a single newline character, to play nice with +Unix text tools like ``diff`` and ``hg``. + +Hidden Files +^^^^^^^^^^^^ + +Any file which ends with a single ``^`` is inaccessible when querying +the web server; if you try to access such a file you'll get a +``404 File Not Found`` page instead. If for some reason you need to +serve a file ending with a ``^``, just tack another ``^`` onto the end +of the file name and the file will then become available at the +single-``^`` location. + +At the moment this feature is basically a way to smuggle header +modification for files into the file system without making those files +accessible to clients; it remains to be seen whether and how hidden-file +capabilities will otherwise be used. + +SJS: Server-Side Scripts +^^^^^^^^^^^^^^^^^^^^^^^^ + +Support for server-side scripts is provided through the SJS mechanism. +Essentially an SJS is a file with a particular extension, chosen by the +creator of the server, which contains a function with the name +``handleRequest`` which is called to determine the response the server +will generate. That function acts exactly like the ``handle`` function +on the ``nsIHttpRequestHandler`` interface. First, tell the server what +extension you're using: + +.. code:: JavaScript + + const SJS_EXTENSION = "cgi"; + server.registerContentType(SJS_EXTENSION, "sjs"); + +Now just create an SJS with the extension ``cgi`` and write whatever you +want. For example: + +.. code:: JavaScript + + function handleRequest(request, response) + { + response.setStatusLine(request.httpVersion, 200, "OK"); + response.write("Hello world! This request was dynamically " + + "generated at " + new Date().toUTCString()); + } + +Further examples may be found `in the Mozilla source +tree <https://searchfox.org/mozilla-central/search?q=&path=.sjs>`__ +in existing tests. The request object is an instance of +``nsIHttpRequest`` and the response is a ``nsIHttpResponse``. +Please refer to the `IDL +documentation <https://searchfox.org/mozilla-central/source/netwerk/test/httpserver/nsIHttpServer.idl>` +for more details. + +Storing Information Across Requests +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +HTTP is basically a stateless protocol, and the httpd.js server API is +for the most part similarly stateless. If you're using the server +through the XPCOM interface you can simply store whatever state you want +in enclosing environments or global variables. However, if you're using +it through an SJS your request is processed in a near-empty environment +every time processing occurs. To support stateful SJS behavior, the +following functions have been added to the global scope in which a SJS +handler executes, providing a simple key-value state storage mechanism: + +.. code:: JavaScript + + /* + * v : T means v is of type T + * function A() : T means A() has type T + */ + + function getState(key : string) : string + function setState(key : string, value : string) + function getSharedState(key : string) : string + function setSharedState(key : string, value : string) + function getObjectState(key : string, callback : function(value : object) : void) // SJS API, XPCOM differs, see below + function setObjectState(key : string, value : object) + +A key is a string with arbitrary contents. The corresponding value is +also a string, for the non-object-saving functions. For the +object-saving functions, it is (wait for it) an object, or also +``null``. Initially all keys are associated with the empty string or +with ``null``, depending on whether the function accesses string- or +object-valued storage. A stored value persists across requests and +across server shutdowns and restarts. The state methods are available +both in SJS and, for convenience when working with the server both via +XPCOM and via SJS, XPCOM through the ``nsIHttpServer`` interface. The +variants are designed to support different needs. + +.. warning:: + + **Warning:** Be careful using state: you, the user, are responsible + for synchronizing all uses of state through any of the available + methods. (This includes the methods that act only on per-path state: + you might still run into trouble there if your request handler + generates responses asynchronously. Further, any code with access to + the server XPCOM component could modify it between requests even if + you only ever used or modified that state while generating + synchronous responses.) JavaScript's run-to-completion behavior will + save you in simple cases, but with anything moderately complex you + are playing with fire, and if you do it wrong you will get burned. + +``getState`` and ``setState`` +''''''''''''''''''''''''''''' + +``getState`` and ``setState`` are designed for the case where a single +request handler needs to store information from a first request of it +for use in processing a second request of it — say, for example, if you +wanted to implement a request handler implementing a counter: + +.. code:: JavaScript + + /** + * Generates a response whose body is "0", "1", "2", and so on. each time a + * request is made. (Note that browser caching might make it appear + * to not quite have that behavior; a Cache-Control header would fix + * that issue if desired.) + */ + function handleRequest(request, response) + { + var counter = +getState("counter"); // convert to number; +"" === 0 + response.write("" + counter); + setState("counter", "" + ++counter); + } + +The useful feature of these two methods is that this state doesn't bleed +outside the single path at which it resides. For example, if the above +SJS were at ``/counter``, the value returned by ``getState("counter")`` +at some other path would be completely distinct from the counter +implemented above. This makes it much simpler to write stateful handlers +without state accidentally bleeding between unrelated handlers. + +.. note:: + + State saved by this method is specific to the HTTP path, + excluding query string and hash reference. ``/counter``, + ``/counter?foo``, and ``/counter?bar#baz`` all share the same state + for the purposes of these methods. (Indeed, non-shared state would be + significantly less useful if it changed when the query string + changed!) + +.. note:: + + The predefined ``__LOCATION__`` state + contains the native path of the SJS file itself. You can pass the + result directly to the ``nsILocalFile.initWithPath()``. Example: + ``thisSJSfile.initWithPath(getState('__LOCATION__'));`` + +``getSharedState`` and ``setSharedState`` +''''''''''''''''''''''''''''''''''''''''' + +``getSharedState`` and ``setSharedState`` make up the functionality +intentionally not supported by ``getState`` and set\ ``State``: state +that exists between different paths. If you used the above handler at +the paths ``/sharedCounters/1`` and ``/sharedCounters/2`` (changing the +state-calls to use shared state, of course), the first load of either +handler would return "0", a second load of either handler would return +"1", a third load either handler would return "2", and so on. This more +powerful functionality allows you to write cooperative handlers that +expose and manipulate a piece of shared state. Be careful! One test can +screw up another test pretty easily if it's not careful what it does +with this functionality. + +``getObjectState`` and ``setObjectState`` +''''''''''''''''''''''''''''''''''''''''' + +``getObjectState`` and ``setObjectState`` support the remaining +functionality not provided by the above methods: storing non-string +values (object values or ``null``). These two methods are the same as +``getSharedState`` and ``setSharedState``\ in that state is visible +across paths; ``setObjectState`` in one handler will expose that value +in another handler that uses ``getObjectState`` with the same key. (This +choice was intentional, because object values already expose mutable +state that you have to be careful about using.) This functionality is +particularly useful for cooperative request handlers where one request +*suspends* another, and that second request must then be *resumed* at a +later time by a third request. Without object-valued storage you'd need +to resort to polling on a string value using either of the previous +state APIs; with this, however, you can make precise callbacks exactly +when a particular event occurs. + +``getObjectState`` in an SJS differs in one important way from +``getObjectState`` accessed via XPCOM. In XPCOM the method takes a +single string argument and returns the object or ``null`` directly. In +SJS, however, the process to return the value is slightly different: + +.. code:: JavaScript + + function handleRequest(request, response) + { + var key = request.hasHeader("key") + ? request.getHeader("key") + : "unspecified"; + var obj = null; + getObjectState(key, function(objval) + { + // This function is called synchronously with the object value + // associated with key. + obj = objval; + }); + response.write("Keyed object " + + (obj && Object.prototype.hasOwnProperty.call(obj, "doStuff") + ? "has " + : "does not have ") + + "a doStuff method."); + } + +This idiosyncratic API is a restriction imposed by how sandboxes +currently work: external functions added to the sandbox can't return +object values when called within the sandbox. However, such functions +can accept and call callback functions, so we simply use a callback +function here to return the object value associated with the key. + +Advanced Dynamic Response Creation +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The default behavior of request handlers is to fully construct the +response, return, and only then send the generated data. For certain use +cases, however, this is infeasible. For example, a handler which wanted +to return an extremely large amount of data (say, over 4GB on a 32-bit +system) might run out of memory doing so. Alternatively, precise control +over the timing of data transmission might be required so that, for +example, one request is received, "paused" while another request is +received and completes, and then finished. httpd.js solves this problem +by defining a ``processAsync()`` method which indicates to the server +that the response will be written and finished by the handler. Here's an +example of an SJS file which writes some data, waits five seconds, and +then writes some more data and finishes the response: + +.. code:: JavaScript + + var timer = null; + + function handleRequest(request, response) + { + response.processAsync(); + response.setHeader("Content-Type", "text/plain", false); + response.write("hello..."); + + timer = Cc["@mozilla.org/timer;1"].createInstance(Ci.nsITimer); + timer.initWithCallback(function() + { + response.write("world!"); + response.finish(); + }, 5 * 1000 /* milliseconds */, Ci.nsITimer.TYPE_ONE_SHOT); + } + +The basic flow is simple: call ``processAsync`` to mark the response as +being sent asynchronously, write data to the response body as desired, +and when complete call ``finish()``. At the moment if you drop such a +response on the floor, nothing will ever terminate the connection, and +the server cannot be stopped (the stop API is asynchronous and +callback-based); in the future a default connection timeout will likely +apply, but for now, "don't do that". + +Full documentation for ``processAsync()`` and its interactions with +other methods may, as always, be found in +``netwerk/test/httpserver/nsIHttpServer.idl``. + +Manual, Arbitrary Response Creation +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The standard mode of response creation is fully synchronous and is +guaranteed to produce syntactically correct responses (excluding +headers, which for the most part may be set to arbitrary values). +Asynchronous processing enables the introduction of response handling +coordinated with external events, but again, for the most part only +syntactically correct responses may be generated. The third method of +processing removes the correct-syntax property by allowing a response to +contain completely arbitrary data through the ``seizePower()`` method. +After this method is called, any data subsequently written to the +response is written directly to the network as the response, skipping +headers and making no attempt whatsoever to ensure any formatting of the +transmitted data. As with asynchronous processing, the response is +generated asynchronously and must be finished manually for the +connection to be closed. (Again, nothing will terminate the connection +for a response dropped on the floor, so again, "don't do that".) This +mode of processing is useful for testing particular data formats that +are either not HTTP or which do not match the precise, canonical +representation that httpd.js generates. Here's an example of an SJS file +which writes an apparent HTTP response whose status text contains a null +byte (not allowed by HTTP/1.1, and attempting to set such status text +through httpd.js would throw an exception) and which has a header that +spans multiple lines (httpd.js responses otherwise generate only +single-line headers): + +.. code:: JavaScript + + function handleRequest(request, response) + { + response.seizePower(); + response.write("HTTP/1.1 200 OK Null byte \u0000 makes this response malformed\r\n" + + "X-Underpants-Gnomes-Strategy:\r\n" + + " Phase 1: Collect underpants.\r\n" + + " Phase 2: ...\r\n" + + " Phase 3: Profit!\r\n" + + "\r\n" + + "FAIL"); + response.finish(); + } + +While the asynchronous mode is capable of producing certain forms of +invalid responses (through setting a bogus Content-Length header prior +to the start of body transmission, among others), it must not be used in +this manner. No effort will be made to preserve such implementation +quirks (indeed, some are even likely to be removed over time): if you +want to send malformed data, use ``seizePower()`` instead. + +Full documentation for ``seizePower()`` and its interactions with other +methods may, as always, be found in +``netwerk/test/httpserver/nsIHttpServer.idl``. + +Example Uses of the Server +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Shorter examples (for tests which only do one test): + +- ``netwerk/test/unit/test_bug331825.js`` +- ``netwerk/test/unit/test_httpcancel.js`` +- ``netwerk/test/unit/test_cookie_header.js`` + +Longer tests (where you'd need to do multiple async server requests): + +- ``netwerk/test/httpserver/test/test_setstatusline.js`` +- ``netwerk/test/unit/test_content_sniffer.js`` +- ``netwerk/test/unit/test_authentication.js`` +- ``netwerk/test/unit/test_event_sink.js`` +- ``netwerk/test/httpserver/test/`` + +Examples of modifying HTTP headers in files may be found at +``netwerk/test/httpserver/test/data/cern_meta/``. + +Future Directions +~~~~~~~~~~~~~~~~~ + +The server, while very functional, is not yet complete. There are a +number of things to fix and features to add, among them support for +pipelining, support for incrementally-received requests (rather than +buffering the entire body before invoking a request handler), and better +conformance to the MUSTs and SHOULDs of HTTP/1.1. If you have +suggestions for functionality or find bugs, file them in +`Testing-httpd.js <https://bugzilla.mozilla.org/enter_bug.cgi?product=Testing&component=General>`__ +. |