1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
|
# Implementing specifications using WHATWG Streams API
[Streams API](https://streams.spec.whatwg.org/) is [a modern way to produce and consume data progressively and asynchronously](https://developer.mozilla.org/en-US/docs/Web/API/Streams_API). Multiple specifications are starting to use it, namely [Fetch](https://fetch.spec.whatwg.org/), [File Stream](https://fs.spec.whatwg.org/), [WebTransport](https://w3c.github.io/webtransport/), and so on. This documentation will briefly explain how to implement such specifications in Gecko.
## Calling functions on stream objects
You can mostly follow the steps in a given spec as-is, as the implementation in Gecko is deliberately written in a way that a given spec prose can match 1:1 to a function call. Let's say the spec says:
> [Enqueue](https://streams.spec.whatwg.org/#readablestream-enqueue) `view` into `stream`.
The prose can be written in C++ as:
```cpp
stream->EnqueueNative(cx, view, rv);
```
Note that the function name ends with `Native` to disambiguate itself from the Web IDL `enqueue` method. See the [list below](#mapping-spec-terms-to-functions) for the complete mapping between spec terms and functions.
## Creating a stream
The stream creation can be generally done by calling `CreateNative()`. You may need to call something else if the spec:
* Wants a byte stream and uses the term "Set up with byte reading support". In that case you need to call `ByteNative` variant.
* Defines a new interface that inherits the base stream interfaces. In this case you need to define a subclass and call `SetUpNative()` inside its init method.
* To make the cycle collection happy, you need to pass `HoldDropJSObjectsCaller::Explicit` to the superclass constructor and call `mozilla::HoldJSObjects(this)`/`mozilla::DropJSObjects(this)` respectively in the constructor/destructor.
Both `CreateNative()`/`SetUpNative()` functions require an argument to implement custom algorithms for callbacks, whose corresponding spec phrases could be:
> 1. Let `readable` be a [new](https://webidl.spec.whatwg.org/#new) [`ReadableStream`](https://streams.spec.whatwg.org/#readablestream).
> 1. Let `pullAlgorithm` be the following steps:
> 1. (...)
> 1. Set up `stream` with `pullAlgorithm` set to `pullAlgorithm`.
This can roughly translate to the following C++:
```cpp
class MySourceAlgorithms : UnderlyingSourceAlgorithmsWrapper {
already_AddRefed<Promise> PullCallbackImpl(
JSContext* aCx, ReadableStreamController& aController,
ErrorResult& aRv) override;
};
already_AddRefed<ReadableStream> CreateMyReadableStream(
JSContext* aCx, nsIGlobalObject* aGlobal, ErrorResult& aRv) {
// Step 2: Let pullAlgorithm be the following steps:
auto algorithms = MakeRefPtr<MySourceAlgorithms>();
// Step 1: Let readable be a new ReadableStream.
// Step 3: Set up stream with pullAlgorithm set to pullAlgorithm.
RefPtr<ReadableStream> readable = ReadableStream::CreateNative(
aCx,
aGlobal,
*algorithms,
/* aHighWaterMark */ Nothing(),
/* aSizeAlgorithm */ nullptr,
aRv
);
}
```
Note that the `new ReadableStream()` and "Set up" steps are done together inside `CreateNative()` for convenience. For subclasses this needs to be split again:
```cpp
class MyReadableStream : public ReadableStream {
public:
MyReadableStream(nsIGlobalObject* aGlobal)
: ReadableStream(aGlobal, ReadableStream::HoldDropJSObjectsCaller::Explicit) {
mozilla::HoldJSObjects(this);
}
~MyReadableStream() {
mozilla::DropJSObjects(this);
}
void Init(ErrorResult& aRv) {
// Step 2: Let pullAlgorithm be the following steps:
auto algorithms = MakeRefPtr<MySourceAlgorithms>();
// Step 3: Set up stream with pullAlgorithm set to pullAlgorithm.
//
// NOTE:
// For now there's no SetUpNative but only SetUpByteNative.
// File a bug on DOM: Streams if you need to create a subclass
// for non-byte ReadableStream.
SetUpNative(aCx, *algorithms, Nothing(), nullptr, aRv);
}
}
```
After creating the stream with the algorithms, the rough flow will look like this:
```{mermaid}
sequenceDiagram
JavaScript->>ReadableStream: await reader.read()
ReadableStream->>UnderlyingSourceAlgorithmsWrapper: PullCallback()
UnderlyingSourceAlgorithmsWrapper->>(Data source): (implementation detail)
NOTE left of (Data source): (Can be file IO, network IO, etc.)
(Data source)->>UnderlyingSourceAlgorithmsWrapper: (notifies back)
UnderlyingSourceAlgorithmsWrapper->>ReadableStream: EnqueueNative()
ReadableStream->>JavaScript: Resolves reader.read()
```
### Implementing the callbacks
As the flow says, the real implementation will be done inside the algorithms, in this case PullCallbackImpl(). Let's say there's a spec term:
> 1. Let `pullAlgorithm` be the following steps:
> 1. [Enqueue](https://streams.spec.whatwg.org/#readablestream-enqueue) a JavaScript string value "Hello Fox!".
This can translate to the following C++:
```cpp
class MySourceAlgorithms : UnderlyingSourceAlgorithmsWrapper {
// Step 1: Let `pullAlgorithm` be the following steps:
already_AddRefed<Promise> PullCallbackImpl(
JSContext* aCx, ReadableStreamController& aController, ErrorResult& aRv) {
RefPtr<ReadableStream> stream = aController.Stream();
// Step 1.1: Enqueue a JavaScript string value "Hello Fox!".
JS::Rooted<JSString*> hello(aCx, JS_NewStringCopyZ(aCx, "Hello Fox!"));
stream->EnqueueNative(aCx, JS::StringValue(hello), aRv);
// Return a promise if the task is asynchronous, or nullptr if not.
return nullptr;
// NOTE:
// Please don't use aController directly, as it's more for JavaScript.
// The *Native() functions are safer with additional assertions and more
// automatic state management.
// Please file a bug if there's no *Native() function that fulfills your need.
// In the future this function should receive a ReadableStream instead.
// Also note that you'll need to touch JS APIs frequently as the functions
// often expect JS::Value.
};
};
```
Note that `PullCallbackImpl` returns a promise. The function will not be called again until the promise resolves. The call sequence would be roughly look like the following with repeated read requests:
1. `await read()` from JS
1. `PullCallbackImpl()` call, which returns a Promise
1. The second `await read()` from JS
1. (Time flies)
1. The promise resolves
1. The second `PullCallbackImpl()` call
The same applies to write and transform callbacks in `WritableStream` and `TransformStream`, except they use `UnderlyingSinkAlgorithmsWrapper` and `TransformerAlgorithmsWrapper` respectively.
## Exposing existing XPCOM streams as WHATWG Streams
You may simply want to expose an existing XPCOM stream to JavaScript without any more customization. Fortunately there are some helper functions for this. You can use:
* `InputToReadableStreamAlgorithms` to send data from nsIAsyncInputStream to ReadableStream
* `WritableStreamToOutputAlgorithms` to receive data from WritableStream to nsIAsyncOutputStream
The usage would look like the following:
```cpp
// For nsIAsyncInputStream:
already_AddRefed<ReadableStream> ConvertInputStreamToReadableStream(
JSContext* aCx, nsIGlobalObject* aGlobal, nsIAsyncInputStream* aInput,
ErrorResult& aRv) {
auto algorithms = MakeRefPtr<InputToReadableStreamAlgorithms>(
stream->GetParentObject(), aInput);
return do_AddRef(ReadableStream::CreateNative(aCx, aGlobal, *algorithms,
Nothing(), nullptr, aRv));
}
// For nsIAsyncOutputStream
already_AddRefed<ReadableStream> ConvertOutputStreamToWritableStream(
JSContext* aCx, nsIGlobalObject* aGlobal, nsIAsyncOutputStream* aInput,
ErrorResult& aRv) {
auto algorithms = MakeRefPtr<WritableStreamToOutputAlgorithms>(
stream->GetParentObject(), aInput);
return do_AddRef(WritableStream::CreateNative(aCx, aGlobal, *algorithms,
Nothing(), nullptr, aRv));
}
```
## Mapping spec terms to functions
1. [ReadableStream](https://streams.spec.whatwg.org/#other-specs-rs)
* [Set up](https://streams.spec.whatwg.org/#readablestream-set-up): [`CreateNative()`](https://searchfox.org/mozilla-central/rev/31f5847a4494b3646edabbdd7ea39cb88509afe2/dom/streams/ReadableStream.h#132)
* [Set up with byte reading support](https://streams.spec.whatwg.org/#readablestream-set-up-with-byte-reading-support):
- [`CreateByteNative()`](https://searchfox.org/mozilla-central/rev/31f5847a4494b3646edabbdd7ea39cb88509afe2/dom/streams/ReadableStream.h#143): You can call this when the spec uses the term with `new ReadableStream`.
- [`SetUpByteNative()`](https://searchfox.org/mozilla-central/rev/31f5847a4494b3646edabbdd7ea39cb88509afe2/dom/streams/ReadableStream.h#150): You need to use this instead when the spec uses the term with a subclass of `ReadableStream`. Call this inside the constructor of the subclass.
* [Close](https://streams.spec.whatwg.org/#readablestream-close): [`CloseNative()`](https://searchfox.org/mozilla-central/rev/31f5847a4494b3646edabbdd7ea39cb88509afe2/dom/streams/ReadableStream.h#160)
* [Error](https://streams.spec.whatwg.org/#readablestream-error): [`ErrorNative()`](https://searchfox.org/mozilla-central/rev/31f5847a4494b3646edabbdd7ea39cb88509afe2/dom/streams/ReadableStream.h#163)
* [Enqueue](https://streams.spec.whatwg.org/#readablestream-enqueue): [`EnqueueNative()`](https://searchfox.org/mozilla-central/rev/31f5847a4494b3646edabbdd7ea39cb88509afe2/dom/streams/ReadableStream.h#167)
* [Get a reader](https://streams.spec.whatwg.org/#readablestream-get-a-reader): [`GetReader()`](https://searchfox.org/mozilla-central/rev/31f5847a4494b3646edabbdd7ea39cb88509afe2/dom/streams/ReadableStream.h#177)
* [Read a chunk](https://streams.spec.whatwg.org/#readablestreamdefaultreader-read-a-chunk) on reader: [`ReadChunk()`](https://searchfox.org/mozilla-central/rev/31f5847a4494b3646edabbdd7ea39cb88509afe2/dom/streams/ReadableStreamDefaultReader.h#81) on ReadableStreamDefaultReader
2. [WritableStream](https://streams.spec.whatwg.org/#other-specs-ws)
* [Set up](https://streams.spec.whatwg.org/#writablestream-set-up):
- [`CreateNative()`](https://searchfox.org/mozilla-central/rev/31f5847a4494b3646edabbdd7ea39cb88509afe2/dom/streams/WritableStream.h#182): You can call this when the spec uses the term with `new WritableStream`.
- [`SetUpNative()`](https://searchfox.org/mozilla-central/rev/31f5847a4494b3646edabbdd7ea39cb88509afe2/dom/streams/WritableStream.h#174): You need to use this instead when the spec uses the term with a subclass of `WritableStream`. Call this inside the constructor of the subclass.
* [Error](https://streams.spec.whatwg.org/#writablestream-error): [`ErrorNative()`](https://searchfox.org/mozilla-central/rev/31f5847a4494b3646edabbdd7ea39cb88509afe2/dom/streams/WritableStream.h#192)
3. [TransformStream](https://streams.spec.whatwg.org/#other-specs-ts): For now this just uses the functions in TransfromStreamDefaultController, which will be provided as an argument of transform or flush algorithms.
* [Enqueue](https://streams.spec.whatwg.org/#transformstream-enqueue): [`Enqueue()`](https://searchfox.org/mozilla-central/rev/31f5847a4494b3646edabbdd7ea39cb88509afe2/dom/streams/TransformStreamDefaultController.h#47) on TransformStreamDefaultController
* [Terminate](https://streams.spec.whatwg.org/#transformstream-terminate): [`Terminate()`](https://searchfox.org/mozilla-central/rev/31f5847a4494b3646edabbdd7ea39cb88509afe2/dom/streams/TransformStreamDefaultController.h#51) on TransformStreamDefaultController
* [Error](https://streams.spec.whatwg.org/#transformstream-error): [`Error()`](https://searchfox.org/mozilla-central/rev/31f5847a4494b3646edabbdd7ea39cb88509afe2/dom/streams/TransformStreamDefaultController.h#49) on TransformStreamDefaultController
The mapping is only implemented on demand and does not cover every function in the spec. Please file a bug on [DOM: Streams](https://bugzilla.mozilla.org/describecomponents.cgi?product=Core&component=DOM%3A%20Streams#DOM%3A%20Streams) component in Bugzilla if you need something that is missing here.
|
