1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
|
[![Rust][Rust Badge]][Rust CI Link]
[![NotImplemented Counter][NotImplemented Badge]][NotImplemented Search]
[![Fuzzbug days since][Fuzzbug Days Badge]][Fuzzbugs]
[![Fuzzbug open][Fuzzbug Open Badge]][Open Fuzzbugs]
[![SmooshMonkey Build Result][SmooshMonkey Build Badge]][SmooshMonkey Build TreeHerder]
[![SmooshMonkey Test Result][SmooshMonkey Test Badge]][SmooshMonkey Test TreeHerder]
# jsparagus - A JavaScript parser written in Rust
jsparagus is intended to replace the JavaScript parser in Firefox.
Current status:
* jsparagus is not on crates.io yet. The AST design is not stable
enough. We do have a build of the JS shell that includes jsparagus
as an option (falling back on C++ for features jsparagus doesn't
support). See
[mozilla-spidermonkey/rust-frontend](https://github.com/mozilla-spidermonkey/rust-frontend).
* It can parse a lot of JS scripts, and will eventually be able to parse everything.
See the current limitations below, or our GitHub issues.
* Our immediate goal is to [support parsing everything in Mozilla's JS
test suite and the features in test262 that Firefox already
supports](https://github.com/mozilla-spidermonkey/jsparagus/milestone/1).
Join us on Discord: https://discord.gg/tUFFk9Y
## Building jsparagus
To build the parser by itself:
```sh
make init
make all
```
The build takes about 3 minutes to run on my laptop.
When it's done, you can:
* Run `make check` to make sure things are working.
* `cd crates/driver && cargo run -- -D` to try out the JS parser and bytecode emitter.
## Building and running SpiderMonkey with jsparagus
* To build SpiderMonkey with jsparagus, `configure` with `--enable-smoosh`.
This builds with a specific known-good revision of jsparagus.
* Building SpiderMonkey with your own local jsparagus repo, for
development, takes more work; see [the jsparagus + SpiderMonkey wiki
page](https://github.com/mozilla-spidermonkey/jsparagus/wiki/SpiderMonkey)
for details.
**NOTE: Even after building with jsparagus, you must run the shell with
`--smoosh`** to enable jsparagus at run time.
## Benchmarking
### Fine-grain Benchmarks
Fine-grain benchmarks are used to detect regression by focusing on each part of
the parser at one time, exercising only this one part. The benchmarks are not
meant to represent any real code sample, but to focus on executing specific
functions of the parser.
To run this parser, you should execute the following command at the root of the
repository:
```sh
cd crates/parser
cargo bench
```
### Real-world JavaScript
Real world benchmarks are used to track the overall evolution of performance over
time. The benchmarks are meant to represent realistic production use cases.
To benchmark the AST generation, we use SpiderMonkey integration to execute the
parser and compare it against SpiderMonkey's default parser. Therefore, to run
this benchmark, we have to first compile SpiderMonkey, then execute SpiderMonkey
shell on the benchmark. (The following instructions assume that `~` is the
directory where all projects are checked out)
* Generate Parse Tables:
```sh
cd ~/jsparagus/
make init
make all
```
* Compile an optimized version of [SpiderMonkey's JavaScript shell](https://github.com/mozilla/gecko-dev):
```sh
cd ~/mozilla/js/src/
# set the jsparagus' path to the abosulte path to ~/jsparagus.
$EDITOR frontend/smoosh/Cargo.toml
../../mach vendor rust
# Create a build directory
mkdir obj.opt
cd obj.opt
# Build SpiderMonkey
../configure --enable-nspr-build --enable-smoosh --enable-debug-symbols=-ggdb3 --disable-debug --enable-optimize --enable-release --disable-tests
make
```
* Execute the [real-js-samples](https://github.com/nbp/real-js-samples/) benchmark:
```sh
cd ~/real-js-samples/
~/mozilla/js/src/obj.opt/dist/bin/js ./20190416.js
```
This should return the overall time taken to parse all the Script once, in the
cases where there is no error. The goal is to minimize the number of
nano-seconds per bytes.
## Limitations
It's *all* limitations, but I'll try to list the ones that are relevant
to parsing JS.
* Features that are not implemented in the parser yet include `let`,
`import` and `export`, `async` functions, `yield` expressions, the
use of `await` and `yield` as identifiers, template strings,
`BigInt`, Unicode escape sequences that evaluate to surrogate code
points, legacy octal integer literals, legacy octal escape
sequences, some RegExp flags, strict mode code, `__proto__` in
object literals, some features of destructuring assignment.
Many more features are not yet supported in the bytecode emitter.
* Error messages are poor.
We're currently working on parser performance and completeness, as well
as the bytecode emitter and further integration with SpiderMonkey.
[Rust Badge]: https://github.com/mozilla-spidermonkey/jsparagus/workflows/Rust/badge.svg
[Rust CI Link]: https://github.com/mozilla-spidermonkey/jsparagus/actions?query=branch%3Amaster
[NotImplemented Badge]: https://img.shields.io/endpoint?url=https%3A%2F%2Fraw.githubusercontent.com%2Fmozilla-spidermonkey%2Fjsparagus%2Fci_results%2F.metrics%2Fbadges%2Fnot-implemented.json
[NotImplemented Search]: https://github.com/mozilla-spidermonkey/jsparagus/search?q=notimplemented&unscoped_q=notimplemented
[Fuzzbug days Badge]: https://img.shields.io/endpoint?url=https%3A%2F%2Fraw.githubusercontent.com%2Fmozilla-spidermonkey%2Fjsparagus%2Fci_results%2F.metrics%2Fbadges%2Fsince-last-fuzzbug.json
[Fuzzbug Open Badge]: https://img.shields.io/endpoint?url=https%3A%2F%2Fraw.githubusercontent.com%2Fmozilla-spidermonkey%2Fjsparagus%2Fci_results%2F.metrics%2Fbadges%2Fopen-fuzzbug.json
[Fuzzbugs]: https://github.com/mozilla-spidermonkey/jsparagus/issues?utf8=%E2%9C%93&q=label%3AlibFuzzer+
[Open Fuzzbugs]: https://github.com/mozilla-spidermonkey/jsparagus/labels/libFuzzer
[SmooshMonkey Build Badge]: https://img.shields.io/endpoint?url=https%3A%2F%2Fraw.githubusercontent.com%2Fmozilla-spidermonkey%2Fjsparagus%2Fci_smoosh_status%2Fsmoosh_build.json
[SmooshMonkey Build TreeHerder]: https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&tier=1%2C2%2C3&searchStr=sm-nonunified
[SmooshMonkey Test Badge]: https://img.shields.io/endpoint?url=https%3A%2F%2Fraw.githubusercontent.com%2Fmozilla-spidermonkey%2Fjsparagus%2Fci_smoosh_status%2Fsmoosh_test.json
[SmooshMonkey Test TreeHerder]: https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&tier=1%2C2%2C3&searchStr=sm-smoosh
|
