[![Rust][Rust Badge]][Rust CI Link] [![NotImplemented Counter][NotImplemented Badge]][NotImplemented Search] [![Fuzzbug days since][Fuzzbug Days Badge]][Fuzzbugs] [![Fuzzbug open][Fuzzbug Open Badge]][Open Fuzzbugs] [![SmooshMonkey Build Result][SmooshMonkey Build Badge]][SmooshMonkey Build TreeHerder] [![SmooshMonkey Test Result][SmooshMonkey Test Badge]][SmooshMonkey Test TreeHerder] # jsparagus - A JavaScript parser written in Rust jsparagus is intended to replace the JavaScript parser in Firefox. Current status: * jsparagus is not on crates.io yet. The AST design is not stable enough. We do have a build of the JS shell that includes jsparagus as an option (falling back on C++ for features jsparagus doesn't support). See [mozilla-spidermonkey/rust-frontend](https://github.com/mozilla-spidermonkey/rust-frontend). * It can parse a lot of JS scripts, and will eventually be able to parse everything. See the current limitations below, or our GitHub issues. * Our immediate goal is to [support parsing everything in Mozilla's JS test suite and the features in test262 that Firefox already supports](https://github.com/mozilla-spidermonkey/jsparagus/milestone/1). Join us on Discord: https://discord.gg/tUFFk9Y ## Building jsparagus To build the parser by itself: ```sh make init make all ``` The build takes about 3 minutes to run on my laptop. When it's done, you can: * Run `make check` to make sure things are working. * `cd crates/driver && cargo run -- -D` to try out the JS parser and bytecode emitter. ## Building and running SpiderMonkey with jsparagus * To build SpiderMonkey with jsparagus, `configure` with `--enable-smoosh`. This builds with a specific known-good revision of jsparagus. * Building SpiderMonkey with your own local jsparagus repo, for development, takes more work; see [the jsparagus + SpiderMonkey wiki page](https://github.com/mozilla-spidermonkey/jsparagus/wiki/SpiderMonkey) for details. **NOTE: Even after building with jsparagus, you must run the shell with `--smoosh`** to enable jsparagus at run time. ## Benchmarking ### Fine-grain Benchmarks Fine-grain benchmarks are used to detect regression by focusing on each part of the parser at one time, exercising only this one part. The benchmarks are not meant to represent any real code sample, but to focus on executing specific functions of the parser. To run this parser, you should execute the following command at the root of the repository: ```sh cd crates/parser cargo bench ``` ### Real-world JavaScript Real world benchmarks are used to track the overall evolution of performance over time. The benchmarks are meant to represent realistic production use cases. To benchmark the AST generation, we use SpiderMonkey integration to execute the parser and compare it against SpiderMonkey's default parser. Therefore, to run this benchmark, we have to first compile SpiderMonkey, then execute SpiderMonkey shell on the benchmark. (The following instructions assume that `~` is the directory where all projects are checked out) * Generate Parse Tables: ```sh cd ~/jsparagus/ make init make all ``` * Compile an optimized version of [SpiderMonkey's JavaScript shell](https://github.com/mozilla/gecko-dev): ```sh cd ~/mozilla/js/src/ # set the jsparagus' path to the abosulte path to ~/jsparagus. $EDITOR frontend/smoosh/Cargo.toml ../../mach vendor rust # Create a build directory mkdir obj.opt cd obj.opt # Build SpiderMonkey ../configure --enable-nspr-build --enable-smoosh --enable-debug-symbols=-ggdb3 --disable-debug --enable-optimize --enable-release --disable-tests make ``` * Execute the [real-js-samples](https://github.com/nbp/real-js-samples/) benchmark: ```sh cd ~/real-js-samples/ ~/mozilla/js/src/obj.opt/dist/bin/js ./20190416.js ``` This should return the overall time taken to parse all the Script once, in the cases where there is no error. The goal is to minimize the number of nano-seconds per bytes. ## Limitations It's *all* limitations, but I'll try to list the ones that are relevant to parsing JS. * Features that are not implemented in the parser yet include `let`, `import` and `export`, `async` functions, `yield` expressions, the use of `await` and `yield` as identifiers, template strings, `BigInt`, Unicode escape sequences that evaluate to surrogate code points, legacy octal integer literals, legacy octal escape sequences, some RegExp flags, strict mode code, `__proto__` in object literals, some features of destructuring assignment. Many more features are not yet supported in the bytecode emitter. * Error messages are poor. We're currently working on parser performance and completeness, as well as the bytecode emitter and further integration with SpiderMonkey. [Rust Badge]: https://github.com/mozilla-spidermonkey/jsparagus/workflows/Rust/badge.svg [Rust CI Link]: https://github.com/mozilla-spidermonkey/jsparagus/actions?query=branch%3Amaster [NotImplemented Badge]: https://img.shields.io/endpoint?url=https%3A%2F%2Fraw.githubusercontent.com%2Fmozilla-spidermonkey%2Fjsparagus%2Fci_results%2F.metrics%2Fbadges%2Fnot-implemented.json [NotImplemented Search]: https://github.com/mozilla-spidermonkey/jsparagus/search?q=notimplemented&unscoped_q=notimplemented [Fuzzbug days Badge]: https://img.shields.io/endpoint?url=https%3A%2F%2Fraw.githubusercontent.com%2Fmozilla-spidermonkey%2Fjsparagus%2Fci_results%2F.metrics%2Fbadges%2Fsince-last-fuzzbug.json [Fuzzbug Open Badge]: https://img.shields.io/endpoint?url=https%3A%2F%2Fraw.githubusercontent.com%2Fmozilla-spidermonkey%2Fjsparagus%2Fci_results%2F.metrics%2Fbadges%2Fopen-fuzzbug.json [Fuzzbugs]: https://github.com/mozilla-spidermonkey/jsparagus/issues?utf8=%E2%9C%93&q=label%3AlibFuzzer+ [Open Fuzzbugs]: https://github.com/mozilla-spidermonkey/jsparagus/labels/libFuzzer [SmooshMonkey Build Badge]: https://img.shields.io/endpoint?url=https%3A%2F%2Fraw.githubusercontent.com%2Fmozilla-spidermonkey%2Fjsparagus%2Fci_smoosh_status%2Fsmoosh_build.json [SmooshMonkey Build TreeHerder]: https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&tier=1%2C2%2C3&searchStr=sm-nonunified [SmooshMonkey Test Badge]: https://img.shields.io/endpoint?url=https%3A%2F%2Fraw.githubusercontent.com%2Fmozilla-spidermonkey%2Fjsparagus%2Fci_smoosh_status%2Fsmoosh_test.json [SmooshMonkey Test TreeHerder]: https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&tier=1%2C2%2C3&searchStr=sm-smoosh