summaryrefslogtreecommitdiffstats
path: root/README.md
diff options
context:
space:
mode:
Diffstat (limited to '')
-rw-r--r--README.md432
1 files changed, 432 insertions, 0 deletions
diff --git a/README.md b/README.md
new file mode 100644
index 0000000..4722445
--- /dev/null
+++ b/README.md
@@ -0,0 +1,432 @@
+# zmodem.js - ZMODEM for JavaScript
+
+[![build status](https://api.travis-ci.org/FGasper/zmodemjs.svg?branch=master)](http://travis-ci.org/FGasper/zmodemjs)
+
+# SYNOPSIS
+
+ let zsentry = new Zmodem.Sentry( {
+ to_terminal(octets) { .. }, //i.e. send to the terminal
+
+ sender(octets) { .. }, //i.e. send to the ZMODEM peer
+
+ on_detect(detection) { .. }, //for when Sentry detects a new ZMODEM
+
+ on_retract() { .. }, //for when Sentry retracts a Detection
+ } );
+
+ //We have to configure whatever gives us new input to send that
+ //input to zsentry.
+ //
+ //EXAMPLE: From web browsers that use WebSocket …
+ //
+ ws.addEventListener("message", function(evt) {
+ zsentry.consume(evt.data);
+ } );
+
+The `on_detect(detection)` function call is probably the most complex
+piece of the above; one potential implementation might look like:
+
+ on_detect(detection) {
+
+ //Do this if we determine that what looked like a ZMODEM session
+ //is actually not meant to be ZMODEM.
+ if (no_good) {
+ detection.deny();
+ return;
+ }
+
+ zsession = detection.confirm();
+
+ if (zsession.type === "send") {
+
+ //Send a group of files, e.g., from an <input>’s “.files”.
+ //There are events you can listen for here as well,
+ //e.g., to update a progress meter.
+ Zmodem.Browser.send_files( zsession, files_obj );
+ }
+ else {
+ zsession.on("offer", (xfer) => {
+
+ //Do this if you don’t want the offered file.
+ if (no_good) {
+ xfer.skip();
+ return;
+ }
+
+ xfer.accept().then( () => {
+
+ //Now you need some mechanism to save the file.
+ //An example of how you can do this in a browser:
+ Zmodem.Browser.save_to_disk(
+ xfer.get_payloads(),
+ xfer.get_details().name
+ );
+ } );
+ });
+
+ zsession.start();
+ }
+ }
+
+# DESCRIPTION
+
+zmodem.js is a JavaScript implementation of the ZMODEM
+file transfer protocol, which facilitates file transfers via a terminal.
+
+# STATUS
+
+This library is BETA quality. It should be safe for general use, but
+breaking changes may still happen.
+
+# HOW TO USE THIS LIBRARY
+
+The basic workflow is:
+
+1. Create a `Zmodem.Sentry` object. This object must scan all input for
+a ZMODEM initialization string. See `zsentry.js`’s documentation for more
+details.
+
+2. Once that initialization is found, the `on_detect` event is fired
+with a `Detection` object as parameter. At this point you can `deny()`
+that Detection or `confirm()` it; the latter will return a Session
+object.
+
+3. Now you do the actual file transfer(s):
+
+ * If the session is a receive session, do something like this:
+
+ zsession.on("offer", (offer) => { ... });
+ let { name, size, mtime, mode, serial, files_remaining, bytes_remaining } = offer.get_details();
+
+ offer.skip();
+
+ //...or:
+
+ offer.on("input", (octets) => { ... });
+
+ //accept()’s return resolves when the transfer is complete.
+ offer.accept().then(() => { ... });
+ });
+ zsession.on("session_end", () => { ... });
+ zsession.start();
+
+ The `offer` handler receives an Offer object. This object exposes the details
+ about the transfer offer. The object also exposes controls for skipping or
+ accepting the offer.
+
+ * Otherwise, your session is a send session. Now the user chooses
+zero or more files to send. For each of these you should do:
+
+ zsession.send_offer( { ... } ).then( (xfer) => {
+ if (!xfer) ... //skipped
+
+ else {
+ xfer.send( chunk );
+ xfer.end( chunk ).then(after_end);
+ }
+ } );
+
+ Note that `xfer.end()`’s return is a Promise. The resolution of this
+Promise is the point at which either to send another offer or to do:
+
+ zsession.close().then( () => { ... } );
+
+ The `close()` Promise’s resolution is the point at which the session
+has ended successfully.
+
+That should be all you need. If you want to go deeper, though, each module
+in this distribution has JSDoc and unit tests.
+
+# RATIONALE
+
+ZMODEM facilitates terminal-based file transfers.
+This was an important capability in the 1980s and early 1990s because
+most modem use was for terminal applications, especially
+[BBS](https://en.wikipedia.org/wiki/Bulletin_board_system)es.
+(This was how, for example,
+popular shareware games like [Wolfenstein 3D](http://3d.wolfenstein.com)
+were often distributed.) The World Wide Web in the
+mid-1990s, however, proved a more convenient way to accomplish most of
+what BBSes were useful for, as a result of which the problem that ZMODEM
+solved became a much less important one.
+
+ZMODEM stuck around, though, as it remained a convenient solution
+for terminal users who didn’t want open a separate session to transfer a
+file. [Uwe Ohse](https://uwe.ohse.de/)’s
+[lrzsz](https://ohse.de/uwe/software/lrzsz.html) package
+provided a portable C implementation of the protocol (reworked from
+the last public domain release of the original code) that is installed on
+many systems today.
+
+Where `lrzsz` can’t reach, though, is terminals that don’t have command-line
+access—such as terminals that run in JavaScript. Now that
+[WebSocket](https://en.wikipedia.org/wiki/WebSocket) makes real-time
+applications like terminals possible in a web browser,
+there is a use case for a JavaScript
+implementation of ZMODEM to allow file transfers in this context.
+
+# GENERAL FLOW OF A ZMODEM SESSION:
+
+The following is an overview of an error-free ZMODEM session.
+
+0. If you call the `sz` command (or equivalent), that command will send
+a special ZRQINIT “pre-header” to signal your terminal to be a ZMODEM
+receiver.
+
+1. The receiver, upon recognizing the ZRQINIT header, responds with
+a ZRINIT header.
+
+2. The sender sends a ZFILE header along with information about the file.
+(This may also include the size and file count for the entire batch of files.)
+
+3. The recipient either accepts the file or skips it.
+
+4. If the recipient did not skip the file, then the sender sends the file
+contents. At the end the sender sends a ZEOF header to let the recipient
+know this file is done.
+
+5. The recipient sends another ZRINIT header. This lets the sender know that
+the recipient confirms receipt of the entire file.
+
+6. Repeat steps 2-5 until the sender has no more files to send.
+
+7. Once the sender has no more files to send, the sender sends a ZEOF header,
+which the recipient echoes back. The sender closes the session by sending
+`OO` (“over and out”).
+
+# PROTOCOL NOTES AND ASSUMPTIONS
+
+Here are some notes about this particular implementation.
+
+Particular notes:
+
+* We send with a maximum data subpacket size of 8 KiB (8,192 bytes). While
+the ZMODEM specification stipulates a maximum of 1 KiB, `lrzsz` accepts
+the larger size, and it seems to have become a de facto standard extension
+to the protocol.
+
+* Remote command execution (i.e., ZCOMMAND) is unimplemented. It probably
+wouldn’t work in browsers, which is zmodem.js’s principal use case.
+
+* No file translations are done. (Unix/Windows line endings are a
+future feature possibility.)
+
+* It is assumed that no error correction will be needed. All connections
+are assumed to be **“reliable”**; i.e.,
+data is received exactly as sent. We take this for granted today,
+but ZMODEM’s original application was over raw modem connections that
+often didn’t have reliable hardware error correction. TCP also wasn’t
+in play to do software error correction as generally happens
+today over remote connections. Because the forseeable use of zmodem.js
+is either over TCP or a local socket—both of which are reliable—it seems
+safe to assume that zmodem.js will not need to implement error correction.
+
+* zmodem.js sends with CRC-16 by default. Ideally we would just use CRC-16
+for everything, but lsz 0.12.20 has a [buffer overflow bug](https://github.com/gooselinux/lrzsz/blob/master/lrzsz-0.12.20.patch) that rears its
+head when you try to abort a ZMODEM session in the middle of a CRC-16 file
+transfer. To avoid this bug, zmodem.js advertises CRC-32 support when it
+receives a file, which makes lsz avoid the buffer overflow bug by using
+CRC-32.
+
+ The bug is reported, incidentally, and a fix is expected (nearly 20 years
+ after the last official lrzsz release!).
+
+* There is no XMODEM/YMODEM fallback.
+
+* Occasionally lrzsz will output things to the console that aren’t
+actual ZMODEM—for example, if you skip an offered file, `sz` will write a
+message about it to the console. For the most part we can accommodate these
+because they happen between ZMODEM headers; however, it’s possible to
+“poison” such messages, e.g., by sending a file whose name includes a
+ZMODEM header. So don’t do that. :-P
+
+# IMPLEMENTATION NOTES
+
+* I initially had success integrating zmodem.js with
+[xterm.js](https://xtermjs.org); however, that library’s plugin interface
+changed dramatically, and I haven’t created a new plugin to replace the
+old one. (It should be relatively straightforward if someone else wants to
+pick it up.)
+
+* Browsers don’t have an easy way to download only part of a file;
+as a result, anything the browser saves to disk must be the entire file.
+
+* ZMODEM is a _binary_ protocol. (There was an extension planned
+to escape everything down to 7-bit ASCII, but it doesn’t seem to have
+been implemented?) Hence, **if you use WebSocket, you’ll need to use
+binary messages, not text**.
+
+* lrzsz is the only widely-distributed ZMODEM implementation nowadays,
+which makes it a de facto standard in its
+own right. Thus far all end-to-end testing has been against it. It is
+thus possible that resolutions to disparities between `lrzsz` and the
+protocol specification may need to favor the implementation.
+
+* It is a generally-unavoidable byproduct of how ZMODEM works that
+the first header in a ZMODEM session will echo to the terminal. This
+explains the unsightly `**B0000…` stuff that you’ll see when you run
+either `rz` or `sz`.
+
+ That header
+ will include some form of line break. (From `lrzsz` means bytes 0x0d
+ and 0x8a—**not** 0x0a). Your terminal might react oddly to that;
+ if it does, try stripping out one or the other line ending character.
+
+# PROTOCOL CHOICE
+
+Both XMODEM and YMODEM (including the latter’s many variants) require the
+receiver to initiate the session by sending a “magic character” (ASCII SOH);
+the problem is that there’s nothing in the protocol to prompt the receiver
+to do so. ZMODEM is sender-driven, so the terminal can show a notice that
+says, “Do you want to receive a file?”
+
+This is a shame because these other two protocols are a good deal simpler
+than ZMODEM. The YMODEM-g variant in particular would be better-suited to
+our purpose because it doesn’t “litter” the transfer with CRCs.
+
+There is also [Kermit](http://www.columbia.edu/kermit/kermit.html), which
+seems to be more standardized than ZMODEM but **much** more complex.
+
+# DESIGN NOTES
+
+zmodem.js tries to avoid “useless” states:
+either we fail completely, or we succeed. To that end, some callbacks are
+required arguments (e.g., the Sentry constructor’s `to_terminal` argument),
+while others are registered separately.
+
+Likewise, for this reason some of the session-level logic is exposed only
+through the Transfer and Offer objects. The Session creates these
+internally then exposes them via callback
+
+# SOURCES
+
+ZMODEM is not standardized in a nice, clean, official RFC like DNS or HTTP;
+rather, it was one guy’s solution to a particular problem. There is
+documentation, but it’s not as helpful as it might be; for example,
+there’s only one example workflow given, and it’s a “happy-path”
+transmission of a single file.
+
+As part of writing zmodem.js I’ve culled together various resources
+about the protocol. As far as I know these are the best sources for
+information on ZMODEM.
+
+Two documents that describe ZMODEM are saved in the repository for reference.
+The first is the closest there is to an official ZMODEM specification:
+a description of the protocol from its author, Chuck Forsberg. The second
+seems to be based on the first and comes from
+[Jacques Mattheij](https://jacquesmattheij.com).
+
+**HISTORICAL:** The included `rzsz.zip` file (fetched from [ftp://archives.thebbs.org/file_transfer_protocols/](ftp://archives.thebbs.org/file_transfer_protocols/) on 16 October 2017)
+is the last public domain release
+from Forsberg. [http://freeware.nekochan.net/source/rzsz/](http://freeware.nekochan.net/source/rzsz/) has what is supposedly Forsberg’s last shareware release;
+I have not looked at it except for the README. I’m not sure of the
+copyright status of this software: Forsberg is deceased, and his company
+appears to be defunct. Regardless, neither it nor its public domain
+predecessor is likely in widespread use.
+
+Here are some other available ZMODEM implementations:
+
+* [lrzsz](https://ohse.de/uwe/software/lrzsz.html)
+
+ A widely-deployed adaptation of Forsberg’s last public domain ZMODEM
+ code. This is the de facto “reference” implementation, both by virtue
+ of its wide availability and its derivation from Forsberg’s original.
+ If your server has the `rz` and `sz` commands, they’re probably
+ from this package.
+
+* [SyncTERM](http://syncterm.bbsdev.net)
+
+ Based on Jacques Mattheij’s ZMODEM implementation, originally called
+ zmtx/zmrx. This is a much more readable implementation than lrzsz
+ but lamentably one that doesn’t seem to compile as readily.
+
+* [Qodem](https://github.com/klamonte/qodem)
+
+ This terminal emulator package appears to contain its own ZMODEM
+ implementation.
+
+* [PD Zmodem](http://pcmicro.com/netfoss/pdzmodem.html)
+
+ I know nothing of this one.
+
+* [zmodem (Rust)](https://github.com/lexxvir/zmodem)
+
+ A pure [Rust](http://rust-lang.org) implementation of ZMODEM.
+
+# REQUIREMENTS
+
+This library only supports modern browsers. There is no support for
+Internet Explorer or other older browsers planned.
+
+The tests have run successfully against node.js version 8.
+
+# DOCUMENTATION
+
+Besides this document, each module has inline [jsdoc](http://usejsdoc.org).
+You can see it by running `yarn` in the repository’s root directory;
+the documentation will build in a newly-created `documentation` directory.
+
+# CONTRIBUTING
+
+Contributions are welcome via
+[https://github.com/FGasper/zmodemjs](https://github.com/FGasper/zmodemjs).
+
+# TROUBLESHOOTING
+
+Before you do anything else, set `Zmodem.DEBUG` to true. This will log
+useful information about the ZMODEM session to your JavaScript console. That
+may give you all you need to fix your problem.
+
+If you have trouble transferring files, try these diagnostics:
+
+1. Transfer an empty file. (Run `touch empty.bin` to create one named `empty.bin`.)
+
+2. Transfer a small file. (`echo hello > small.txt`)
+
+3. Transfer a file that contains all possible octets. (`perl -e 'print chr for 0 .. 255' > all_bytes.bin`)
+
+4. If a specific file fails, does it still fail if you `truncate` a copy of
+the file down to, say, half size and transfer that truncated file? Does it
+work if you truncate the file down to 1 byte? If so, then use this method
+to determine which specific place in the file triggers the transfer error.
+
+**IF YOU HAVE DONE THE ABOVE** and still think the problem is with zmodem.js,
+you can file a bug report. Note that, historically, most bug reports have
+reflected implementation errors rather than bugs in zmodem.js.
+
+# TODO
+
+* Teach send sessions to “fast-forward” so as to honor requests for
+append-style sessions.
+
+* Implement newline conversions.
+
+* Teach Session how to do and to handle pre-CRC checks.
+
+* Possible: command-line `rz`, if there’s demand for it, e.g., in
+environments where `lrzsz` can’t run. (NB: The distribution includes
+a bare-bones, proof-of-concept `sz` replacement.)
+
+# KNOWN ISSUERS
+
+* In testing, Microsoft Edge appeared not to care what string was given
+to `<a>`’s `download` attribute; the saved filename was based on the
+browser’s internal Blob object URL instead.
+
+# COPYRIGHT
+
+Copyright 2017 Gasper Software Consulting
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+
+Parts of the CRC-16 logic are adapted from crc-js by Johannes Rudolph.