diff options
Diffstat (limited to 'vendor/winnow/src/_topic/partial.rs')
-rw-r--r-- | vendor/winnow/src/_topic/partial.rs | 46 |
1 files changed, 46 insertions, 0 deletions
diff --git a/vendor/winnow/src/_topic/partial.rs b/vendor/winnow/src/_topic/partial.rs new file mode 100644 index 000000000..263d7f127 --- /dev/null +++ b/vendor/winnow/src/_topic/partial.rs @@ -0,0 +1,46 @@ +//! # Parsing Partial Input +//! +//! Typically, the input being parsed is all in-memory, or is complete. Some data sources are too +//! large to fit into memory, only allowing parsing an incomplete or [`Partial`] subset of the +//! data, requiring incrementally parsing. +//! +//! By wrapping a stream, like `&[u8]`, with [`Partial`], parsers will report when the data is +//! [`Incomplete`] and more input is [`Needed`], allowing the caller to stream-in additional data +//! to be parsed. The data is then parsed a chunk at a time. +//! +//! Chunks are typically defined by either: +//! - A header reporting the number of bytes, like with [`length_value`] +//! - [`Partial`] can explicitly be changed to being complete once the specified bytes are +//! acquired via [`StreamIsPartial::complete`]. +//! - A delimiter, like with [ndjson](http://ndjson.org/) +//! - You can parse up-to the delimiter or do a `take_until0(delim).and_then(parser)` +//! +//! If the chunks are not homogeneous, a state machine will be needed to track what the expected +//! parser is for the next chunk. +//! +//! Caveats: +//! - `winnow` takes the approach of re-parsing from scratch. Chunks should be relatively small to +//! prevent the re-parsing overhead from dominating. +//! - Parsers like [`many0`] do not know when an `eof` is from insufficient data or the end of the +//! stream, causing them to always report [`Incomplete`]. +//! +//! # Example +//! +//! `main.rs`: +//! ```rust,ignore +#![doc = include_str!("../../examples/ndjson/main.rs")] +//! ``` +//! +//! `parser.rs`: +//! ```rust,ignore +#![doc = include_str!("../../examples/ndjson/parser.rs")] +//! ``` + +#![allow(unused_imports)] // Used for intra-doc links + +use crate::error::ErrMode::Incomplete; +use crate::error::Needed; +use crate::multi::length_value; +use crate::multi::many0; +use crate::stream::Partial; +use crate::stream::StreamIsPartial; |