summaryrefslogtreecommitdiffstats
path: root/vendor/winnow/src/_topic/partial.rs
diff options
context:
space:
mode:
Diffstat (limited to 'vendor/winnow/src/_topic/partial.rs')
-rw-r--r--vendor/winnow/src/_topic/partial.rs46
1 files changed, 46 insertions, 0 deletions
diff --git a/vendor/winnow/src/_topic/partial.rs b/vendor/winnow/src/_topic/partial.rs
new file mode 100644
index 000000000..263d7f127
--- /dev/null
+++ b/vendor/winnow/src/_topic/partial.rs
@@ -0,0 +1,46 @@
+//! # Parsing Partial Input
+//!
+//! Typically, the input being parsed is all in-memory, or is complete. Some data sources are too
+//! large to fit into memory, only allowing parsing an incomplete or [`Partial`] subset of the
+//! data, requiring incrementally parsing.
+//!
+//! By wrapping a stream, like `&[u8]`, with [`Partial`], parsers will report when the data is
+//! [`Incomplete`] and more input is [`Needed`], allowing the caller to stream-in additional data
+//! to be parsed. The data is then parsed a chunk at a time.
+//!
+//! Chunks are typically defined by either:
+//! - A header reporting the number of bytes, like with [`length_value`]
+//! - [`Partial`] can explicitly be changed to being complete once the specified bytes are
+//! acquired via [`StreamIsPartial::complete`].
+//! - A delimiter, like with [ndjson](http://ndjson.org/)
+//! - You can parse up-to the delimiter or do a `take_until0(delim).and_then(parser)`
+//!
+//! If the chunks are not homogeneous, a state machine will be needed to track what the expected
+//! parser is for the next chunk.
+//!
+//! Caveats:
+//! - `winnow` takes the approach of re-parsing from scratch. Chunks should be relatively small to
+//! prevent the re-parsing overhead from dominating.
+//! - Parsers like [`many0`] do not know when an `eof` is from insufficient data or the end of the
+//! stream, causing them to always report [`Incomplete`].
+//!
+//! # Example
+//!
+//! `main.rs`:
+//! ```rust,ignore
+#![doc = include_str!("../../examples/ndjson/main.rs")]
+//! ```
+//!
+//! `parser.rs`:
+//! ```rust,ignore
+#![doc = include_str!("../../examples/ndjson/parser.rs")]
+//! ```
+
+#![allow(unused_imports)] // Used for intra-doc links
+
+use crate::error::ErrMode::Incomplete;
+use crate::error::Needed;
+use crate::multi::length_value;
+use crate::multi::many0;
+use crate::stream::Partial;
+use crate::stream::StreamIsPartial;