Adding upstream version 6.338.2.upstream/6.338.2upstream

Signed-off-by: Daniel Baumann <daniel.baumann@progress-linux.org>

1 files changed, 302 insertions, 0 deletions

diff --git a/docs/renderer.md b/docs/renderer.md
new file mode 100644
index 0000000..3104b0d
--- /dev/null
+++ b/docs/renderer.md
@@ -0,0 +1,302 @@
+# Rendering content: pl_frame, pl_renderer, and pl_queue
+
+This example roughly builds off the [previous entry](./basic-rendering.md),
+and as such will not cover the basics of how to create a window, initialize a
+`pl_gpu` and get pixels onto the screen.
+
+## Renderer
+
+The `pl_renderer` set of APIs represents the highest-level interface into
+libplacebo, and is what most users who simply want to display e.g. a video
+feed on-screen will want to be using.
+
+The basic initialization is straightforward, requiring no extra parameters:
+
+``` c linenums="1"
+pl_renderer renderer;
+
+init()
+{
+    renderer = pl_renderer_create(pllog, gpu);
+    if (!renderer)
+        goto error;
+
+    // ...
+}
+
+uninit()
+{
+    pl_renderer_destroy(&renderer);
+}
+```
+
+What makes the renderer powerful is the large number of `pl_render_params` it
+exposes. By default, libplacebo provides several presets to use:
+
+* **pl_render_fast_params**: Disables everything except for defaults. This is
+  the fastest possible configuration.
+* **pl_render_default_params**: Contains the recommended default parameters,
+  including some slightly higher quality scaling, as well as dithering.
+* **pl_render_high_quality_params**: A preset of reasonable defaults for a
+  higher-end machine (i.e. anything with a discrete GPU). This enables most
+  of the basic functionality, including upscaling, downscaling, debanding
+  and better HDR tone mapping.
+
+Covering all of the possible options exposed by `pl_render_params` is
+out-of-scope of this example and would be better served by looking at [the API
+documentation](https://code.videolan.org/videolan/libplacebo/-/blob/master/src/include/libplacebo/renderer.h#L94).
+
+### Frames
+
+[`pl_frame`](https://code.videolan.org/videolan/libplacebo/-/blob/master/src/include/libplacebo/renderer.h#L503)
+is the struct libplacebo uses to group textures and their metadata together
+into a coherent unit that can be rendered using the renderer. This is not
+currently a dynamically allocated or refcounted heap object, it is merely a
+struct that can live on the stack (or anywhere else). The actual data lives in
+corresponding `pl_tex` objects referenced in each of the frame's planes.
+
+``` c linenums="1"
+bool render_frame(const struct pl_frame *image,
+                  const struct pl_swapchain_frame *swframe)
+{
+    struct pl_frame target;
+    pl_frame_from_swapchain(&target, swframe);
+
+    return pl_render_image(renderer, image, target,
+                           &pl_render_default_params);
+}
+```
+
+!!! note "Renderer state"
+    The `pl_renderer` is conceptually (almost) stateless. The only thing that
+    is needed to get a different result is to change the render params, which
+    can be varied freely on every call, if the user desires.
+
+    The one case where this is not entirely true is when using frame mixing
+    (see below), or when using HDR peak detection. In this case, the renderer
+    can be explicitly reset using `pl_renderer_flush_cache`.
+
+To upload frames, the easiest methods are made available as dedicated helpers
+in
+[`<libplacebo/utils/upload.h>`](https://code.videolan.org/videolan/libplacebo/-/blob/master/src/include/libplacebo/utils/upload.h),
+and
+[`<libplacebo/utils/libav.h>`](https://code.videolan.org/videolan/libplacebo/-/blob/master/src/include/libplacebo/utils/libav.h)
+(for AVFrames). In general, I recommend checking out the [demo
+programs](https://code.videolan.org/videolan/libplacebo/-/tree/master/demos)
+for a clearer illustration of how to use them in practice.
+
+### Shader cache
+
+The renderer internally generates, compiles and caches a potentially large
+number of shader programs, some of which can be complex. On some platforms
+(notably D3D11), these can be quite costly to recompile on every program
+launch.
+
+As such, the renderer offers a way to save/restore its internal shader cache
+from some external location (managed by the API user). The use of this API is
+highly recommended:
+
+``` c linenums="1" hl_lines="1-2 10-14 21-27"
+static uint8_t *load_saved_cache();
+static void store_saved_cache(uint8_t *cache, size_t bytes);
+
+void init()
+{
+    renderer = pl_renderer_create(pllog, gpu);
+    if (!renderer)
+        goto error;
+
+    uint8_t *cache = load_saved_cache();
+    if (cache) {
+        pl_renderer_load(renderer, cache);
+        free(cache);
+    }
+
+    // ...
+}
+
+void uninit()
+{
+    size_t cache_bytes = pl_renderer_save(renderer, NULL);
+    uint8_t *cache = malloc(cache_bytes);
+    if (cache) {
+        pl_renderer_save(renderer, cache);
+        store_saved_cache(cache, cache_bytes);
+        free(cache);
+    }
+
+    pl_renderer_destroy(&renderer);
+}
+```
+
+!!! warning "Cache safety"
+    libplacebo performs only minimal validity checking on the shader cache,
+    and in general, cannot possibly guard against malicious alteration of such
+    files. Loading a cache from an untrusted source represents a remote code
+    execution vector.
+
+## Frame mixing
+
+One of the renderer's most powerful features is its ability to compensate
+for differences in framerates between the source and display by using [frame
+mixing](https://github.com/mpv-player/mpv/wiki/Interpolation) to blend
+adjacent frames together.
+
+Using this API requires presenting the renderer, at each vsync, with a
+`pl_frame_mix` struct, describing the current state of the vsync. In
+principle, such structs can be constructed by hand. To do this, all of the
+relevant frames (nearby the vsync timestamp) must be collected, and their
+relative distances to the vsync determined, by normalizing all PTS values such
+that the vsync represents time `0.0` (and a distance of `1.0` represents the
+nominal duration between adjacent frames). Note that timing vsyncs, and
+determining the correct vsync duration, are both left as problems for the user
+to solve.[^timing]. Here could be an example of a valid struct:
+
+[^timing]: However, this may change in the future, as the recent introduction of
+  the Vulkan display timing extension may result in display timing feedback
+  being added to the `pl_swapchain` API. That said, as of writing, this has
+  not yet happened.
+
+``` c
+(struct pl_frame_mix) {
+    .num_frames = 6
+    .frames = (const struct pl_frame *[]) {
+        /* frame 0 */
+        /* frame 1 */
+        /* ... */
+        /* frame 5 */
+    },
+    .signatures = (uint64_t[]) {
+        0x0, 0x1, 0x2, 0x3, 0x4, 0x5 // (1)
+    },
+    .timestamps = (float[]) {
+        -2.4, -1.4, -0.4, 0.6, 1.6, 2.6, // (2)
+    },
+    .vsync_duration = 0.4, // 24 fps video on 60 fps display
+}
+```
+
+1.  These must be unique per frame, but always refer to the same frame. For
+    example, this could be based on the frame's PTS, the frame's numerical ID
+    (in order of decoding), or some sort of hash. The details don't matter,
+    only that this uniquely identifies specific frames.
+
+2.  Typically, for CFR sources, frame timestamps will always be separated in
+    this list by a distance of 1.0. In this example, the vsync falls roughly
+    halfway (but not quite) in between two adjacent frames (with IDs 0x2 and
+    0x3).
+
+!!! note "Frame mixing radius"
+    In this example, the frame mixing radius (as determined by
+    `pl_frame_mix_radius` is `3.0`, so we include all frames that fall within
+    the timestamp interval of `[-3, 3)`. In general, you should consult this
+    function to determine what frames need to be included in the
+    `pl_frame_mix` - though including more frames than needed is not an error.
+
+### Frame queue
+
+Because this API is rather unwieldy and clumsy to use directly, libplacebo
+provides a helper abstraction known as `pl_queue` to assist in transforming
+some arbitrary source of frames (such as a video decoder) into nicely packed
+`pl_frame_mix` structs ready for consumption by the `pl_renderer`:
+
+``` c linenums="1"
+#include <libplacebo/utils/frame_queue.h>
+
+pl_queue queue;
+
+void init()
+{
+    queue = pl_queue_create(gpu);
+}
+
+void uninit()
+{
+    pl_queue_destroy(&queue);
+    // ...
+}
+```
+
+This queue can be interacted with through a number of mechanisms: either
+pushing frames (blocking or non-blocking), or by having the queue poll frames
+(via blocking or non-blocking callback) as-needed. For a full overview of the
+various methods of pushing and polling frames, check the [API
+documentation](https://code.videolan.org/videolan/libplacebo/-/blob/master/src/include/libplacebo/utils/frame_queue.h#L115).
+
+In this example, I will assume that we have a separate decoder thread pushing
+frames into the `pl_queue` in a blocking manner:
+
+``` c linenums="1"
+static void decoder_thread(void)
+{
+    void *frame;
+
+    while ((frame = /* decode new frame */)) {
+        pl_queue_push_block(queue, UINT64_MAX, &(struct pl_source_frame) {
+            .pts        = /* frame pts */,
+            .duration   = /* frame duration */,
+            .map        = /* map callback */,
+            .unmap      = /* unmap callback */,
+            .frame_data = frame,
+        });
+    }
+
+    pl_queue_push(queue, NULL); // signal EOF
+}
+```
+
+Now, in our render loop, we want to call `pl_queue_update` with appropriate
+values to retrieve the correct frame mix for each vsync:
+
+``` c linenums="1" hl_lines="3-10 12-21 27"
+bool render_frame(const struct pl_swapchain_frame *swframe)
+{
+    struct pl_frame_mix mix;
+    enum pl_queue_status res;
+    res = pl_queue_update(queue, &mix, pl_queue_params(
+        .pts            = /* time of next vsync */,
+        .radius         = pl_frame_mix_radius(&render_params),
+        .vsync_duration = /* if known */,
+        .timeout        = UINT64_MAX, // (2)
+    ));
+
+    switch (res) {
+    case PL_QUEUE_OK:
+        break;
+    case PL_QUEUE_EOF:
+        /* no more frames */
+        return false;
+    case PL_QUEUE_ERR:
+        goto error;
+    // (1)
+    }
+
+
+    struct pl_frame target;
+    pl_frame_from_swapchain(&target, swframe);
+
+    return pl_render_image_mix(renderer, &mix, target,
+                               &pl_render_default_params);
+}
+```
+
+1.  There is a fourth status, `PL_QUEUE_MORE`, which is returned only if the
+    resulting frame mix is incomplete (and the timeout was reached) -
+    basically this can only happen if the queue runs dry due to frames not
+    being supplied fast enough.
+
+    In this example, since we are setting `timeout` to `UINT64_MAX`, we will
+    never get this return value.
+
+2.  Setting this makes `pl_queue_update` block indefinitely until sufficiently
+    many frames have been pushed into the `pl_queue` from our separate
+    decoding thread.
+
+### Deinterlacing
+
+The frame queue also vastly simplifies the process of performing
+motion-adaptive temporal deinterlacing, by automatically linking together
+adjacent fields/frames. To take advantage of this, all you need to do is set
+the appropriate field (`pl_source_frame.first_frame`), as well as enabling
+[deinterlacing
+parameters](https://code.videolan.org/videolan/libplacebo/-/blob/master/src/include/libplacebo/renderer.h#L186).