Adding upstream version 4.19.249.upstream/4.19.249upstream

Signed-off-by: Daniel Baumann <daniel.baumann@progress-linux.org>

1 files changed, 264 insertions, 0 deletions

diff --git a/Documentation/media/kapi/mc-core.rst b/Documentation/media/kapi/mc-core.rst
new file mode 100644
index 000000000..0c05503ea
--- /dev/null
+++ b/Documentation/media/kapi/mc-core.rst
@@ -0,0 +1,264 @@
+Media Controller devices
+------------------------
+
+Media Controller
+~~~~~~~~~~~~~~~~
+
+The media controller userspace API is documented in
+:ref:`the Media Controller uAPI book <media_controller>`. This document focus
+on the kernel-side implementation of the media framework.
+
+Abstract media device model
+^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Discovering a device internal topology, and configuring it at runtime, is one
+of the goals of the media framework. To achieve this, hardware devices are
+modelled as an oriented graph of building blocks called entities connected
+through pads.
+
+An entity is a basic media hardware building block. It can correspond to
+a large variety of logical blocks such as physical hardware devices
+(CMOS sensor for instance), logical hardware devices (a building block
+in a System-on-Chip image processing pipeline), DMA channels or physical
+connectors.
+
+A pad is a connection endpoint through which an entity can interact with
+other entities. Data (not restricted to video) produced by an entity
+flows from the entity's output to one or more entity inputs. Pads should
+not be confused with physical pins at chip boundaries.
+
+A link is a point-to-point oriented connection between two pads, either
+on the same entity or on different entities. Data flows from a source
+pad to a sink pad.
+
+Media device
+^^^^^^^^^^^^
+
+A media device is represented by a struct :c:type:`media_device`
+instance, defined in ``include/media/media-device.h``.
+Allocation of the structure is handled by the media device driver, usually by
+embedding the :c:type:`media_device` instance in a larger driver-specific
+structure.
+
+Drivers register media device instances by calling
+:c:func:`__media_device_register()` via the macro ``media_device_register()``
+and unregistered by calling :c:func:`media_device_unregister()`.
+
+Entities
+^^^^^^^^
+
+Entities are represented by a struct :c:type:`media_entity`
+instance, defined in ``include/media/media-entity.h``. The structure is usually
+embedded into a higher-level structure, such as
+:c:type:`v4l2_subdev` or :c:type:`video_device`
+instances, although drivers can allocate entities directly.
+
+Drivers initialize entity pads by calling
+:c:func:`media_entity_pads_init()`.
+
+Drivers register entities with a media device by calling
+:c:func:`media_device_register_entity()`
+and unregistred by calling
+:c:func:`media_device_unregister_entity()`.
+
+Interfaces
+^^^^^^^^^^
+
+Interfaces are represented by a
+struct :c:type:`media_interface` instance, defined in
+``include/media/media-entity.h``. Currently, only one type of interface is
+defined: a device node. Such interfaces are represented by a
+struct :c:type:`media_intf_devnode`.
+
+Drivers initialize and create device node interfaces by calling
+:c:func:`media_devnode_create()`
+and remove them by calling:
+:c:func:`media_devnode_remove()`.
+
+Pads
+^^^^
+Pads are represented by a struct :c:type:`media_pad` instance,
+defined in ``include/media/media-entity.h``. Each entity stores its pads in
+a pads array managed by the entity driver. Drivers usually embed the array in
+a driver-specific structure.
+
+Pads are identified by their entity and their 0-based index in the pads
+array.
+
+Both information are stored in the struct :c:type:`media_pad`,
+making the struct :c:type:`media_pad` pointer the canonical way
+to store and pass link references.
+
+Pads have flags that describe the pad capabilities and state.
+
+``MEDIA_PAD_FL_SINK`` indicates that the pad supports sinking data.
+``MEDIA_PAD_FL_SOURCE`` indicates that the pad supports sourcing data.
+
+.. note::
+
+  One and only one of ``MEDIA_PAD_FL_SINK`` or ``MEDIA_PAD_FL_SOURCE`` must
+  be set for each pad.
+
+Links
+^^^^^
+
+Links are represented by a struct :c:type:`media_link` instance,
+defined in ``include/media/media-entity.h``. There are two types of links:
+
+**1. pad to pad links**:
+
+Associate two entities via their PADs. Each entity has a list that points
+to all links originating at or targeting any of its pads.
+A given link is thus stored twice, once in the source entity and once in
+the target entity.
+
+Drivers create pad to pad links by calling:
+:c:func:`media_create_pad_link()` and remove with
+:c:func:`media_entity_remove_links()`.
+
+**2. interface to entity links**:
+
+Associate one interface to a Link.
+
+Drivers create interface to entity links by calling:
+:c:func:`media_create_intf_link()` and remove with
+:c:func:`media_remove_intf_links()`.
+
+.. note::
+
+   Links can only be created after having both ends already created.
+
+Links have flags that describe the link capabilities and state. The
+valid values are described at :c:func:`media_create_pad_link()` and
+:c:func:`media_create_intf_link()`.
+
+Graph traversal
+^^^^^^^^^^^^^^^
+
+The media framework provides APIs to iterate over entities in a graph.
+
+To iterate over all entities belonging to a media device, drivers can use
+the media_device_for_each_entity macro, defined in
+``include/media/media-device.h``.
+
+..  code-block:: c
+
+    struct media_entity *entity;
+
+    media_device_for_each_entity(entity, mdev) {
+    // entity will point to each entity in turn
+    ...
+    }
+
+Drivers might also need to iterate over all entities in a graph that can be
+reached only through enabled links starting at a given entity. The media
+framework provides a depth-first graph traversal API for that purpose.
+
+.. note::
+
+   Graphs with cycles (whether directed or undirected) are **NOT**
+   supported by the graph traversal API. To prevent infinite loops, the graph
+   traversal code limits the maximum depth to ``MEDIA_ENTITY_ENUM_MAX_DEPTH``,
+   currently defined as 16.
+
+Drivers initiate a graph traversal by calling
+:c:func:`media_graph_walk_start()`
+
+The graph structure, provided by the caller, is initialized to start graph
+traversal at the given entity.
+
+Drivers can then retrieve the next entity by calling
+:c:func:`media_graph_walk_next()`
+
+When the graph traversal is complete the function will return ``NULL``.
+
+Graph traversal can be interrupted at any moment. No cleanup function call
+is required and the graph structure can be freed normally.
+
+Helper functions can be used to find a link between two given pads, or a pad
+connected to another pad through an enabled link
+:c:func:`media_entity_find_link()` and
+:c:func:`media_entity_remote_pad()`.
+
+Use count and power handling
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Due to the wide differences between drivers regarding power management
+needs, the media controller does not implement power management. However,
+the struct :c:type:`media_entity` includes a ``use_count``
+field that media drivers
+can use to track the number of users of every entity for power management
+needs.
+
+The :c:type:`media_entity<media_entity>`.\ ``use_count`` field is owned by
+media drivers and must not be
+touched by entity drivers. Access to the field must be protected by the
+:c:type:`media_device`.\ ``graph_mutex`` lock.
+
+Links setup
+^^^^^^^^^^^
+
+Link properties can be modified at runtime by calling
+:c:func:`media_entity_setup_link()`.
+
+Pipelines and media streams
+^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+When starting streaming, drivers must notify all entities in the pipeline to
+prevent link states from being modified during streaming by calling
+:c:func:`media_pipeline_start()`.
+
+The function will mark all entities connected to the given entity through
+enabled links, either directly or indirectly, as streaming.
+
+The struct :c:type:`media_pipeline` instance pointed to by
+the pipe argument will be stored in every entity in the pipeline.
+Drivers should embed the struct :c:type:`media_pipeline`
+in higher-level pipeline structures and can then access the
+pipeline through the struct :c:type:`media_entity`
+pipe field.
+
+Calls to :c:func:`media_pipeline_start()` can be nested.
+The pipeline pointer must be identical for all nested calls to the function.
+
+:c:func:`media_pipeline_start()` may return an error. In that case,
+it will clean up any of the changes it did by itself.
+
+When stopping the stream, drivers must notify the entities with
+:c:func:`media_pipeline_stop()`.
+
+If multiple calls to :c:func:`media_pipeline_start()` have been
+made the same number of :c:func:`media_pipeline_stop()` calls
+are required to stop streaming.
+The :c:type:`media_entity`.\ ``pipe`` field is reset to ``NULL`` on the last
+nested stop call.
+
+Link configuration will fail with ``-EBUSY`` by default if either end of the
+link is a streaming entity. Links that can be modified while streaming must
+be marked with the ``MEDIA_LNK_FL_DYNAMIC`` flag.
+
+If other operations need to be disallowed on streaming entities (such as
+changing entities configuration parameters) drivers can explicitly check the
+media_entity stream_count field to find out if an entity is streaming. This
+operation must be done with the media_device graph_mutex held.
+
+Link validation
+^^^^^^^^^^^^^^^
+
+Link validation is performed by :c:func:`media_pipeline_start()`
+for any entity which has sink pads in the pipeline. The
+:c:type:`media_entity`.\ ``link_validate()`` callback is used for that
+purpose. In ``link_validate()`` callback, entity driver should check
+that the properties of the source pad of the connected entity and its own
+sink pad match. It is up to the type of the entity (and in the end, the
+properties of the hardware) what matching actually means.
+
+Subsystems should facilitate link validation by providing subsystem specific
+helper functions to provide easy access for commonly needed information, and
+in the end provide a way to use driver-specific callbacks.
+
+.. kernel-doc:: include/media/media-device.h
+
+.. kernel-doc:: include/media/media-devnode.h
+
+.. kernel-doc:: include/media/media-entity.h