diff options
| author | Daniel Baumann <daniel.baumann@progress-linux.org> | 2024-04-07 18:49:45 +0000 |
|---|---|---|
| committer | Daniel Baumann <daniel.baumann@progress-linux.org> | 2024-04-07 18:49:45 +0000 |
| commit | 2c3c1048746a4622d8c89a29670120dc8fab93c4 (patch) | |
| tree | 848558de17fb3008cdf4d861b01ac7781903ce39 /Documentation/userspace-api/media/mediactl | |
| parent | Initial commit. (diff) | |
| download | linux-b8823030eac27fc7a3d149e3a443a0b68810a78f.tar.xz linux-b8823030eac27fc7a3d149e3a443a0b68810a78f.zip | |
Adding upstream version 6.1.76.upstream/6.1.76upstream
Signed-off-by: Daniel Baumann <daniel.baumann@progress-linux.org>
Diffstat (limited to 'Documentation/userspace-api/media/mediactl')
21 files changed, 2150 insertions, 0 deletions
diff --git a/Documentation/userspace-api/media/mediactl/media-controller-intro.rst b/Documentation/userspace-api/media/mediactl/media-controller-intro.rst new file mode 100644 index 000000000..fce7eafc3 --- /dev/null +++ b/Documentation/userspace-api/media/mediactl/media-controller-intro.rst @@ -0,0 +1,33 @@ +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later + +.. _media-controller-intro: + +Introduction +============ + +Media devices increasingly handle multiple related functions. Many USB +cameras include microphones, video capture hardware can also output +video, or SoC camera interfaces also perform memory-to-memory operations +similar to video codecs. + +Independent functions, even when implemented in the same hardware, can +be modelled as separate devices. A USB camera with a microphone will be +presented to userspace applications as V4L2 and ALSA capture devices. +The devices' relationships (when using a webcam, end-users shouldn't +have to manually select the associated USB microphone), while not made +available directly to applications by the drivers, can usually be +retrieved from sysfs. + +With more and more advanced SoC devices being introduced, the current +approach will not scale. Device topologies are getting increasingly +complex and can't always be represented by a tree structure. Hardware +blocks are shared between different functions, creating dependencies +between seemingly unrelated devices. + +Kernel abstraction APIs such as V4L2 and ALSA provide means for +applications to access hardware parameters. As newer hardware expose an +increasingly high number of those parameters, drivers need to guess what +applications really require based on limited information, thereby +implementing policies that belong to userspace. + +The media controller API aims at solving those problems. diff --git a/Documentation/userspace-api/media/mediactl/media-controller-model.rst b/Documentation/userspace-api/media/mediactl/media-controller-model.rst new file mode 100644 index 000000000..78bfdfb2a --- /dev/null +++ b/Documentation/userspace-api/media/mediactl/media-controller-model.rst @@ -0,0 +1,41 @@ +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later + +.. _media-controller-model: + +Media device model +================== + +Discovering a device internal topology, and configuring it at runtime, +is one of the goals of the media controller API. To achieve this, +hardware devices and Linux Kernel interfaces are modelled as graph +objects on an oriented graph. The object types that constitute the graph +are: + +- An **entity** is a basic media hardware or software building block. + It can correspond to a large variety of logical blocks such as + physical hardware devices (CMOS sensor for instance), logical + hardware devices (a building block in a System-on-Chip image + processing pipeline), DMA channels or physical connectors. + +- An **interface** is a graph representation of a Linux Kernel + userspace API interface, like a device node or a sysfs file that + controls one or more entities in the graph. + +- A **pad** is a data connection endpoint through which an entity can + interact with other entities. Data (not restricted to video) produced + by an entity flows from the entity's output to one or more entity + inputs. Pads should not be confused with physical pins at chip + boundaries. + +- A **data link** is a point-to-point oriented connection between two + pads, either on the same entity or on different entities. Data flows + from a source pad to a sink pad. + +- An **interface link** is a point-to-point bidirectional control + connection between a Linux Kernel interface and an entity. + +- An **ancillary link** is a point-to-point connection denoting that two + entities form a single logical unit. For example this could represent the + fact that a particular camera sensor and lens controller form a single + physical module, meaning this lens controller drives the lens for this + camera sensor.
\ No newline at end of file diff --git a/Documentation/userspace-api/media/mediactl/media-controller.rst b/Documentation/userspace-api/media/mediactl/media-controller.rst new file mode 100644 index 000000000..508dd693b --- /dev/null +++ b/Documentation/userspace-api/media/mediactl/media-controller.rst @@ -0,0 +1,54 @@ +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later +.. include:: <isonum.txt> + +.. _media_controller: + +############################## +Part IV - Media Controller API +############################## + +.. only:: html + + .. class:: toc-title + + Table of Contents + +.. toctree:: + :maxdepth: 5 + :numbered: + + media-controller-intro + media-controller-model + media-types + request-api + media-funcs + media-header + + +********************** +Revision and Copyright +********************** + +Authors: + +- Pinchart, Laurent <laurent.pinchart@ideasonboard.com> + + - Initial version. + +- Carvalho Chehab, Mauro <mchehab@kernel.org> + + - MEDIA_IOC_G_TOPOLOGY documentation and documentation improvements. + +**Copyright** |copy| 2010 : Laurent Pinchart + +**Copyright** |copy| 2015-2016 : Mauro Carvalho Chehab + +**************** +Revision History +**************** + +:revision: 1.1.0 / 2015-12-12 (*mcc*) + +:revision: 1.0.0 / 2010-11-10 (*lp*) + +Initial revision diff --git a/Documentation/userspace-api/media/mediactl/media-func-close.rst b/Documentation/userspace-api/media/mediactl/media-func-close.rst new file mode 100644 index 000000000..8ac2443e7 --- /dev/null +++ b/Documentation/userspace-api/media/mediactl/media-func-close.rst @@ -0,0 +1,43 @@ +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later +.. c:namespace:: MC + +.. _media-func-close: + +************* +media close() +************* + +Name +==== + +media-close - Close a media device + +Synopsis +======== + +.. code-block:: c + + #include <unistd.h> + +.. c:function:: int close( int fd ) + +Arguments +========= + +``fd`` + File descriptor returned by :c:func:`open()`. + +Description +=========== + +Closes the media device. Resources associated with the file descriptor +are freed. The device configuration remain unchanged. + +Return Value +============ + +:c:func:`close()` returns 0 on success. On error, -1 is returned, and +``errno`` is set appropriately. Possible error codes are: + +EBADF + ``fd`` is not a valid open file descriptor. diff --git a/Documentation/userspace-api/media/mediactl/media-func-ioctl.rst b/Documentation/userspace-api/media/mediactl/media-func-ioctl.rst new file mode 100644 index 000000000..9e9a838f4 --- /dev/null +++ b/Documentation/userspace-api/media/mediactl/media-func-ioctl.rst @@ -0,0 +1,63 @@ +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later +.. c:namespace:: MC + +.. _media-func-ioctl: + +************* +media ioctl() +************* + +Name +==== + +media-ioctl - Control a media device + +Synopsis +======== + +.. code-block:: c + + #include <sys/ioctl.h> + +``int ioctl(int fd, int request, void *argp)`` + +Arguments +========= + +``fd`` + File descriptor returned by :c:func:`open()`. + +``request`` + Media ioctl request code as defined in the media.h header file, for + example MEDIA_IOC_SETUP_LINK. + +``argp`` + Pointer to a request-specific structure. + +Description +=========== + +The :ref:`ioctl() <media-func-ioctl>` function manipulates media device +parameters. The argument ``fd`` must be an open file descriptor. + +The ioctl ``request`` code specifies the media function to be called. It +has encoded in it whether the argument is an input, output or read/write +parameter, and the size of the argument ``argp`` in bytes. + +Macros and structures definitions specifying media ioctl requests and +their parameters are located in the media.h header file. All media ioctl +requests, their respective function and parameters are specified in +:ref:`media-user-func`. + +Return Value +============ + +On success 0 is returned, on error -1 and the ``errno`` variable is set +appropriately. The generic error codes are described at the +:ref:`Generic Error Codes <gen-errors>` chapter. + +Request-specific error codes are listed in the individual requests +descriptions. + +When an ioctl that takes an output or read/write parameter fails, the +parameter remains unmodified. diff --git a/Documentation/userspace-api/media/mediactl/media-func-open.rst b/Documentation/userspace-api/media/mediactl/media-func-open.rst new file mode 100644 index 000000000..24487cb0a --- /dev/null +++ b/Documentation/userspace-api/media/mediactl/media-func-open.rst @@ -0,0 +1,65 @@ +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later +.. c:namespace:: MC + +.. _media-func-open: + +************ +media open() +************ + +Name +==== + +media-open - Open a media device + +Synopsis +======== + +.. code-block:: c + + #include <fcntl.h> + +.. c:function:: int open( const char *device_name, int flags ) + +Arguments +========= + +``device_name`` + Device to be opened. + +``flags`` + Open flags. Access mode must be either ``O_RDONLY`` or ``O_RDWR``. + Other flags have no effect. + +Description +=========== + +To open a media device applications call :c:func:`open()` with the +desired device name. The function has no side effects; the device +configuration remain unchanged. + +When the device is opened in read-only mode, attempts to modify its +configuration will result in an error, and ``errno`` will be set to +EBADF. + +Return Value +============ + +:c:func:`open()` returns the new file descriptor on success. On error, +-1 is returned, and ``errno`` is set appropriately. Possible error codes +are: + +EACCES + The requested access to the file is not allowed. + +EMFILE + The process already has the maximum number of files open. + +ENFILE + The system limit on the total number of open files has been reached. + +ENOMEM + Insufficient kernel memory was available. + +ENXIO + No device corresponding to this device special file exists. diff --git a/Documentation/userspace-api/media/mediactl/media-funcs.rst b/Documentation/userspace-api/media/mediactl/media-funcs.rst new file mode 100644 index 000000000..e89629681 --- /dev/null +++ b/Documentation/userspace-api/media/mediactl/media-funcs.rst @@ -0,0 +1,26 @@ +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later + +.. _media-user-func: + +****************** +Function Reference +****************** + + +.. toctree:: + :maxdepth: 1 + + media-func-open + media-func-close + media-func-ioctl + media-ioc-device-info + media-ioc-g-topology + media-ioc-enum-entities + media-ioc-enum-links + media-ioc-setup-link + media-ioc-request-alloc + request-func-close + request-func-ioctl + request-func-poll + media-request-ioc-queue + media-request-ioc-reinit diff --git a/Documentation/userspace-api/media/mediactl/media-header.rst b/Documentation/userspace-api/media/mediactl/media-header.rst new file mode 100644 index 000000000..c674271c9 --- /dev/null +++ b/Documentation/userspace-api/media/mediactl/media-header.rst @@ -0,0 +1,10 @@ +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later + +.. _media_header: + +**************************** +Media Controller Header File +**************************** + +.. kernel-include:: $BUILDDIR/media.h.rst + diff --git a/Documentation/userspace-api/media/mediactl/media-ioc-device-info.rst b/Documentation/userspace-api/media/mediactl/media-ioc-device-info.rst new file mode 100644 index 000000000..d56ee6669 --- /dev/null +++ b/Documentation/userspace-api/media/mediactl/media-ioc-device-info.rst @@ -0,0 +1,106 @@ +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later +.. c:namespace:: MC + +.. _media_ioc_device_info: + +*************************** +ioctl MEDIA_IOC_DEVICE_INFO +*************************** + +Name +==== + +MEDIA_IOC_DEVICE_INFO - Query device information + +Synopsis +======== + +.. c:macro:: MEDIA_IOC_DEVICE_INFO + +``int ioctl(int fd, MEDIA_IOC_DEVICE_INFO, struct media_device_info *argp)`` + +Arguments +========= + +``fd`` + File descriptor returned by :c:func:`open()`. + +``argp`` + Pointer to struct :c:type:`media_device_info`. + +Description +=========== + +All media devices must support the ``MEDIA_IOC_DEVICE_INFO`` ioctl. To +query device information, applications call the ioctl with a pointer to +a struct :c:type:`media_device_info`. The driver +fills the structure and returns the information to the application. The +ioctl never fails. + +.. c:type:: media_device_info + +.. tabularcolumns:: |p{4.4cm}|p{4.4cm}|p{8.5cm}| + +.. flat-table:: struct media_device_info + :header-rows: 0 + :stub-columns: 0 + :widths: 1 1 2 + + * - char + - ``driver``\ [16] + - Name of the driver implementing the media API as a NUL-terminated + ASCII string. The driver version is stored in the + ``driver_version`` field. + + Driver specific applications can use this information to verify + the driver identity. It is also useful to work around known bugs, + or to identify drivers in error reports. + + * - char + - ``model``\ [32] + - Device model name as a NUL-terminated UTF-8 string. The device + version is stored in the ``device_version`` field and is not be + appended to the model name. + + * - char + - ``serial``\ [40] + - Serial number as a NUL-terminated ASCII string. + + * - char + - ``bus_info``\ [32] + - Location of the device in the system as a NUL-terminated ASCII + string. This includes the bus type name (PCI, USB, ...) and a + bus-specific identifier. + + * - __u32 + - ``media_version`` + - Media API version, formatted with the ``KERNEL_VERSION()`` macro. + + * - __u32 + - ``hw_revision`` + - Hardware device revision in a driver-specific format. + + * - __u32 + - ``driver_version`` + - Media device driver version, formatted with the + ``KERNEL_VERSION()`` macro. Together with the ``driver`` field + this identifies a particular driver. + + * - __u32 + - ``reserved``\ [31] + - Reserved for future extensions. Drivers and applications must set + this array to zero. + +The ``serial`` and ``bus_info`` fields can be used to distinguish +between multiple instances of otherwise identical hardware. The serial +number takes precedence when provided and can be assumed to be unique. +If the serial number is an empty string, the ``bus_info`` field can be +used instead. The ``bus_info`` field is guaranteed to be unique, but can +vary across reboots or device unplug/replug. + +Return Value +============ + +On success 0 is returned, on error -1 and the ``errno`` variable is set +appropriately. The generic error codes are described at the +:ref:`Generic Error Codes <gen-errors>` chapter. diff --git a/Documentation/userspace-api/media/mediactl/media-ioc-enum-entities.rst b/Documentation/userspace-api/media/mediactl/media-ioc-enum-entities.rst new file mode 100644 index 000000000..73bda0249 --- /dev/null +++ b/Documentation/userspace-api/media/mediactl/media-ioc-enum-entities.rst @@ -0,0 +1,146 @@ +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later +.. c:namespace:: MC + +.. _media_ioc_enum_entities: + +***************************** +ioctl MEDIA_IOC_ENUM_ENTITIES +***************************** + +Name +==== + +MEDIA_IOC_ENUM_ENTITIES - Enumerate entities and their properties + +Synopsis +======== + +.. c:macro:: MEDIA_IOC_ENUM_ENTITIES + +``int ioctl(int fd, MEDIA_IOC_ENUM_ENTITIES, struct media_entity_desc *argp)`` + +Arguments +========= + +``fd`` + File descriptor returned by :c:func:`open()`. + +``argp`` + Pointer to struct :c:type:`media_entity_desc`. + +Description +=========== + +To query the attributes of an entity, applications set the id field of a +struct :c:type:`media_entity_desc` structure and +call the MEDIA_IOC_ENUM_ENTITIES ioctl with a pointer to this +structure. The driver fills the rest of the structure or returns an +EINVAL error code when the id is invalid. + +.. _media-ent-id-flag-next: + +Entities can be enumerated by or'ing the id with the +``MEDIA_ENT_ID_FLAG_NEXT`` flag. The driver will return information +about the entity with the smallest id strictly larger than the requested +one ('next entity'), or the ``EINVAL`` error code if there is none. + +Entity IDs can be non-contiguous. Applications must *not* try to +enumerate entities by calling MEDIA_IOC_ENUM_ENTITIES with increasing +id's until they get an error. + +.. c:type:: media_entity_desc + +.. tabularcolumns:: |p{1.5cm}|p{1.7cm}|p{1.6cm}|p{1.5cm}|p{10.6cm}| + +.. flat-table:: struct media_entity_desc + :header-rows: 0 + :stub-columns: 0 + :widths: 2 2 1 8 + + * - __u32 + - ``id`` + - + - Entity ID, set by the application. When the ID is or'ed with + ``MEDIA_ENT_ID_FLAG_NEXT``, the driver clears the flag and returns + the first entity with a larger ID. Do not expect that the ID will + always be the same for each instance of the device. In other words, + do not hardcode entity IDs in an application. + + * - char + - ``name``\ [32] + - + - Entity name as an UTF-8 NULL-terminated string. This name must be unique + within the media topology. + + * - __u32 + - ``type`` + - + - Entity type, see :ref:`media-entity-functions` for details. + + * - __u32 + - ``revision`` + - + - Entity revision. Always zero (obsolete) + + * - __u32 + - ``flags`` + - + - Entity flags, see :ref:`media-entity-flag` for details. + + * - __u32 + - ``group_id`` + - + - Entity group ID. Always zero (obsolete) + + * - __u16 + - ``pads`` + - + - Number of pads + + * - __u16 + - ``links`` + - + - Total number of outbound links. Inbound links are not counted in + this field. + + * - __u32 + - ``reserved[4]`` + - + - Reserved for future extensions. Drivers and applications must set + the array to zero. + + * - union { + - (anonymous) + + * - struct + - ``dev`` + - + - Valid for (sub-)devices that create a single device node. + + * - + - __u32 + - ``major`` + - Device node major number. + + * - + - __u32 + - ``minor`` + - Device node minor number. + + * - __u8 + - ``raw``\ [184] + - + - + * - } + - + +Return Value +============ + +On success 0 is returned, on error -1 and the ``errno`` variable is set +appropriately. The generic error codes are described at the +:ref:`Generic Error Codes <gen-errors>` chapter. + +EINVAL + The struct :c:type:`media_entity_desc` ``id`` + references a non-existing entity. diff --git a/Documentation/userspace-api/media/mediactl/media-ioc-enum-links.rst b/Documentation/userspace-api/media/mediactl/media-ioc-enum-links.rst new file mode 100644 index 000000000..381804a91 --- /dev/null +++ b/Documentation/userspace-api/media/mediactl/media-ioc-enum-links.rst @@ -0,0 +1,145 @@ +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later +.. c:namespace:: MC + +.. _media_ioc_enum_links: + +************************** +ioctl MEDIA_IOC_ENUM_LINKS +************************** + +Name +==== + +MEDIA_IOC_ENUM_LINKS - Enumerate all pads and links for a given entity + +Synopsis +======== + +.. c:macro:: MEDIA_IOC_ENUM_LINKS + +``int ioctl(int fd, MEDIA_IOC_ENUM_LINKS, struct media_links_enum *argp)`` + +Arguments +========= + +``fd`` + File descriptor returned by :c:func:`open()`. + +``argp`` + Pointer to struct :c:type:`media_links_enum`. + +Description +=========== + +To enumerate pads and/or links for a given entity, applications set the +entity field of a struct :c:type:`media_links_enum` +structure and initialize the struct +:c:type:`media_pad_desc` and struct +:c:type:`media_link_desc` structure arrays pointed by +the ``pads`` and ``links`` fields. They then call the +MEDIA_IOC_ENUM_LINKS ioctl with a pointer to this structure. + +If the ``pads`` field is not NULL, the driver fills the ``pads`` array +with information about the entity's pads. The array must have enough +room to store all the entity's pads. The number of pads can be retrieved +with :ref:`MEDIA_IOC_ENUM_ENTITIES`. + +If the ``links`` field is not NULL, the driver fills the ``links`` array +with information about the entity's outbound links. The array must have +enough room to store all the entity's outbound links. The number of +outbound links can be retrieved with :ref:`MEDIA_IOC_ENUM_ENTITIES`. + +Only forward links that originate at one of the entity's source pads are +returned during the enumeration process. + +.. c:type:: media_links_enum + +.. tabularcolumns:: |p{4.4cm}|p{4.4cm}|p{8.5cm}| + +.. flat-table:: struct media_links_enum + :header-rows: 0 + :stub-columns: 0 + :widths: 1 1 2 + + * - __u32 + - ``entity`` + - Entity id, set by the application. + + * - struct :c:type:`media_pad_desc` + - \*\ ``pads`` + - Pointer to a pads array allocated by the application. Ignored if + NULL. + + * - struct :c:type:`media_link_desc` + - \*\ ``links`` + - Pointer to a links array allocated by the application. Ignored if + NULL. + + * - __u32 + - ``reserved[4]`` + - Reserved for future extensions. Drivers and applications must set + the array to zero. + +.. c:type:: media_pad_desc + +.. tabularcolumns:: |p{4.4cm}|p{4.4cm}|p{8.5cm}| + +.. flat-table:: struct media_pad_desc + :header-rows: 0 + :stub-columns: 0 + :widths: 1 1 2 + + * - __u32 + - ``entity`` + - ID of the entity this pad belongs to. + + * - __u16 + - ``index`` + - Pad index, starts at 0. + + * - __u32 + - ``flags`` + - Pad flags, see :ref:`media-pad-flag` for more details. + + * - __u32 + - ``reserved[2]`` + - Reserved for future extensions. Drivers and applications must set + the array to zero. + + +.. c:type:: media_link_desc + +.. tabularcolumns:: |p{4.4cm}|p{4.4cm}|p{8.5cm}| + +.. flat-table:: struct media_link_desc + :header-rows: 0 + :stub-columns: 0 + :widths: 1 1 2 + + * - struct :c:type:`media_pad_desc` + - ``source`` + - Pad at the origin of this link. + + * - struct :c:type:`media_pad_desc` + - ``sink`` + - Pad at the target of this link. + + * - __u32 + - ``flags`` + - Link flags, see :ref:`media-link-flag` for more details. + + * - __u32 + - ``reserved[2]`` + - Reserved for future extensions. Drivers and applications must set + the array to zero. + +Return Value +============ + +On success 0 is returned, on error -1 and the ``errno`` variable is set +appropriately. The generic error codes are described at the +:ref:`Generic Error Codes <gen-errors>` chapter. + +EINVAL + The struct :c:type:`media_links_enum` ``id`` + references a non-existing entity. diff --git a/Documentation/userspace-api/media/mediactl/media-ioc-g-topology.rst b/Documentation/userspace-api/media/mediactl/media-ioc-g-topology.rst new file mode 100644 index 000000000..77ea5c5e9 --- /dev/null +++ b/Documentation/userspace-api/media/mediactl/media-ioc-g-topology.rst @@ -0,0 +1,294 @@ +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later +.. c:namespace:: MC + +.. _media_ioc_g_topology: + +************************** +ioctl MEDIA_IOC_G_TOPOLOGY +************************** + +Name +==== + +MEDIA_IOC_G_TOPOLOGY - Enumerate the graph topology and graph element properties + +Synopsis +======== + +.. c:macro:: MEDIA_IOC_G_TOPOLOGY + +``int ioctl(int fd, MEDIA_IOC_G_TOPOLOGY, struct media_v2_topology *argp)`` + +Arguments +========= + +``fd`` + File descriptor returned by :c:func:`open()`. + +``argp`` + Pointer to struct :c:type:`media_v2_topology`. + +Description +=========== + +The typical usage of this ioctl is to call it twice. On the first call, +the structure defined at struct +:c:type:`media_v2_topology` should be zeroed. At +return, if no errors happen, this ioctl will return the +``topology_version`` and the total number of entities, interfaces, pads +and links. + +Before the second call, the userspace should allocate arrays to store +the graph elements that are desired, putting the pointers to them at the +ptr_entities, ptr_interfaces, ptr_links and/or ptr_pads, keeping the +other values untouched. + +If the ``topology_version`` remains the same, the ioctl should fill the +desired arrays with the media graph elements. + +.. tabularcolumns:: |p{1.6cm}|p{3.4cm}|p{12.3cm}| + +.. c:type:: media_v2_topology + +.. flat-table:: struct media_v2_topology + :header-rows: 0 + :stub-columns: 0 + :widths: 1 2 8 + + * - __u64 + - ``topology_version`` + - Version of the media graph topology. When the graph is created, + this field starts with zero. Every time a graph element is added + or removed, this field is incremented. + + * - __u32 + - ``num_entities`` + - Number of entities in the graph + + * - __u32 + - ``reserved1`` + - Applications and drivers shall set this to 0. + + * - __u64 + - ``ptr_entities`` + - A pointer to a memory area where the entities array will be + stored, converted to a 64-bits integer. It can be zero. if zero, + the ioctl won't store the entities. It will just update + ``num_entities`` + + * - __u32 + - ``num_interfaces`` + - Number of interfaces in the graph + + * - __u32 + - ``reserved2`` + - Applications and drivers shall set this to 0. + + * - __u64 + - ``ptr_interfaces`` + - A pointer to a memory area where the interfaces array will be + stored, converted to a 64-bits integer. It can be zero. if zero, + the ioctl won't store the interfaces. It will just update + ``num_interfaces`` + + * - __u32 + - ``num_pads`` + - Total number of pads in the graph + + * - __u32 + - ``reserved3`` + - Applications and drivers shall set this to 0. + + * - __u64 + - ``ptr_pads`` + - A pointer to a memory area where the pads array will be stored, + converted to a 64-bits integer. It can be zero. if zero, the ioctl + won't store the pads. It will just update ``num_pads`` + + * - __u32 + - ``num_links`` + - Total number of data and interface links in the graph + + * - __u32 + - ``reserved4`` + - Applications and drivers shall set this to 0. + + * - __u64 + - ``ptr_links`` + - A pointer to a memory area where the links array will be stored, + converted to a 64-bits integer. It can be zero. if zero, the ioctl + won't store the links. It will just update ``num_links`` + +.. tabularcolumns:: |p{1.6cm}|p{3.2cm}|p{12.5cm}| + +.. c:type:: media_v2_entity + +.. flat-table:: struct media_v2_entity + :header-rows: 0 + :stub-columns: 0 + :widths: 1 2 8 + + * - __u32 + - ``id`` + - Unique ID for the entity. Do not expect that the ID will + always be the same for each instance of the device. In other words, + do not hardcode entity IDs in an application. + + * - char + - ``name``\ [64] + - Entity name as an UTF-8 NULL-terminated string. This name must be unique + within the media topology. + + * - __u32 + - ``function`` + - Entity main function, see :ref:`media-entity-functions` for details. + + * - __u32 + - ``flags`` + - Entity flags, see :ref:`media-entity-flag` for details. + Only valid if ``MEDIA_V2_ENTITY_HAS_FLAGS(media_version)`` + returns true. The ``media_version`` is defined in struct + :c:type:`media_device_info` and can be retrieved using + :ref:`MEDIA_IOC_DEVICE_INFO`. + + * - __u32 + - ``reserved``\ [5] + - Reserved for future extensions. Drivers and applications must set + this array to zero. + +.. tabularcolumns:: |p{1.6cm}|p{3.2cm}|p{12.5cm}| + +.. c:type:: media_v2_interface + +.. flat-table:: struct media_v2_interface + :header-rows: 0 + :stub-columns: 0 + :widths: 1 2 8 + + * - __u32 + - ``id`` + - Unique ID for the interface. Do not expect that the ID will + always be the same for each instance of the device. In other words, + do not hardcode interface IDs in an application. + + * - __u32 + - ``intf_type`` + - Interface type, see :ref:`media-intf-type` for details. + + * - __u32 + - ``flags`` + - Interface flags. Currently unused. + + * - __u32 + - ``reserved``\ [9] + - Reserved for future extensions. Drivers and applications must set + this array to zero. + + * - struct media_v2_intf_devnode + - ``devnode`` + - Used only for device node interfaces. See + :c:type:`media_v2_intf_devnode` for details. + +.. tabularcolumns:: |p{1.6cm}|p{3.2cm}|p{12.5cm}| + +.. c:type:: media_v2_intf_devnode + +.. flat-table:: struct media_v2_intf_devnode + :header-rows: 0 + :stub-columns: 0 + :widths: 1 2 8 + + * - __u32 + - ``major`` + - Device node major number. + + * - __u32 + - ``minor`` + - Device node minor number. + +.. tabularcolumns:: |p{1.6cm}|p{3.2cm}|p{12.5cm}| + +.. c:type:: media_v2_pad + +.. flat-table:: struct media_v2_pad + :header-rows: 0 + :stub-columns: 0 + :widths: 1 2 8 + + * - __u32 + - ``id`` + - Unique ID for the pad. Do not expect that the ID will + always be the same for each instance of the device. In other words, + do not hardcode pad IDs in an application. + + * - __u32 + - ``entity_id`` + - Unique ID for the entity where this pad belongs. + + * - __u32 + - ``flags`` + - Pad flags, see :ref:`media-pad-flag` for more details. + + * - __u32 + - ``index`` + - Pad index, starts at 0. Only valid if ``MEDIA_V2_PAD_HAS_INDEX(media_version)`` + returns true. The ``media_version`` is defined in struct + :c:type:`media_device_info` and can be retrieved using + :ref:`MEDIA_IOC_DEVICE_INFO`. + + * - __u32 + - ``reserved``\ [4] + - Reserved for future extensions. Drivers and applications must set + this array to zero. + +.. tabularcolumns:: |p{1.6cm}|p{3.2cm}|p{12.5cm}| + +.. c:type:: media_v2_link + +.. flat-table:: struct media_v2_link + :header-rows: 0 + :stub-columns: 0 + :widths: 1 2 8 + + * - __u32 + - ``id`` + - Unique ID for the link. Do not expect that the ID will + always be the same for each instance of the device. In other words, + do not hardcode link IDs in an application. + + * - __u32 + - ``source_id`` + - On pad to pad links: unique ID for the source pad. + + On interface to entity links: unique ID for the interface. + + * - __u32 + - ``sink_id`` + - On pad to pad links: unique ID for the sink pad. + + On interface to entity links: unique ID for the entity. + + * - __u32 + - ``flags`` + - Link flags, see :ref:`media-link-flag` for more details. + + * - __u32 + - ``reserved``\ [6] + - Reserved for future extensions. Drivers and applications must set + this array to zero. + +Return Value +============ + +On success 0 is returned, on error -1 and the ``errno`` variable is set +appropriately. The generic error codes are described at the +:ref:`Generic Error Codes <gen-errors>` chapter. + +ENOSPC + This is returned when either one or more of the num_entities, + num_interfaces, num_links or num_pads are non-zero and are + smaller than the actual number of elements inside the graph. This + may happen if the ``topology_version`` changed when compared to the + last time this ioctl was called. Userspace should usually free the + area for the pointers, zero the struct elements and call this ioctl + again. diff --git a/Documentation/userspace-api/media/mediactl/media-ioc-request-alloc.rst b/Documentation/userspace-api/media/mediactl/media-ioc-request-alloc.rst new file mode 100644 index 000000000..9195b4b8b --- /dev/null +++ b/Documentation/userspace-api/media/mediactl/media-ioc-request-alloc.rst @@ -0,0 +1,65 @@ +.. SPDX-License-Identifier: GPL-2.0 OR GFDL-1.1-no-invariants-or-later +.. c:namespace:: MC + +.. _media_ioc_request_alloc: + +***************************** +ioctl MEDIA_IOC_REQUEST_ALLOC +***************************** + +Name +==== + +MEDIA_IOC_REQUEST_ALLOC - Allocate a request + +Synopsis +======== + +.. c:macro:: MEDIA_IOC_REQUEST_ALLOC + +``int ioctl(int fd, MEDIA_IOC_REQUEST_ALLOC, int *argp)`` + +Arguments +========= + +``fd`` + File descriptor returned by :c:func:`open()`. + +``argp`` + Pointer to an integer. + +Description +=========== + +If the media device supports :ref:`requests <media-request-api>`, then +this ioctl can be used to allocate a request. If it is not supported, then +``errno`` is set to ``ENOTTY``. A request is accessed through a file descriptor +that is returned in ``*argp``. + +If the request was successfully allocated, then the request file descriptor +can be passed to the :ref:`VIDIOC_QBUF <VIDIOC_QBUF>`, +:ref:`VIDIOC_G_EXT_CTRLS <VIDIOC_G_EXT_CTRLS>`, +:ref:`VIDIOC_S_EXT_CTRLS <VIDIOC_G_EXT_CTRLS>` and +:ref:`VIDIOC_TRY_EXT_CTRLS <VIDIOC_G_EXT_CTRLS>` ioctls. + +In addition, the request can be queued by calling +:ref:`MEDIA_REQUEST_IOC_QUEUE` and re-initialized by calling +:ref:`MEDIA_REQUEST_IOC_REINIT`. + +Finally, the file descriptor can be :ref:`polled <request-func-poll>` to wait +for the request to complete. + +The request will remain allocated until all the file descriptors associated +with it are closed by :c:func:`close()` and the driver no +longer uses the request internally. See also +:ref:`here <media-request-life-time>` for more information. + +Return Value +============ + +On success 0 is returned, on error -1 and the ``errno`` variable is set +appropriately. The generic error codes are described at the +:ref:`Generic Error Codes <gen-errors>` chapter. + +ENOTTY + The driver has no support for requests. diff --git a/Documentation/userspace-api/media/mediactl/media-ioc-setup-link.rst b/Documentation/userspace-api/media/mediactl/media-ioc-setup-link.rst new file mode 100644 index 000000000..23208300c --- /dev/null +++ b/Documentation/userspace-api/media/mediactl/media-ioc-setup-link.rst @@ -0,0 +1,65 @@ +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later +.. c:namespace:: MC + +.. _media_ioc_setup_link: + +************************** +ioctl MEDIA_IOC_SETUP_LINK +************************** + +Name +==== + +MEDIA_IOC_SETUP_LINK - Modify the properties of a link + +Synopsis +======== + +.. c:macro:: MEDIA_IOC_SETUP_LINK + +``int ioctl(int fd, MEDIA_IOC_SETUP_LINK, struct media_link_desc *argp)`` + +Arguments +========= + +``fd`` + File descriptor returned by :c:func:`open()`. + +``argp`` + Pointer to struct :c:type:`media_link_desc`. + +Description +=========== + +To change link properties applications fill a struct +:c:type:`media_link_desc` with link identification +information (source and sink pad) and the new requested link flags. They +then call the MEDIA_IOC_SETUP_LINK ioctl with a pointer to that +structure. + +The only configurable property is the ``ENABLED`` link flag to +enable/disable a link. Links marked with the ``IMMUTABLE`` link flag can +not be enabled or disabled. + +Link configuration has no side effect on other links. If an enabled link +at the sink pad prevents the link from being enabled, the driver returns +with an ``EBUSY`` error code. + +Only links marked with the ``DYNAMIC`` link flag can be enabled/disabled +while streaming media data. Attempting to enable or disable a streaming +non-dynamic link will return an ``EBUSY`` error code. + +If the specified link can't be found the driver returns with an ``EINVAL`` +error code. + +Return Value +============ + +On success 0 is returned, on error -1 and the ``errno`` variable is set +appropriately. The generic error codes are described at the +:ref:`Generic Error Codes <gen-errors>` chapter. + +EINVAL + The struct :c:type:`media_link_desc` references a + non-existing link, or the link is immutable and an attempt to modify + its configuration was made. diff --git a/Documentation/userspace-api/media/mediactl/media-request-ioc-queue.rst b/Documentation/userspace-api/media/mediactl/media-request-ioc-queue.rst new file mode 100644 index 000000000..04b33db2b --- /dev/null +++ b/Documentation/userspace-api/media/mediactl/media-request-ioc-queue.rst @@ -0,0 +1,77 @@ +.. SPDX-License-Identifier: GPL-2.0 OR GFDL-1.1-no-invariants-or-later +.. c:namespace:: MC + +.. _media_request_ioc_queue: + +***************************** +ioctl MEDIA_REQUEST_IOC_QUEUE +***************************** + +Name +==== + +MEDIA_REQUEST_IOC_QUEUE - Queue a request + +Synopsis +======== + +.. c:macro:: MEDIA_REQUEST_IOC_QUEUE + +``int ioctl(int request_fd, MEDIA_REQUEST_IOC_QUEUE)`` + +Arguments +========= + +``request_fd`` + File descriptor returned by :ref:`MEDIA_IOC_REQUEST_ALLOC`. + +Description +=========== + +If the media device supports :ref:`requests <media-request-api>`, then +this request ioctl can be used to queue a previously allocated request. + +If the request was successfully queued, then the file descriptor can be +:ref:`polled <request-func-poll>` to wait for the request to complete. + +If the request was already queued before, then ``EBUSY`` is returned. +Other errors can be returned if the contents of the request contained +invalid or inconsistent data, see the next section for a list of +common error codes. On error both the request and driver state are unchanged. + +Once a request is queued, then the driver is required to gracefully handle +errors that occur when the request is applied to the hardware. The +exception is the ``EIO`` error which signals a fatal error that requires +the application to stop streaming to reset the hardware state. + +It is not allowed to mix queuing requests with queuing buffers directly +(without a request). ``EBUSY`` will be returned if the first buffer was +queued directly and you next try to queue a request, or vice versa. + +A request must contain at least one buffer, otherwise this ioctl will +return an ``ENOENT`` error. + +Return Value +============ + +On success 0 is returned, on error -1 and the ``errno`` variable is set +appropriately. The generic error codes are described at the +:ref:`Generic Error Codes <gen-errors>` chapter. + +EBUSY + The request was already queued or the application queued the first + buffer directly, but later attempted to use a request. It is not permitted + to mix the two APIs. +ENOENT + The request did not contain any buffers. All requests are required + to have at least one buffer. This can also be returned if some required + configuration is missing in the request. +ENOMEM + Out of memory when allocating internal data structures for this + request. +EINVAL + The request has invalid data. +EIO + The hardware is in a bad state. To recover, the application needs to + stop streaming to reset the hardware state and then try to restart + streaming. diff --git a/Documentation/userspace-api/media/mediactl/media-request-ioc-reinit.rst b/Documentation/userspace-api/media/mediactl/media-request-ioc-reinit.rst new file mode 100644 index 000000000..57567b87b --- /dev/null +++ b/Documentation/userspace-api/media/mediactl/media-request-ioc-reinit.rst @@ -0,0 +1,51 @@ +.. SPDX-License-Identifier: GPL-2.0 OR GFDL-1.1-no-invariants-or-later +.. c:namespace:: MC + +.. _media_request_ioc_reinit: + +****************************** +ioctl MEDIA_REQUEST_IOC_REINIT +****************************** + +Name +==== + +MEDIA_REQUEST_IOC_REINIT - Re-initialize a request + +Synopsis +======== + +.. c:macro:: MEDIA_REQUEST_IOC_REINIT + +``int ioctl(int request_fd, MEDIA_REQUEST_IOC_REINIT)`` + +Arguments +========= + +``request_fd`` + File descriptor returned by :ref:`MEDIA_IOC_REQUEST_ALLOC`. + +Description +=========== + +If the media device supports :ref:`requests <media-request-api>`, then +this request ioctl can be used to re-initialize a previously allocated +request. + +Re-initializing a request will clear any existing data from the request. +This avoids having to :c:func:`close()` a completed +request and allocate a new request. Instead the completed request can just +be re-initialized and it is ready to be used again. + +A request can only be re-initialized if it either has not been queued +yet, or if it was queued and completed. Otherwise it will set ``errno`` +to ``EBUSY``. No other error codes can be returned. + +Return Value +============ + +On success 0 is returned, on error -1 and the ``errno`` variable is set +appropriately. + +EBUSY + The request is queued but not yet completed. diff --git a/Documentation/userspace-api/media/mediactl/media-types.rst b/Documentation/userspace-api/media/mediactl/media-types.rst new file mode 100644 index 000000000..0ffeece1e --- /dev/null +++ b/Documentation/userspace-api/media/mediactl/media-types.rst @@ -0,0 +1,432 @@ +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later + +.. _media-controller-types: + +Types and flags used to represent the media graph elements +========================================================== + +.. tabularcolumns:: |p{8.2cm}|p{9.3cm}| + +.. _media-entity-functions: +.. _MEDIA-ENT-F-UNKNOWN: +.. _MEDIA-ENT-F-V4L2-SUBDEV-UNKNOWN: +.. _MEDIA-ENT-F-IO-V4L: +.. _MEDIA-ENT-F-IO-VBI: +.. _MEDIA-ENT-F-IO-SWRADIO: +.. _MEDIA-ENT-F-IO-DTV: +.. _MEDIA-ENT-F-DTV-DEMOD: +.. _MEDIA-ENT-F-TS-DEMUX: +.. _MEDIA-ENT-F-DTV-CA: +.. _MEDIA-ENT-F-DTV-NET-DECAP: +.. _MEDIA-ENT-F-CONN-RF: +.. _MEDIA-ENT-F-CONN-SVIDEO: +.. _MEDIA-ENT-F-CONN-COMPOSITE: +.. _MEDIA-ENT-F-CAM-SENSOR: +.. _MEDIA-ENT-F-FLASH: +.. _MEDIA-ENT-F-LENS: +.. _MEDIA-ENT-F-ATV-DECODER: +.. _MEDIA-ENT-F-TUNER: +.. _MEDIA-ENT-F-IF-VID-DECODER: +.. _MEDIA-ENT-F-IF-AUD-DECODER: +.. _MEDIA-ENT-F-AUDIO-CAPTURE: +.. _MEDIA-ENT-F-AUDIO-PLAYBACK: +.. _MEDIA-ENT-F-AUDIO-MIXER: +.. _MEDIA-ENT-F-PROC-VIDEO-COMPOSER: +.. _MEDIA-ENT-F-PROC-VIDEO-PIXEL-FORMATTER: +.. _MEDIA-ENT-F-PROC-VIDEO-PIXEL-ENC-CONV: +.. _MEDIA-ENT-F-PROC-VIDEO-LUT: +.. _MEDIA-ENT-F-PROC-VIDEO-SCALER: +.. _MEDIA-ENT-F-PROC-VIDEO-STATISTICS: +.. _MEDIA-ENT-F-PROC-VIDEO-ENCODER: +.. _MEDIA-ENT-F-PROC-VIDEO-DECODER: +.. _MEDIA-ENT-F-PROC-VIDEO-ISP: +.. _MEDIA-ENT-F-VID-MUX: +.. _MEDIA-ENT-F-VID-IF-BRIDGE: +.. _MEDIA-ENT-F-DV-DECODER: +.. _MEDIA-ENT-F-DV-ENCODER: + +.. cssclass:: longtable + +.. flat-table:: Media entity functions + :header-rows: 0 + :stub-columns: 0 + + * - ``MEDIA_ENT_F_UNKNOWN`` and + ``MEDIA_ENT_F_V4L2_SUBDEV_UNKNOWN`` + - Unknown entity. That generally indicates that a driver didn't + initialize properly the entity, which is a Kernel bug + + * - ``MEDIA_ENT_F_IO_V4L`` + - Data streaming input and/or output entity. + + * - ``MEDIA_ENT_F_IO_VBI`` + - V4L VBI streaming input or output entity + + * - ``MEDIA_ENT_F_IO_SWRADIO`` + - V4L Software Digital Radio (SDR) streaming input or output entity + + * - ``MEDIA_ENT_F_IO_DTV`` + - DVB Digital TV streaming input or output entity + + * - ``MEDIA_ENT_F_DTV_DEMOD`` + - Digital TV demodulator entity. + + * - ``MEDIA_ENT_F_TS_DEMUX`` + - MPEG Transport stream demux entity. Could be implemented on + hardware or in Kernelspace by the Linux DVB subsystem. + + * - ``MEDIA_ENT_F_DTV_CA`` + - Digital TV Conditional Access module (CAM) entity + + * - ``MEDIA_ENT_F_DTV_NET_DECAP`` + - Digital TV network ULE/MLE desencapsulation entity. Could be + implemented on hardware or in Kernelspace + + * - ``MEDIA_ENT_F_CONN_RF`` + - Connector for a Radio Frequency (RF) signal. + + * - ``MEDIA_ENT_F_CONN_SVIDEO`` + - Connector for a S-Video signal. + + * - ``MEDIA_ENT_F_CONN_COMPOSITE`` + - Connector for a RGB composite signal. + + * - ``MEDIA_ENT_F_CAM_SENSOR`` + - Camera video sensor entity. + + * - ``MEDIA_ENT_F_FLASH`` + - Flash controller entity. + + * - ``MEDIA_ENT_F_LENS`` + - Lens controller entity. + + * - ``MEDIA_ENT_F_ATV_DECODER`` + - Analog video decoder, the basic function of the video decoder is + to accept analogue video from a wide variety of sources such as + broadcast, DVD players, cameras and video cassette recorders, in + either NTSC, PAL, SECAM or HD format, separating the stream into + its component parts, luminance and chrominance, and output it in + some digital video standard, with appropriate timing signals. + + * - ``MEDIA_ENT_F_TUNER`` + - Digital TV, analog TV, radio and/or software radio tuner, with + consists on a PLL tuning stage that converts radio frequency (RF) + signal into an Intermediate Frequency (IF). Modern tuners have + internally IF-PLL decoders for audio and video, but older models + have those stages implemented on separate entities. + + * - ``MEDIA_ENT_F_IF_VID_DECODER`` + - IF-PLL video decoder. It receives the IF from a PLL and decodes + the analog TV video signal. This is commonly found on some very + old analog tuners, like Philips MK3 designs. They all contain a + tda9887 (or some software compatible similar chip, like tda9885). + Those devices use a different I2C address than the tuner PLL. + + * - ``MEDIA_ENT_F_IF_AUD_DECODER`` + - IF-PLL sound decoder. It receives the IF from a PLL and decodes + the analog TV audio signal. This is commonly found on some very + old analog hardware, like Micronas msp3400, Philips tda9840, + tda985x, etc. Those devices use a different I2C address than the + tuner PLL and should be controlled together with the IF-PLL video + decoder. + + * - ``MEDIA_ENT_F_AUDIO_CAPTURE`` + - Audio Capture Function Entity. + + * - ``MEDIA_ENT_F_AUDIO_PLAYBACK`` + - Audio Playback Function Entity. + + * - ``MEDIA_ENT_F_AUDIO_MIXER`` + - Audio Mixer Function Entity. + + * - ``MEDIA_ENT_F_PROC_VIDEO_COMPOSER`` + - Video composer (blender). An entity capable of video + composing must have at least two sink pads and one source + pad, and composes input video frames onto output video + frames. Composition can be performed using alpha blending, + color keying, raster operations (ROP), stitching or any other + means. + + * - ``MEDIA_ENT_F_PROC_VIDEO_PIXEL_FORMATTER`` + - Video pixel formatter. An entity capable of pixel formatting + must have at least one sink pad and one source pad. Read + pixel formatters read pixels from memory and perform a subset + of unpacking, cropping, color keying, alpha multiplication + and pixel encoding conversion. Write pixel formatters perform + a subset of dithering, pixel encoding conversion and packing + and write pixels to memory. + + * - ``MEDIA_ENT_F_PROC_VIDEO_PIXEL_ENC_CONV`` + - Video pixel encoding converter. An entity capable of pixel + encoding conversion must have at least one sink pad and one + source pad, and convert the encoding of pixels received on + its sink pad(s) to a different encoding output on its source + pad(s). Pixel encoding conversion includes but isn't limited + to RGB to/from HSV, RGB to/from YUV and CFA (Bayer) to RGB + conversions. + + * - ``MEDIA_ENT_F_PROC_VIDEO_LUT`` + - Video look-up table. An entity capable of video lookup table + processing must have one sink pad and one source pad. It uses + the values of the pixels received on its sink pad to look up + entries in internal tables and output them on its source pad. + The lookup processing can be performed on all components + separately or combine them for multi-dimensional table + lookups. + + * - ``MEDIA_ENT_F_PROC_VIDEO_SCALER`` + - Video scaler. An entity capable of video scaling must have + at least one sink pad and one source pad, and scale the + video frame(s) received on its sink pad(s) to a different + resolution output on its source pad(s). The range of + supported scaling ratios is entity-specific and can differ + between the horizontal and vertical directions (in particular + scaling can be supported in one direction only). Binning and + sub-sampling (occasionally also referred to as skipping) are + considered as scaling. + + * - ``MEDIA_ENT_F_PROC_VIDEO_STATISTICS`` + - Video statistics computation (histogram, 3A, etc.). An entity + capable of statistics computation must have one sink pad and + one source pad. It computes statistics over the frames + received on its sink pad and outputs the statistics data on + its source pad. + + * - ``MEDIA_ENT_F_PROC_VIDEO_ENCODER`` + - Video (MPEG, HEVC, VPx, etc.) encoder. An entity capable of + compressing video frames. Must have one sink pad and at least + one source pad. + + * - ``MEDIA_ENT_F_PROC_VIDEO_DECODER`` + - Video (MPEG, HEVC, VPx, etc.) decoder. An entity capable of + decompressing a compressed video stream into uncompressed video + frames. Must have one sink pad and at least one source pad. + + * - ``MEDIA_ENT_F_PROC_VIDEO_ISP`` + - An Image Signal Processor (ISP) device. ISPs generally are one of a + kind devices that have their specific control interfaces using a + combination of custom V4L2 controls and IOCTLs, and parameters + supplied in a metadata buffer. + + * - ``MEDIA_ENT_F_VID_MUX`` + - Video multiplexer. An entity capable of multiplexing must have at + least two sink pads and one source pad, and must pass the video + frame(s) received from the active sink pad to the source pad. + + * - ``MEDIA_ENT_F_VID_IF_BRIDGE`` + - Video interface bridge. A video interface bridge entity must have at + least one sink pad and at least one source pad. It receives video + frames on its sink pad from an input video bus of one type (HDMI, eDP, + MIPI CSI-2, etc.), and outputs them on its source pad to an output + video bus of another type (eDP, MIPI CSI-2, parallel, etc.). + + * - ``MEDIA_ENT_F_DV_DECODER`` + - Digital video decoder. The basic function of the video decoder is + to accept digital video from a wide variety of sources + and output it in some digital video standard, with appropriate + timing signals. + + * - ``MEDIA_ENT_F_DV_ENCODER`` + - Digital video encoder. The basic function of the video encoder is + to accept digital video from some digital video standard with + appropriate timing signals (usually a parallel video bus with sync + signals) and output this to a digital video output connector such + as HDMI or DisplayPort. + +.. tabularcolumns:: |p{5.5cm}|p{12.0cm}| + +.. _media-entity-flag: +.. _MEDIA-ENT-FL-DEFAULT: +.. _MEDIA-ENT-FL-CONNECTOR: + +.. flat-table:: Media entity flags + :header-rows: 0 + :stub-columns: 0 + + * - ``MEDIA_ENT_FL_DEFAULT`` + - Default entity for its type. Used to discover the default audio, + VBI and video devices, the default camera sensor, etc. + + * - ``MEDIA_ENT_FL_CONNECTOR`` + - The entity represents a connector. + + +.. tabularcolumns:: |p{6.5cm}|p{6.0cm}|p{4.8cm}| + +.. _media-intf-type: +.. _MEDIA-INTF-T-DVB-FE: +.. _MEDIA-INTF-T-DVB-DEMUX: +.. _MEDIA-INTF-T-DVB-DVR: +.. _MEDIA-INTF-T-DVB-CA: +.. _MEDIA-INTF-T-DVB-NET: +.. _MEDIA-INTF-T-V4L-VIDEO: +.. _MEDIA-INTF-T-V4L-VBI: +.. _MEDIA-INTF-T-V4L-RADIO: +.. _MEDIA-INTF-T-V4L-SUBDEV: +.. _MEDIA-INTF-T-V4L-SWRADIO: +.. _MEDIA-INTF-T-V4L-TOUCH: +.. _MEDIA-INTF-T-ALSA-PCM-CAPTURE: +.. _MEDIA-INTF-T-ALSA-PCM-PLAYBACK: +.. _MEDIA-INTF-T-ALSA-CONTROL: +.. _MEDIA-INTF-T-ALSA-COMPRESS: +.. _MEDIA-INTF-T-ALSA-RAWMIDI: +.. _MEDIA-INTF-T-ALSA-HWDEP: +.. _MEDIA-INTF-T-ALSA-SEQUENCER: +.. _MEDIA-INTF-T-ALSA-TIMER: + +.. flat-table:: Media interface types + :header-rows: 0 + :stub-columns: 0 + + * - ``MEDIA_INTF_T_DVB_FE`` + - Device node interface for the Digital TV frontend + - typically, /dev/dvb/adapter?/frontend? + + * - ``MEDIA_INTF_T_DVB_DEMUX`` + - Device node interface for the Digital TV demux + - typically, /dev/dvb/adapter?/demux? + + * - ``MEDIA_INTF_T_DVB_DVR`` + - Device node interface for the Digital TV DVR + - typically, /dev/dvb/adapter?/dvr? + + * - ``MEDIA_INTF_T_DVB_CA`` + - Device node interface for the Digital TV Conditional Access + - typically, /dev/dvb/adapter?/ca? + + * - ``MEDIA_INTF_T_DVB_NET`` + - Device node interface for the Digital TV network control + - typically, /dev/dvb/adapter?/net? + + * - ``MEDIA_INTF_T_V4L_VIDEO`` + - Device node interface for video (V4L) + - typically, /dev/video? + + * - ``MEDIA_INTF_T_V4L_VBI`` + - Device node interface for VBI (V4L) + - typically, /dev/vbi? + + * - ``MEDIA_INTF_T_V4L_RADIO`` + - Device node interface for radio (V4L) + - typically, /dev/radio? + + * - ``MEDIA_INTF_T_V4L_SUBDEV`` + - Device node interface for a V4L subdevice + - typically, /dev/v4l-subdev? + + * - ``MEDIA_INTF_T_V4L_SWRADIO`` + - Device node interface for Software Defined Radio (V4L) + - typically, /dev/swradio? + + * - ``MEDIA_INTF_T_V4L_TOUCH`` + - Device node interface for Touch device (V4L) + - typically, /dev/v4l-touch? + + * - ``MEDIA_INTF_T_ALSA_PCM_CAPTURE`` + - Device node interface for ALSA PCM Capture + - typically, /dev/snd/pcmC?D?c + + * - ``MEDIA_INTF_T_ALSA_PCM_PLAYBACK`` + - Device node interface for ALSA PCM Playback + - typically, /dev/snd/pcmC?D?p + + * - ``MEDIA_INTF_T_ALSA_CONTROL`` + - Device node interface for ALSA Control + - typically, /dev/snd/controlC? + + * - ``MEDIA_INTF_T_ALSA_COMPRESS`` + - Device node interface for ALSA Compress + - typically, /dev/snd/compr? + + * - ``MEDIA_INTF_T_ALSA_RAWMIDI`` + - Device node interface for ALSA Raw MIDI + - typically, /dev/snd/midi? + + * - ``MEDIA_INTF_T_ALSA_HWDEP`` + - Device node interface for ALSA Hardware Dependent + - typically, /dev/snd/hwC?D? + + * - ``MEDIA_INTF_T_ALSA_SEQUENCER`` + - Device node interface for ALSA Sequencer + - typically, /dev/snd/seq + + * - ``MEDIA_INTF_T_ALSA_TIMER`` + - Device node interface for ALSA Timer + - typically, /dev/snd/timer + + +.. tabularcolumns:: |p{5.5cm}|p{12.0cm}| + +.. _media-pad-flag: +.. _MEDIA-PAD-FL-SINK: +.. _MEDIA-PAD-FL-SOURCE: +.. _MEDIA-PAD-FL-MUST-CONNECT: + +.. flat-table:: Media pad flags + :header-rows: 0 + :stub-columns: 0 + + * - ``MEDIA_PAD_FL_SINK`` + - Input pad, relative to the entity. Input pads sink data and are + targets of links. + + * - ``MEDIA_PAD_FL_SOURCE`` + - Output pad, relative to the entity. Output pads source data and + are origins of links. + + * - ``MEDIA_PAD_FL_MUST_CONNECT`` + - If this flag is set and the pad is linked to any other pad, then + at least one of those links must be enabled for the entity to be + able to stream. There could be temporary reasons (e.g. device + configuration dependent) for the pad to need enabled links even + when this flag isn't set; the absence of the flag doesn't imply + there is none. + + +One and only one of ``MEDIA_PAD_FL_SINK`` and ``MEDIA_PAD_FL_SOURCE`` +must be set for every pad. + +.. tabularcolumns:: |p{5.5cm}|p{12.0cm}| + +.. _media-link-flag: +.. _MEDIA-LNK-FL-ENABLED: +.. _MEDIA-LNK-FL-IMMUTABLE: +.. _MEDIA-LNK-FL-DYNAMIC: +.. _MEDIA-LNK-FL-LINK-TYPE: + +.. flat-table:: Media link flags + :header-rows: 0 + :stub-columns: 0 + + * - ``MEDIA_LNK_FL_ENABLED`` + - The link is enabled and can be used to transfer media data. When + two or more links target a sink pad, only one of them can be + enabled at a time. + + * - ``MEDIA_LNK_FL_IMMUTABLE`` + - The link enabled state can't be modified at runtime. An immutable + link is always enabled. + + * - ``MEDIA_LNK_FL_DYNAMIC`` + - The link enabled state can be modified during streaming. This flag + is set by drivers and is read-only for applications. + + * - ``MEDIA_LNK_FL_LINK_TYPE`` + - This is a bitmask that defines the type of the link. The following + link types are currently supported: + + .. _MEDIA-LNK-FL-DATA-LINK: + + ``MEDIA_LNK_FL_DATA_LINK`` for links that represent a data connection + between two pads. + + .. _MEDIA-LNK-FL-INTERFACE-LINK: + + ``MEDIA_LNK_FL_INTERFACE_LINK`` for links that associate an entity to its + interface. + + .. _MEDIA-LNK-FL-ANCILLARY-LINK: + + ``MEDIA_LNK_FL_ANCILLARY_LINK`` for links that represent a physical + relationship between two entities. The link may or may not be + immutable, so applications must not assume either case. diff --git a/Documentation/userspace-api/media/mediactl/request-api.rst b/Documentation/userspace-api/media/mediactl/request-api.rst new file mode 100644 index 000000000..6c4cbd9f0 --- /dev/null +++ b/Documentation/userspace-api/media/mediactl/request-api.rst @@ -0,0 +1,253 @@ +.. SPDX-License-Identifier: GPL-2.0 OR GFDL-1.1-no-invariants-or-later +.. c:namespace:: MC + +.. _media-request-api: + +Request API +=========== + +The Request API has been designed to allow V4L2 to deal with requirements of +modern devices (stateless codecs, complex camera pipelines, ...) and APIs +(Android Codec v2). One such requirement is the ability for devices belonging to +the same pipeline to reconfigure and collaborate closely on a per-frame basis. +Another is support of stateless codecs, which require controls to be applied +to specific frames (aka 'per-frame controls') in order to be used efficiently. + +While the initial use-case was V4L2, it can be extended to other subsystems +as well, as long as they use the media controller. + +Supporting these features without the Request API is not always possible and if +it is, it is terribly inefficient: user-space would have to flush all activity +on the media pipeline, reconfigure it for the next frame, queue the buffers to +be processed with that configuration, and wait until they are all available for +dequeuing before considering the next frame. This defeats the purpose of having +buffer queues since in practice only one buffer would be queued at a time. + +The Request API allows a specific configuration of the pipeline (media +controller topology + configuration for each media entity) to be associated with +specific buffers. This allows user-space to schedule several tasks ("requests") +with different configurations in advance, knowing that the configuration will be +applied when needed to get the expected result. Configuration values at the time +of request completion are also available for reading. + +General Usage +------------- + +The Request API extends the Media Controller API and cooperates with +subsystem-specific APIs to support request usage. At the Media Controller +level, requests are allocated from the supporting Media Controller device +node. Their life cycle is then managed through the request file descriptors in +an opaque way. Configuration data, buffer handles and processing results +stored in requests are accessed through subsystem-specific APIs extended for +request support, such as V4L2 APIs that take an explicit ``request_fd`` +parameter. + +Request Allocation +------------------ + +User-space allocates requests using :ref:`MEDIA_IOC_REQUEST_ALLOC` +for the media device node. This returns a file descriptor representing the +request. Typically, several such requests will be allocated. + +Request Preparation +------------------- + +Standard V4L2 ioctls can then receive a request file descriptor to express the +fact that the ioctl is part of said request, and is not to be applied +immediately. See :ref:`MEDIA_IOC_REQUEST_ALLOC` for a list of ioctls that +support this. Configurations set with a ``request_fd`` parameter are stored +instead of being immediately applied, and buffers queued to a request do not +enter the regular buffer queue until the request itself is queued. + +Request Submission +------------------ + +Once the configuration and buffers of the request are specified, it can be +queued by calling :ref:`MEDIA_REQUEST_IOC_QUEUE` on the request file descriptor. +A request must contain at least one buffer, otherwise ``ENOENT`` is returned. +A queued request cannot be modified anymore. + +.. caution:: + For :ref:`memory-to-memory devices <mem2mem>` you can use requests only for + output buffers, not for capture buffers. Attempting to add a capture buffer + to a request will result in an ``EBADR`` error. + +If the request contains configurations for multiple entities, individual drivers +may synchronize so the requested pipeline's topology is applied before the +buffers are processed. Media controller drivers do a best effort implementation +since perfect atomicity may not be possible due to hardware limitations. + +.. caution:: + + It is not allowed to mix queuing requests with directly queuing buffers: + whichever method is used first locks this in place until + :ref:`VIDIOC_STREAMOFF <VIDIOC_STREAMON>` is called or the device is + :ref:`closed <func-close>`. Attempts to directly queue a buffer when earlier + a buffer was queued via a request or vice versa will result in an ``EBUSY`` + error. + +Controls can still be set without a request and are applied immediately, +regardless of whether a request is in use or not. + +.. caution:: + + Setting the same control through a request and also directly can lead to + undefined behavior! + +User-space can :c:func:`poll()` a request file descriptor in +order to wait until the request completes. A request is considered complete +once all its associated buffers are available for dequeuing and all the +associated controls have been updated with the values at the time of completion. +Note that user-space does not need to wait for the request to complete to +dequeue its buffers: buffers that are available halfway through a request can +be dequeued independently of the request's state. + +A completed request contains the state of the device after the request was +executed. User-space can query that state by calling +:ref:`ioctl VIDIOC_G_EXT_CTRLS <VIDIOC_G_EXT_CTRLS>` with the request file +descriptor. Calling :ref:`ioctl VIDIOC_G_EXT_CTRLS <VIDIOC_G_EXT_CTRLS>` for a +request that has been queued but not yet completed will return ``EBUSY`` +since the control values might be changed at any time by the driver while the +request is in flight. + +.. _media-request-life-time: + +Recycling and Destruction +------------------------- + +Finally, a completed request can either be discarded or be reused. Calling +:c:func:`close()` on a request file descriptor will make +that file descriptor unusable and the request will be freed once it is no +longer in use by the kernel. That is, if the request is queued and then the +file descriptor is closed, then it won't be freed until the driver completed +the request. + +The :ref:`MEDIA_REQUEST_IOC_REINIT` will clear a request's state and make it +available again. No state is retained by this operation: the request is as +if it had just been allocated. + +Example for a Codec Device +-------------------------- + +For use-cases such as :ref:`codecs <mem2mem>`, the request API can be used +to associate specific controls to +be applied by the driver for the OUTPUT buffer, allowing user-space +to queue many such buffers in advance. It can also take advantage of requests' +ability to capture the state of controls when the request completes to read back +information that may be subject to change. + +Put into code, after obtaining a request, user-space can assign controls and one +OUTPUT buffer to it: + +.. code-block:: c + + struct v4l2_buffer buf; + struct v4l2_ext_controls ctrls; + int req_fd; + ... + if (ioctl(media_fd, MEDIA_IOC_REQUEST_ALLOC, &req_fd)) + return errno; + ... + ctrls.which = V4L2_CTRL_WHICH_REQUEST_VAL; + ctrls.request_fd = req_fd; + if (ioctl(codec_fd, VIDIOC_S_EXT_CTRLS, &ctrls)) + return errno; + ... + buf.type = V4L2_BUF_TYPE_VIDEO_OUTPUT; + buf.flags |= V4L2_BUF_FLAG_REQUEST_FD; + buf.request_fd = req_fd; + if (ioctl(codec_fd, VIDIOC_QBUF, &buf)) + return errno; + +Note that it is not allowed to use the Request API for CAPTURE buffers +since there are no per-frame settings to report there. + +Once the request is fully prepared, it can be queued to the driver: + +.. code-block:: c + + if (ioctl(req_fd, MEDIA_REQUEST_IOC_QUEUE)) + return errno; + +User-space can then either wait for the request to complete by calling poll() on +its file descriptor, or start dequeuing CAPTURE buffers. Most likely, it will +want to get CAPTURE buffers as soon as possible and this can be done using a +regular :ref:`VIDIOC_DQBUF <VIDIOC_QBUF>`: + +.. code-block:: c + + struct v4l2_buffer buf; + + memset(&buf, 0, sizeof(buf)); + buf.type = V4L2_BUF_TYPE_VIDEO_CAPTURE; + if (ioctl(codec_fd, VIDIOC_DQBUF, &buf)) + return errno; + +Note that this example assumes for simplicity that for every OUTPUT buffer +there will be one CAPTURE buffer, but this does not have to be the case. + +We can then, after ensuring that the request is completed via polling the +request file descriptor, query control values at the time of its completion via +a call to :ref:`VIDIOC_G_EXT_CTRLS <VIDIOC_G_EXT_CTRLS>`. +This is particularly useful for volatile controls for which we want to +query values as soon as the capture buffer is produced. + +.. code-block:: c + + struct pollfd pfd = { .events = POLLPRI, .fd = req_fd }; + poll(&pfd, 1, -1); + ... + ctrls.which = V4L2_CTRL_WHICH_REQUEST_VAL; + ctrls.request_fd = req_fd; + if (ioctl(codec_fd, VIDIOC_G_EXT_CTRLS, &ctrls)) + return errno; + +Once we don't need the request anymore, we can either recycle it for reuse with +:ref:`MEDIA_REQUEST_IOC_REINIT`... + +.. code-block:: c + + if (ioctl(req_fd, MEDIA_REQUEST_IOC_REINIT)) + return errno; + +... or close its file descriptor to completely dispose of it. + +.. code-block:: c + + close(req_fd); + +Example for a Simple Capture Device +----------------------------------- + +With a simple capture device, requests can be used to specify controls to apply +for a given CAPTURE buffer. + +.. code-block:: c + + struct v4l2_buffer buf; + struct v4l2_ext_controls ctrls; + int req_fd; + ... + if (ioctl(media_fd, MEDIA_IOC_REQUEST_ALLOC, &req_fd)) + return errno; + ... + ctrls.which = V4L2_CTRL_WHICH_REQUEST_VAL; + ctrls.request_fd = req_fd; + if (ioctl(camera_fd, VIDIOC_S_EXT_CTRLS, &ctrls)) + return errno; + ... + buf.type = V4L2_BUF_TYPE_VIDEO_CAPTURE; + buf.flags |= V4L2_BUF_FLAG_REQUEST_FD; + buf.request_fd = req_fd; + if (ioctl(camera_fd, VIDIOC_QBUF, &buf)) + return errno; + +Once the request is fully prepared, it can be queued to the driver: + +.. code-block:: c + + if (ioctl(req_fd, MEDIA_REQUEST_IOC_QUEUE)) + return errno; + +User-space can then dequeue buffers, wait for the request completion, query +controls and recycle the request as in the M2M example above. diff --git a/Documentation/userspace-api/media/mediactl/request-func-close.rst b/Documentation/userspace-api/media/mediactl/request-func-close.rst new file mode 100644 index 000000000..f4b8eb385 --- /dev/null +++ b/Documentation/userspace-api/media/mediactl/request-func-close.rst @@ -0,0 +1,45 @@ +.. SPDX-License-Identifier: GPL-2.0 OR GFDL-1.1-no-invariants-or-later +.. c:namespace:: MC.request + +.. _request-func-close: + +*************** +request close() +*************** + +Name +==== + +request-close - Close a request file descriptor + +Synopsis +======== + +.. code-block:: c + + #include <unistd.h> + +.. c:function:: int close( int fd ) + +Arguments +========= + +``fd`` + File descriptor returned by :ref:`MEDIA_IOC_REQUEST_ALLOC`. + +Description +=========== + +Closes the request file descriptor. Resources associated with the request +are freed once all file descriptors associated with the request are closed +and the driver has completed the request. +See :ref:`here <media-request-life-time>` for more information. + +Return Value +============ + +:c:func:`close()` returns 0 on success. On error, -1 is +returned, and ``errno`` is set appropriately. Possible error codes are: + +EBADF + ``fd`` is not a valid open file descriptor. diff --git a/Documentation/userspace-api/media/mediactl/request-func-ioctl.rst b/Documentation/userspace-api/media/mediactl/request-func-ioctl.rst new file mode 100644 index 000000000..4fb3d2ef3 --- /dev/null +++ b/Documentation/userspace-api/media/mediactl/request-func-ioctl.rst @@ -0,0 +1,63 @@ +.. SPDX-License-Identifier: GPL-2.0 OR GFDL-1.1-no-invariants-or-later +.. c:namespace:: MC + +.. _request-func-ioctl: + +*************** +request ioctl() +*************** + +Name +==== + +request-ioctl - Control a request file descriptor + +Synopsis +======== + +.. code-block:: c + + #include <sys/ioctl.h> + +``int ioctl(int fd, int cmd, void *argp)`` + +Arguments +========= + +``fd`` + File descriptor returned by :ref:`MEDIA_IOC_REQUEST_ALLOC`. + +``cmd`` + The request ioctl command code as defined in the media.h header file, for + example :ref:`MEDIA_REQUEST_IOC_QUEUE`. + +``argp`` + Pointer to a request-specific structure. + +Description +=========== + +The :ref:`ioctl() <request-func-ioctl>` function manipulates request +parameters. The argument ``fd`` must be an open file descriptor. + +The ioctl ``cmd`` code specifies the request function to be called. It +has encoded in it whether the argument is an input, output or read/write +parameter, and the size of the argument ``argp`` in bytes. + +Macros and structures definitions specifying request ioctl commands and +their parameters are located in the media.h header file. All request ioctl +commands, their respective function and parameters are specified in +:ref:`media-user-func`. + +Return Value +============ + +On success 0 is returned, on error -1 and the ``errno`` variable is set +appropriately. The generic error codes are described at the +:ref:`Generic Error Codes <gen-errors>` chapter. + +Command-specific error codes are listed in the individual command +descriptions. + +When an ioctl that takes an output or read/write parameter fails, the +parameter remains unmodified. diff --git a/Documentation/userspace-api/media/mediactl/request-func-poll.rst b/Documentation/userspace-api/media/mediactl/request-func-poll.rst new file mode 100644 index 000000000..ce0043dbe --- /dev/null +++ b/Documentation/userspace-api/media/mediactl/request-func-poll.rst @@ -0,0 +1,73 @@ +.. SPDX-License-Identifier: GPL-2.0 OR GFDL-1.1-no-invariants-or-later +.. c:namespace:: MC + +.. _request-func-poll: + +************** +request poll() +************** + +Name +==== + +request-poll - Wait for some event on a file descriptor + +Synopsis +======== + +.. code-block:: c + + #include <sys/poll.h> + +.. c:function:: int poll( struct pollfd *ufds, unsigned int nfds, int timeout ) + +Arguments +========= + +``ufds`` + List of file descriptor events to be watched + +``nfds`` + Number of file descriptor events at the \*ufds array + +``timeout`` + Timeout to wait for events + +Description +=========== + +With the :c:func:`poll()` function applications can wait +for a request to complete. + +On success :c:func:`poll()` returns the number of file +descriptors that have been selected (that is, file descriptors for which the +``revents`` field of the respective struct :c:type:`pollfd` +is non-zero). Request file descriptor set the ``POLLPRI`` flag in ``revents`` +when the request was completed. When the function times out it returns +a value of zero, on failure it returns -1 and the ``errno`` variable is +set appropriately. + +Attempting to poll for a request that is not yet queued will +set the ``POLLERR`` flag in ``revents``. + +Return Value +============ + +On success, :c:func:`poll()` returns the number of +structures which have non-zero ``revents`` fields, or zero if the call +timed out. On error -1 is returned, and the ``errno`` variable is set +appropriately: + +``EBADF`` + One or more of the ``ufds`` members specify an invalid file + descriptor. + +``EFAULT`` + ``ufds`` references an inaccessible memory area. + +``EINTR`` + The call was interrupted by a signal. + +``EINVAL`` + The ``nfds`` value exceeds the ``RLIMIT_NOFILE`` value. Use + ``getrlimit()`` to obtain this value. |