diff options
Diffstat (limited to 'Documentation/userspace-api/media/mediactl/request-api.rst')
| -rw-r--r-- | Documentation/userspace-api/media/mediactl/request-api.rst | 253 |
1 files changed, 253 insertions, 0 deletions
diff --git a/Documentation/userspace-api/media/mediactl/request-api.rst b/Documentation/userspace-api/media/mediactl/request-api.rst new file mode 100644 index 000000000..6c4cbd9f0 --- /dev/null +++ b/Documentation/userspace-api/media/mediactl/request-api.rst @@ -0,0 +1,253 @@ +.. SPDX-License-Identifier: GPL-2.0 OR GFDL-1.1-no-invariants-or-later +.. c:namespace:: MC + +.. _media-request-api: + +Request API +=========== + +The Request API has been designed to allow V4L2 to deal with requirements of +modern devices (stateless codecs, complex camera pipelines, ...) and APIs +(Android Codec v2). One such requirement is the ability for devices belonging to +the same pipeline to reconfigure and collaborate closely on a per-frame basis. +Another is support of stateless codecs, which require controls to be applied +to specific frames (aka 'per-frame controls') in order to be used efficiently. + +While the initial use-case was V4L2, it can be extended to other subsystems +as well, as long as they use the media controller. + +Supporting these features without the Request API is not always possible and if +it is, it is terribly inefficient: user-space would have to flush all activity +on the media pipeline, reconfigure it for the next frame, queue the buffers to +be processed with that configuration, and wait until they are all available for +dequeuing before considering the next frame. This defeats the purpose of having +buffer queues since in practice only one buffer would be queued at a time. + +The Request API allows a specific configuration of the pipeline (media +controller topology + configuration for each media entity) to be associated with +specific buffers. This allows user-space to schedule several tasks ("requests") +with different configurations in advance, knowing that the configuration will be +applied when needed to get the expected result. Configuration values at the time +of request completion are also available for reading. + +General Usage +------------- + +The Request API extends the Media Controller API and cooperates with +subsystem-specific APIs to support request usage. At the Media Controller +level, requests are allocated from the supporting Media Controller device +node. Their life cycle is then managed through the request file descriptors in +an opaque way. Configuration data, buffer handles and processing results +stored in requests are accessed through subsystem-specific APIs extended for +request support, such as V4L2 APIs that take an explicit ``request_fd`` +parameter. + +Request Allocation +------------------ + +User-space allocates requests using :ref:`MEDIA_IOC_REQUEST_ALLOC` +for the media device node. This returns a file descriptor representing the +request. Typically, several such requests will be allocated. + +Request Preparation +------------------- + +Standard V4L2 ioctls can then receive a request file descriptor to express the +fact that the ioctl is part of said request, and is not to be applied +immediately. See :ref:`MEDIA_IOC_REQUEST_ALLOC` for a list of ioctls that +support this. Configurations set with a ``request_fd`` parameter are stored +instead of being immediately applied, and buffers queued to a request do not +enter the regular buffer queue until the request itself is queued. + +Request Submission +------------------ + +Once the configuration and buffers of the request are specified, it can be +queued by calling :ref:`MEDIA_REQUEST_IOC_QUEUE` on the request file descriptor. +A request must contain at least one buffer, otherwise ``ENOENT`` is returned. +A queued request cannot be modified anymore. + +.. caution:: + For :ref:`memory-to-memory devices <mem2mem>` you can use requests only for + output buffers, not for capture buffers. Attempting to add a capture buffer + to a request will result in an ``EBADR`` error. + +If the request contains configurations for multiple entities, individual drivers +may synchronize so the requested pipeline's topology is applied before the +buffers are processed. Media controller drivers do a best effort implementation +since perfect atomicity may not be possible due to hardware limitations. + +.. caution:: + + It is not allowed to mix queuing requests with directly queuing buffers: + whichever method is used first locks this in place until + :ref:`VIDIOC_STREAMOFF <VIDIOC_STREAMON>` is called or the device is + :ref:`closed <func-close>`. Attempts to directly queue a buffer when earlier + a buffer was queued via a request or vice versa will result in an ``EBUSY`` + error. + +Controls can still be set without a request and are applied immediately, +regardless of whether a request is in use or not. + +.. caution:: + + Setting the same control through a request and also directly can lead to + undefined behavior! + +User-space can :c:func:`poll()` a request file descriptor in +order to wait until the request completes. A request is considered complete +once all its associated buffers are available for dequeuing and all the +associated controls have been updated with the values at the time of completion. +Note that user-space does not need to wait for the request to complete to +dequeue its buffers: buffers that are available halfway through a request can +be dequeued independently of the request's state. + +A completed request contains the state of the device after the request was +executed. User-space can query that state by calling +:ref:`ioctl VIDIOC_G_EXT_CTRLS <VIDIOC_G_EXT_CTRLS>` with the request file +descriptor. Calling :ref:`ioctl VIDIOC_G_EXT_CTRLS <VIDIOC_G_EXT_CTRLS>` for a +request that has been queued but not yet completed will return ``EBUSY`` +since the control values might be changed at any time by the driver while the +request is in flight. + +.. _media-request-life-time: + +Recycling and Destruction +------------------------- + +Finally, a completed request can either be discarded or be reused. Calling +:c:func:`close()` on a request file descriptor will make +that file descriptor unusable and the request will be freed once it is no +longer in use by the kernel. That is, if the request is queued and then the +file descriptor is closed, then it won't be freed until the driver completed +the request. + +The :ref:`MEDIA_REQUEST_IOC_REINIT` will clear a request's state and make it +available again. No state is retained by this operation: the request is as +if it had just been allocated. + +Example for a Codec Device +-------------------------- + +For use-cases such as :ref:`codecs <mem2mem>`, the request API can be used +to associate specific controls to +be applied by the driver for the OUTPUT buffer, allowing user-space +to queue many such buffers in advance. It can also take advantage of requests' +ability to capture the state of controls when the request completes to read back +information that may be subject to change. + +Put into code, after obtaining a request, user-space can assign controls and one +OUTPUT buffer to it: + +.. code-block:: c + + struct v4l2_buffer buf; + struct v4l2_ext_controls ctrls; + int req_fd; + ... + if (ioctl(media_fd, MEDIA_IOC_REQUEST_ALLOC, &req_fd)) + return errno; + ... + ctrls.which = V4L2_CTRL_WHICH_REQUEST_VAL; + ctrls.request_fd = req_fd; + if (ioctl(codec_fd, VIDIOC_S_EXT_CTRLS, &ctrls)) + return errno; + ... + buf.type = V4L2_BUF_TYPE_VIDEO_OUTPUT; + buf.flags |= V4L2_BUF_FLAG_REQUEST_FD; + buf.request_fd = req_fd; + if (ioctl(codec_fd, VIDIOC_QBUF, &buf)) + return errno; + +Note that it is not allowed to use the Request API for CAPTURE buffers +since there are no per-frame settings to report there. + +Once the request is fully prepared, it can be queued to the driver: + +.. code-block:: c + + if (ioctl(req_fd, MEDIA_REQUEST_IOC_QUEUE)) + return errno; + +User-space can then either wait for the request to complete by calling poll() on +its file descriptor, or start dequeuing CAPTURE buffers. Most likely, it will +want to get CAPTURE buffers as soon as possible and this can be done using a +regular :ref:`VIDIOC_DQBUF <VIDIOC_QBUF>`: + +.. code-block:: c + + struct v4l2_buffer buf; + + memset(&buf, 0, sizeof(buf)); + buf.type = V4L2_BUF_TYPE_VIDEO_CAPTURE; + if (ioctl(codec_fd, VIDIOC_DQBUF, &buf)) + return errno; + +Note that this example assumes for simplicity that for every OUTPUT buffer +there will be one CAPTURE buffer, but this does not have to be the case. + +We can then, after ensuring that the request is completed via polling the +request file descriptor, query control values at the time of its completion via +a call to :ref:`VIDIOC_G_EXT_CTRLS <VIDIOC_G_EXT_CTRLS>`. +This is particularly useful for volatile controls for which we want to +query values as soon as the capture buffer is produced. + +.. code-block:: c + + struct pollfd pfd = { .events = POLLPRI, .fd = req_fd }; + poll(&pfd, 1, -1); + ... + ctrls.which = V4L2_CTRL_WHICH_REQUEST_VAL; + ctrls.request_fd = req_fd; + if (ioctl(codec_fd, VIDIOC_G_EXT_CTRLS, &ctrls)) + return errno; + +Once we don't need the request anymore, we can either recycle it for reuse with +:ref:`MEDIA_REQUEST_IOC_REINIT`... + +.. code-block:: c + + if (ioctl(req_fd, MEDIA_REQUEST_IOC_REINIT)) + return errno; + +... or close its file descriptor to completely dispose of it. + +.. code-block:: c + + close(req_fd); + +Example for a Simple Capture Device +----------------------------------- + +With a simple capture device, requests can be used to specify controls to apply +for a given CAPTURE buffer. + +.. code-block:: c + + struct v4l2_buffer buf; + struct v4l2_ext_controls ctrls; + int req_fd; + ... + if (ioctl(media_fd, MEDIA_IOC_REQUEST_ALLOC, &req_fd)) + return errno; + ... + ctrls.which = V4L2_CTRL_WHICH_REQUEST_VAL; + ctrls.request_fd = req_fd; + if (ioctl(camera_fd, VIDIOC_S_EXT_CTRLS, &ctrls)) + return errno; + ... + buf.type = V4L2_BUF_TYPE_VIDEO_CAPTURE; + buf.flags |= V4L2_BUF_FLAG_REQUEST_FD; + buf.request_fd = req_fd; + if (ioctl(camera_fd, VIDIOC_QBUF, &buf)) + return errno; + +Once the request is fully prepared, it can be queued to the driver: + +.. code-block:: c + + if (ioctl(req_fd, MEDIA_REQUEST_IOC_QUEUE)) + return errno; + +User-space can then dequeue buffers, wait for the request completion, query +controls and recycle the request as in the M2M example above. |