1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
|
# User Space Drivers {#userspace}
# Controlling Hardware From User Space {#userspace_control}
Much of the documentation for SPDK talks about _user space drivers_, so it's
important to understand what that means at a technical level. First and
foremost, a _driver_ is software that directly controls a particular device
attached to a computer. Second, operating systems segregate the system's
virtual memory into two categories of addresses based on privilege level -
[kernel space and user space](https://en.wikipedia.org/wiki/User_space). This
separation is aided by features on the CPU itself that enforce memory
separation called
[protection rings](https://en.wikipedia.org/wiki/Protection_ring). Typically,
drivers run in kernel space (i.e. ring 0 on x86). SPDK contains drivers that
instead are designed to run in user space, but they still interface directly
with the hardware device that they are controlling.
In order for SPDK to take control of a device, it must first instruct the
operating system to relinquish control. This is often referred to as unbinding
the kernel driver from the device and on Linux is done by
[writing to a file in sysfs](https://lwn.net/Articles/143397/).
SPDK then rebinds the driver to one of two special device drivers that come
bundled with Linux -
[uio](https://www.kernel.org/doc/html/latest/driver-api/uio-howto.html) or
[vfio](https://www.kernel.org/doc/Documentation/vfio.txt). These two drivers
are "dummy" drivers in the sense that they mostly indicate to the operating
system that the device has a driver bound to it so it won't automatically try
to re-bind the default driver. They don't actually initialize the hardware in
any way, nor do they even understand what type of device it is. The primary
difference between uio and vfio is that vfio is capable of programming the
platform's
[IOMMU](https://en.wikipedia.org/wiki/Input%E2%80%93output_memory_management_unit),
which is a critical piece of hardware for ensuring memory safety in user space
drivers. See @ref memory for full details.
Once the device is unbound from the operating system kernel, the operating
system can't use it anymore. For example, if you unbind an NVMe device on Linux,
the devices corresponding to it such as /dev/nvme0n1 will disappear. It further
means that filesystems mounted on the device will also be removed and kernel
filesystems can no longer interact with the device. In fact, the entire kernel
block storage stack is no longer involved. Instead, SPDK provides re-imagined
implementations of most of the layers in a typical operating system storage
stack all as C libraries that can be directly embedded into your application.
This includes a [block device abstraction layer](@ref bdev) primarily, but
also [block allocators](@ref blob) and [filesystem-like components](@ref blobfs).
User space drivers utilize features in uio or vfio to map the
[PCI BAR](https://en.wikipedia.org/wiki/PCI_configuration_space) for the device
into the current process, which allows the driver to perform
[MMIO](https://en.wikipedia.org/wiki/Memory-mapped_I/O) directly. The SPDK @ref
nvme, for instance, maps the BAR for the NVMe device and then follows along
with the
[NVMe Specification](http://nvmexpress.org/wp-content/uploads/NVM_Express_Revision_1.3.pdf)
to initialize the device, create queue pairs, and ultimately send I/O.
# Interrupts {#userspace_interrupts}
SPDK polls devices for completions instead of waiting for interrupts. There
are a number of reasons for doing this: 1) practically speaking, routing an
interrupt to a handler in a user space process just isn't feasible for most
hardware designs, 2) interrupts introduce software jitter and have significant
overhead due to forced context switches. Operations in SPDK are almost
universally asynchronous and allow the user to provide a callback on
completion. The callback is called in response to the user calling a function
to poll for completions. Polling an NVMe device is fast because only host
memory needs to be read (no MMIO) to check a queue pair for a bit flip and
technologies such as Intel's
[DDIO](https://www.intel.com/content/www/us/en/io/data-direct-i-o-technology.html)
will ensure that the host memory being checked is present in the CPU cache
after an update by the device.
# Threading {#userspace_threading}
NVMe devices expose multiple queues for submitting requests to the hardware.
Separate queues can be accessed without coordination, so software can send
requests to the device from multiple threads of execution in parallel without
locks. Unfortunately, kernel drivers must be designed to handle I/O coming
from lots of different places either in the operating system or in various
processes on the system, and the thread topology of those processes changes
over time. Most kernel drivers elect to map hardware queues to cores (as close
to 1:1 as possible), and then when a request is submitted they look up the
correct hardware queue for whatever core the current thread happens to be
running on. Often, they'll need to either acquire a lock around the queue or
temporarily disable interrupts to guard against preemption from threads
running on the same core, which can be expensive. This is a large improvement
from older hardware interfaces that only had a single queue or no queue at
all, but still isn't always optimal.
A user space driver, on the other hand, is embedded into a single application.
This application knows exactly how many threads (or processes) exist
because the application created them. Therefore, the SPDK drivers choose to
expose the hardware queues directly to the application with the requirement
that a hardware queue is only ever accessed from one thread at a time. In
practice, applications assign one hardware queue to each thread (as opposed to
one hardware queue per core in kernel drivers). This guarantees that the thread
can submit requests without having to perform any sort of coordination (i.e.
locking) with the other threads in the system.
|
