summaryrefslogtreecommitdiffstats
path: root/src/spdk/doc/blobfs.md
diff options
context:
space:
mode:
authorDaniel Baumann <daniel.baumann@progress-linux.org>2024-04-27 18:24:20 +0000
committerDaniel Baumann <daniel.baumann@progress-linux.org>2024-04-27 18:24:20 +0000
commit483eb2f56657e8e7f419ab1a4fab8dce9ade8609 (patch)
treee5d88d25d870d5dedacb6bbdbe2a966086a0a5cf /src/spdk/doc/blobfs.md
parentInitial commit. (diff)
downloadceph-483eb2f56657e8e7f419ab1a4fab8dce9ade8609.tar.xz
ceph-483eb2f56657e8e7f419ab1a4fab8dce9ade8609.zip
Adding upstream version 14.2.21.upstream/14.2.21upstream
Signed-off-by: Daniel Baumann <daniel.baumann@progress-linux.org>
Diffstat (limited to '')
-rw-r--r--src/spdk/doc/blobfs.md88
1 files changed, 88 insertions, 0 deletions
diff --git a/src/spdk/doc/blobfs.md b/src/spdk/doc/blobfs.md
new file mode 100644
index 00000000..5a8ef79e
--- /dev/null
+++ b/src/spdk/doc/blobfs.md
@@ -0,0 +1,88 @@
+# BlobFS (Blobstore Filesystem) {#blobfs}
+
+# BlobFS Getting Started Guide {#blobfs_getting_started}
+
+# RocksDB Integration {#blobfs_rocksdb}
+
+Clone and build the SPDK repository as per https://github.com/spdk/spdk
+
+~~~{.sh}
+git clone https://github.com/spdk/spdk.git
+cd spdk
+./configure
+make
+~~~
+
+Clone the RocksDB repository from the SPDK GitHub fork into a separate directory.
+Make sure you check out the `spdk-v5.6.1` branch.
+
+~~~{.sh}
+cd ..
+git clone -b spdk-v5.6.1 https://github.com/spdk/rocksdb.git
+~~~
+
+Build RocksDB. Only the `db_bench` benchmarking tool is integrated with BlobFS.
+(Note: add `DEBUG_LEVEL=0` for a release build.)
+
+~~~{.sh}
+cd rocksdb
+make db_bench SPDK_DIR=path/to/spdk
+~~~
+
+Create an NVMe section in the configuration file using SPDK's `gen_nvme.sh` script.
+
+~~~{.sh}
+scripts/gen_nvme.sh > /usr/local/etc/spdk/rocksdb.conf
+~~~
+
+Verify the configuration file has specified the correct NVMe SSD.
+If there are any NVMe SSDs you do not wish to use for RocksDB/SPDK testing, remove them from the configuration file.
+
+Make sure you have at least 5GB of memory allocated for huge pages.
+By default, the SPDK `setup.sh` script only allocates 2GB.
+The following will allocate 5GB of huge page memory (in addition to binding the NVMe devices to uio/vfio).
+
+~~~{.sh}
+HUGEMEM=5120 scripts/setup.sh
+~~~
+
+Create an empty SPDK blobfs for testing.
+
+~~~{.sh}
+test/blobfs/mkfs/mkfs /usr/local/etc/spdk/rocksdb.conf Nvme0n1
+~~~
+
+At this point, RocksDB is ready for testing with SPDK. Three `db_bench` parameters are used to configure SPDK:
+
+1. `spdk` - Defines the name of the SPDK configuration file. If omitted, RocksDB will use the default PosixEnv implementation
+ instead of SpdkEnv. (Required)
+2. `spdk_bdev` - Defines the name of the SPDK block device which contains the BlobFS to be used for testing. (Required)
+3. `spdk_cache_size` - Defines the amount of userspace cache memory used by SPDK. Specified in terms of megabytes (MB).
+ Default is 4096 (4GB). (Optional)
+
+SPDK has a set of scripts which will run `db_bench` against a variety of workloads and capture performance and profiling
+data. The primary script is `test/blobfs/rocksdb/run_tests.sh`.
+
+# FUSE
+
+BlobFS provides a FUSE plug-in to mount an SPDK BlobFS as a kernel filesystem for inspection or debug purposes.
+The FUSE plug-in requires fuse3 and will be built automatically when fuse3 is detected on the system.
+
+~~~{.sh}
+test/blobfs/fuse/fuse /usr/local/etc/spdk/rocksdb.conf Nvme0n1 /mnt/fuse
+~~~
+
+Note that the FUSE plug-in has some limitations - see the list below.
+
+# Limitations
+
+* BlobFS has primarily been tested with RocksDB so far, so any use cases different from how RocksDB uses a filesystem
+ may run into issues. BlobFS will be tested in a broader range of use cases after this initial release.
+* Only a synchronous API is currently supported. An asynchronous API has been developed but not thoroughly tested
+ yet so is not part of the public interface yet. This will be added in a future release.
+* File renames are not atomic. This will be fixed in a future release.
+* BlobFS currently supports only a flat namespace for files with no directory support. Filenames are currently stored
+ as xattrs in each blob. This means that filename lookup is an O(n) operation. An SPDK btree implementation is
+ underway which will be the underpinning for BlobFS directory support in a future release.
+* Writes to a file must always append to the end of the file. Support for writes to any location within the file
+ will be added in a future release.