diff options
Diffstat (limited to 'src/seastar/doc/io-tester.md')
| -rw-r--r-- | src/seastar/doc/io-tester.md | 81 |
1 files changed, 81 insertions, 0 deletions
diff --git a/src/seastar/doc/io-tester.md b/src/seastar/doc/io-tester.md new file mode 100644 index 000000000..6c75072e1 --- /dev/null +++ b/src/seastar/doc/io-tester.md @@ -0,0 +1,81 @@ +## I/O Tester utility + +The I/O Tester utility, `io_tester` generates a user-defined I/O pattern +spanning one of multiple shards that is designed to simulate the I/O behavior +of a complex Seastar application. + +# Running I/O tester: + +I/O tester takes the same options as Seastar, and those options may be used +to test the behavior of I/O under the circumnstances established by those +options. For instance, one may adjust the `--task-quota-ms` option to see +if that affects higher percentile latencies. + +Aside from the usual seastar options, I/O tester accepts the following options: + +* `duration`: for how long to run the evaluation, +* `directory`: a directory where to run the evaluation (it must be on XFS), +* `conf`: the path to a YAML file describing the evaluation. + +# Describing the evaluation + +The evaluation is described in a YAML file that contains multiple classes. +Each class spans jobs of similar characteristics in different shards and (for now) +all jobs run concurrently. + +The YAML file contains a list of maps where each element of the list describes a class. +A class has some properties that are common to all elements of the class, and a nested map +that contain properties of a job (class instance in a shard) + +For example: + +``` +- name: big_writes + type: seqread + shards: all + shard_info: + parallelism: 10 + reqsize: 256kB + shares: 10 + think_time: 0 +``` + +* `name`: mandatory property, a string that identifies jobs of this class +* `type`: mandatory property, one of seqread, seqwrite, randread, randwrite, append, cpu +* `shards`: mandatory property, either the string "all" or a list of shards where this class should place jobs. + +The properties under `shard_info` represent properties of the job that will +be replicated to each shard. All properties under `shard_info` are optional, and in case not specified, defaults are used. + +* `parallelism`: the amount of parallel requests this job will generate in a specific shard. Requests can be either active or thinking (see `think_time`) +* `reqsize` : (I/O loads only) the size of requests generated by this job +* `shares` : how many shares requests in this job will have in the scheduler +* `think_time`: how long to wait before submitting another request in this job once one finishes. +* `execution_time`: (cpu loads only) for how long to execute a CPU loop + +# Example output + +``` + Creating initial files... + Starting evaluation... + Shard 0 + Class 0(big_writes: 10 shares, 262144-byte SEQ WRITE, 10 concurrent requests, NO think time) + Throughput : 436556 KB/s + Lat average : 5847 usec + Lat quantile= 0.5 : 2678 usec + Lat quantile= 0.95 : 13029 usec + Lat quantile= 0.99 : 20835 usec + Lat quantile=0.999 : 246090 usec + Lat max : 450785 usec +``` + +# Future + +Some ideas for extending I/O tester: + +* allow properties like think time, request size, etc, to be specified as distributions instead of a fixed number +* allow classes to have class-wide properties. For instance, we could define a class with parallelism of 100, and distribute those 100 requests over all shards in which this class is placed +* allow some jobs to be executed sequentially in relationship to others, so we can have preparation jobs. +* support other types, like delete, fsync, etc. +* provide functionality similar to diskplorer. + |