diff options
Diffstat (limited to 'doc/rbd/rbd-live-migration.rst')
| -rw-r--r-- | doc/rbd/rbd-live-migration.rst | 367 |
1 files changed, 367 insertions, 0 deletions
diff --git a/doc/rbd/rbd-live-migration.rst b/doc/rbd/rbd-live-migration.rst new file mode 100644 index 000000000..c3e09193d --- /dev/null +++ b/doc/rbd/rbd-live-migration.rst @@ -0,0 +1,367 @@ +====================== + Image Live-Migration +====================== + +.. index:: Ceph Block Device; live-migration + +RBD images can be live-migrated between different pools within the same cluster; +between different image formats and layouts; or from external data sources. +When started, the source will be deep-copied to the destination image, pulling +all snapshot history while preserving the sparse allocation of data where +possible. + +By default, when live-migrating RBD images within the same Ceph cluster, the +source image will be marked read-only and all clients will instead redirect +IOs to the new target image. In addition, this mode can optionally preserve the +link to the source image's parent to preserve sparseness, or it can flatten the +image during the migration to remove the dependency on the source image's +parent. + +The live-migration process can also be used in an import-only mode where the +source image remains unmodified and the target image can be linked to an +external data source such as a backing file, HTTP(s) file, or S3 object. + +The live-migration copy process can safely run in the background while the new +target image is in use. There is currently a requirement to temporarily stop +using the source image before preparing a migration when not using the +import-only mode of operation. This helps to ensure that the client using the +image is updated to point to the new target image. + +.. note:: + Image live-migration requires the Ceph Nautilus release or later. Support for + external data sources requires the Ceph Pacific release of later. The + ``krbd`` kernel module does not support live-migration at this time. + + +.. ditaa:: + + +-------------+ +-------------+ + | {s} c999 | | {s} | + | Live | Target refers | Live | + | migration |<-------------*| migration | + | source | to Source | target | + | | | | + | (read only) | | (writable) | + +-------------+ +-------------+ + + Source Target + +The live-migration process is comprised of three steps: + +#. **Prepare Migration:** The initial step creates the new target image and + links the target image to the source. When not configured in the import-only + mode, the source image will also be linked to the target image and marked + read-only. + + Similar to `layered images`_, attempts to read uninitialized data extents + within the target image will internally redirect the read to the source + image, and writes to uninitialized extents within the target will internally + deep-copy the overlapping source image block to the target image. + + +#. **Execute Migration:** This is a background operation that deep-copies all + initialized blocks from the source image to the target. This step can be + run while clients are actively using the new target image. + + +#. **Finish Migration:** Once the background migration process has completed, + the migration can be committed or aborted. Committing the migration will + remove the cross-links between the source and target images, and will + remove the source image if not configured in the import-only mode. Aborting + the migration will remove the cross-links, and will remove the target image. + +Prepare Migration +================= + +The default live-migration process for images within the same Ceph cluster is +initiated by running the `rbd migration prepare` command, providing the source +and target images:: + + $ rbd migration prepare migration_source [migration_target] + +The `rbd migration prepare` command accepts all the same layout optionals as the +`rbd create` command, which allows changes to the immutable image on-disk +layout. The `migration_target` can be skipped if the goal is only to change the +on-disk layout, keeping the original image name. + +All clients using the source image must be stopped prior to preparing a +live-migration. The prepare step will fail if it finds any running clients with +the image open in read/write mode. Once the prepare step is complete, the +clients can be restarted using the new target image name. Attempting to restart +the clients using the source image name will result in failure. + +The `rbd status` command will show the current state of the live-migration:: + + $ rbd status migration_target + Watchers: none + Migration: + source: rbd/migration_source (5e2cba2f62e) + destination: rbd/migration_target (5e2ed95ed806) + state: prepared + +Note that the source image will be moved to the RBD trash to avoid mistaken +usage during the migration process:: + + $ rbd info migration_source + rbd: error opening image migration_source: (2) No such file or directory + $ rbd trash ls --all + 5e2cba2f62e migration_source + + +Prepare Import-Only Migration +============================= + +The import-only live-migration process is initiated by running the same +`rbd migration prepare` command, but adding the `--import-only` optional +and providing a JSON-encoded ``source-spec`` to describe how to access +the source image data. This ``source-spec`` can either be passed +directly via the `--source-spec` optional, or via a file or STDIN via the +`--source-spec-path` optional:: + + $ rbd migration prepare --import-only --source-spec "<JSON>" migration_target + +The `rbd migration prepare` command accepts all the same layout optionals as the +`rbd create` command. + +The `rbd status` command will show the current state of the live-migration:: + + $ rbd status migration_target + Watchers: none + Migration: + source: {"stream":{"file_path":"/mnt/image.raw","type":"file"},"type":"raw"} + destination: rbd/migration_target (ac69113dc1d7) + state: prepared + +The general format for the ``source-spec`` JSON is as follows:: + + { + "type": "<format-type>", + <format unique parameters> + "stream": { + "type": "<stream-type>", + <stream unique parameters> + } + } + +The following formats are currently supported: ``native``, ``qcow``, and +``raw``. The following streams are currently supported: ``file``, ``http``, and +``s3``. + +Formats +~~~~~~~ + +The ``native`` format can be used to describe a native RBD image within a +Ceph cluster as the source image. Its ``source-spec`` JSON is encoded +as follows:: + + { + "type": "native", + "pool_name": "<pool-name>", + ["pool_id": <pool-id>,] (optional alternative to "pool_name") + ["pool_namespace": "<pool-namespace",] (optional) + "image_name": "<image-name>", + ["image_id": "<image-id>",] (optional if image in trash) + "snap_name": "<snap-name>", + ["snap_id": "<snap-id>",] (optional alternative to "snap_name") + } + +Note that the ``native`` format does not include the ``stream`` object since +it utilizes native Ceph operations. For example, to import from the image +``rbd/ns1/image1@snap1``, the ``source-spec`` could be encoded as:: + + { + "type": "native", + "pool_name": "rbd", + "pool_namespace": "ns1", + "image_name": "image1", + "snap_name": "snap1" + } + +The ``qcow`` format can be used to describe a QCOW (QEMU copy-on-write) block +device. Both the QCOW (v1) and QCOW2 formats are currently supported with the +exception of advanced features such as compression, encryption, backing +files, and external data files. Support for these missing features may be added +in a future release. The ``qcow`` format data can be linked to any supported +stream source described below. For example, its base ``source-spec`` JSON is +encoded as follows:: + + { + "type": "qcow", + "stream": { + <stream unique parameters> + } + } + +The ``raw`` format can be used to describe a thick-provisioned, raw block device +export (i.e. `rbd export --export-format 1 <snap-spec>`). The ``raw`` format +data can be linked to any supported stream source described below. For example, +its base ``source-spec`` JSON is encoded as follows:: + + { + "type": "raw", + "stream": { + <stream unique parameters for HEAD, non-snapshot revision> + }, + "snapshots": [ + { + "type": "raw", + "name": "<snapshot-name>", + "stream": { + <stream unique parameters for snapshot> + } + }, + ] (optional oldest to newest ordering of snapshots) + } + +The inclusion of the ``snapshots`` array is optional and currently only supports +thick-provisioned ``raw`` snapshot exports. + +Additional formats such as RBD export-format v2 and RBD export-diff +snapshots will be added in a future release. + +Streams +~~~~~~~ + +The ``file`` stream can be used to import from a locally accessible POSIX file +source. Its ``source-spec`` JSON is encoded as follows:: + + { + <format unique parameters> + "stream": { + "type": "file", + "file_path": "<file-path>" + } + } + +For example, to import a raw-format image from a file located at +"/mnt/image.raw", its ``source-spec`` JSON is encoded as follows:: + + { + "type": "raw", + "stream": { + "type": "file", + "file_path": "/mnt/image.raw" + } + } + +The ``http`` stream can be used to import from a remote HTTP or HTTPS web +server. Its ``source-spec`` JSON is encoded as follows:: + + { + <format unique parameters> + "stream": { + "type": "http", + "url": "<url-path>" + } + } + +For example, to import a raw-format image from a file located at +``http://download.ceph.com/image.raw``, its ``source-spec`` JSON is encoded +as follows:: + + { + "type": "raw", + "stream": { + "type": "http", + "url": "http://download.ceph.com/image.raw" + } + } + +The ``s3`` stream can be used to import from a remote S3 bucket. Its +``source-spec`` JSON is encoded as follows:: + + { + <format unique parameters> + "stream": { + "type": "s3", + "url": "<url-path>", + "access_key": "<access-key>", + "secret_key": "<secret-key>" + } + } + +For example, to import a raw-format image from a file located at +`http://s3.ceph.com/bucket/image.raw`, its ``source-spec`` JSON is encoded +as follows:: + + { + "type": "raw", + "stream": { + "type": "s3", + "url": "http://s3.ceph.com/bucket/image.raw", + "access_key": "NX5QOQKC6BH2IDN8HC7A", + "secret_key": "LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn" + } + } + +.. note:: + The ``access_key`` and ``secret_key`` parameters support storing the keys in + the MON config-key store by prefixing the key values with ``config://`` + followed by the path in the MON config-key store to the value. Values can be + stored in the config-key store via ``ceph config-key set <key-path> <value>`` + (e.g. ``ceph config-key set rbd/s3/access_key NX5QOQKC6BH2IDN8HC7A``). + +Execute Migration +================= + +After preparing the live-migration, the image blocks from the source image +must be copied to the target image. This is accomplished by running the +`rbd migration execute` command:: + + $ rbd migration execute migration_target + Image migration: 100% complete...done. + +The `rbd status` command will also provide feedback on the progress of the +migration block deep-copy process:: + + $ rbd status migration_target + Watchers: + watcher=1.2.3.4:0/3695551461 client.123 cookie=123 + Migration: + source: rbd/migration_source (5e2cba2f62e) + destination: rbd/migration_target (5e2ed95ed806) + state: executing (32% complete) + + +Commit Migration +================ + +Once the live-migration has completed deep-copying all data blocks from the +source image to the target, the migration can be committed:: + + $ rbd status migration_target + Watchers: none + Migration: + source: rbd/migration_source (5e2cba2f62e) + destination: rbd/migration_target (5e2ed95ed806) + state: executed + $ rbd migration commit migration_target + Commit image migration: 100% complete...done. + +If the `migration_source` image is a parent of one or more clones, the `--force` +option will need to be specified after ensuring all descendent clone images are +not in use. + +Committing the live-migration will remove the cross-links between the source +and target images, and will remove the source image:: + + $ rbd trash list --all + + +Abort Migration +=============== + +If you wish to revert the prepare or execute step, run the `rbd migration abort` +command to revert the migration process:: + + $ rbd migration abort migration_target + Abort image migration: 100% complete...done. + +Aborting the migration will result in the target image being deleted and access +to the original source image being restored:: + + $ rbd ls + migration_source + + +.. _layered images: ../rbd-snapshot/#layering |