summaryrefslogtreecommitdiffstats
path: root/doc/releases/bobtail.rst
diff options
context:
space:
mode:
Diffstat (limited to '')
-rw-r--r--doc/releases/bobtail.rst546
1 files changed, 546 insertions, 0 deletions
diff --git a/doc/releases/bobtail.rst b/doc/releases/bobtail.rst
new file mode 100644
index 000000000..9cca4f19e
--- /dev/null
+++ b/doc/releases/bobtail.rst
@@ -0,0 +1,546 @@
+=======
+Bobtail
+=======
+
+Bobtail is the second stable release of Ceph. It is named after the
+bobtail squid (order Sepiolida), a group of cephalopods closely related to cuttlefish.
+
+v0.56.7 "bobtail"
+=================
+
+This bobtail update fixes a range of radosgw bugs (including an easily
+triggered crash from multi-delete), a possible data corruption issue
+with power failure on XFS, and several OSD problems, including a
+memory "leak" that will affect aged clusters.
+
+Notable changes
+---------------
+
+* ceph-fuse: create finisher flags after fork()
+* debian: fix prerm/postinst hooks; do not restart daemons on upgrade
+* librados: fix async aio completion wakeup (manifests as rbd hang)
+* librados: fix hang when osd becomes full and then not full
+* librados: fix locking for aio completion refcounting
+* librbd python bindings: fix stripe_unit, stripe_count
+* librbd: make image creation default configurable
+* mon: fix validation of mds ids in mon commands
+* osd: avoid excessive disk updates during peering
+* osd: avoid excessive memory usage on scrub
+* osd: avoid heartbeat failure/suicide when scrubbing
+* osd: misc minor bug fixes
+* osd: use fdatasync instead of sync_file_range (may avoid xfs power-loss corruption)
+* rgw: escape prefix correctly when listing objects
+* rgw: fix copy attrs
+* rgw: fix crash on multi delete
+* rgw: fix locking/crash when using ops log socket
+* rgw: fix usage logging
+* rgw: handle deep uri resources
+
+For more detailed information, see :download:`the complete changelog <../changelog/v0.56.7.txt>`.
+
+
+v0.56.6 "bobtail"
+=================
+
+Notable changes
+---------------
+
+* rgw: fix garbage collection
+* rpm: fix package dependencies
+
+For more detailed information, see :download:`the complete changelog <../changelog/v0.56.6.txt>`.
+
+
+v0.56.5 "bobtail"
+=================
+
+Upgrading
+---------
+
+* ceph-disk[-prepare,-activate] behavior has changed in various ways.
+ There should not be any compatibility issues, but chef users should
+ be aware.
+
+Notable changes
+---------------
+
+* mon: fix recording of quorum feature set (important for argonaut -> bobtail -> cuttlefish mon upgrades)
+* osd: minor peering bug fixes
+* osd: fix a few bugs when pools are renamed
+* osd: fix occasionally corrupted pg stats
+* osd: fix behavior when broken v0.56[.0] clients connect
+* rbd: avoid FIEMAP ioctl on import (it is broken on some kernels)
+* librbd: fixes for several request/reply ordering bugs
+* librbd: only set STRIPINGV2 feature on new images when needed
+* librbd: new async flush method to resolve qemu hangs (requires QEMU update as well)
+* librbd: a few fixes to flatten
+* ceph-disk: support for dm-crypt
+* ceph-disk: many backports to allow bobtail deployments with ceph-deploy, chef
+* sysvinit: do not stop starting daemons on first failure
+* udev: fixed rules for redhat-based distros
+* build fixes for raring
+
+For more detailed information, see :download:`the complete changelog <../changelog/v0.56.5.txt>`.
+
+v0.56.4 "bobtail"
+=================
+
+Upgrading
+---------
+
+* There is a fix in the syntax for the output of 'ceph osd tree --format=json'.
+
+* The MDS disk format has changed from prior releases *and* from v0.57. In particular,
+ upgrades to v0.56.4 are safe, but you cannot move from v0.56.4 to v0.57 if you are using
+ the MDS for CephFS; you must upgrade directly to v0.58 (or later) instead.
+
+Notable changes
+---------------
+
+* mon: fix bug in bringup with IPv6
+* reduce default memory utilization by internal logging (all daemons)
+* rgw: fix for bucket removal
+* rgw: reopen logs after log rotation
+* rgw: fix multipat upload listing
+* rgw: don't copy object when copied onto self
+* osd: fix caps parsing for pools with - or _
+* osd: allow pg log trimming when degraded, scrubbing, recoverying (reducing memory consumption)
+* osd: fix potential deadlock when 'journal aio = true'
+* osd: various fixes for collection creation/removal, rename, temp collections
+* osd: various fixes for PG split
+* osd: deep-scrub omap key/value data
+* osd: fix rare bug in journal replay
+* osd: misc fixes for snapshot tracking
+* osd: fix leak in recovery reservations on pool deletion
+* osd: fix bug in connection management
+* osd: fix for op ordering when rebalancing
+* ceph-fuse: report file system size with correct units
+* mds: get and set directory layout policies via virtual xattrs
+* mds: on-disk format revision (see upgrading note above)
+* mkcephfs, init-ceph: close potential security issues with predictable filenames
+
+For more detailed information, see :download:`the complete changelog <../changelog/v0.56.4.txt>`.
+
+v0.56.3 "bobtail"
+=================
+
+This release has several bug fixes surrounding OSD stability. Most
+significantly, an issue with OSDs being unresponsive shortly after
+startup (and occasionally crashing due to an internal heartbeat check)
+is resolved. Please upgrade.
+
+Upgrading
+---------
+
+* A bug was fixed in which the OSDMap epoch for PGs without any IO
+ requests was not recorded. If there are pools in the cluster that
+ are completely idle (for example, the ``data`` and ``metadata``
+ pools normally used by CephFS), and a large number of OSDMap epochs
+ have elapsed since the ``ceph-osd`` daemon was last restarted, those
+ maps will get reprocessed when the daemon restarts. This process
+ can take a while if there are a lot of maps. A workaround is to
+ 'touch' any idle pools with IO prior to restarting the daemons after
+ packages are upgraded::
+
+ rados bench 10 write -t 1 -b 4096 -p {POOLNAME}
+
+ This will typically generate enough IO to touch every PG in the pool
+ without generating significant cluster load, and also cleans up any
+ temporary objects it creates.
+
+Notable changes
+---------------
+
+* osd: flush peering work queue prior to start
+* osd: persist osdmap epoch for idle PGs
+* osd: fix and simplify connection handling for heartbeats
+* osd: avoid crash on invalid admin command
+* mon: fix rare races with monitor elections and commands
+* mon: enforce that OSD reweights be between 0 and 1 (NOTE: not CRUSH weights)
+* mon: approximate client, recovery bandwidth logging
+* radosgw: fixed some XML formatting to conform to Swift API inconsistency
+* radosgw: fix usage accounting bug; add repair tool
+* radosgw: make fallback URI configurable (necessary on some web servers)
+* librbd: fix handling for interrupted 'unprotect' operations
+* mds, ceph-fuse: allow file and directory layouts to be modified via virtual xattrs
+
+For more detailed information, see :download:`the complete changelog <../changelog/v0.56.3.txt>`.
+
+
+v0.56.2 "bobtail"
+=================
+
+This release has a wide range of bug fixes, stability improvements, and some performance improvements. Please upgrade.
+
+Upgrading
+---------
+
+* The meaning of the 'osd scrub min interval' and 'osd scrub max
+ interval' has changed slightly. The min interval used to be
+ meaningless, while the max interval would only trigger a scrub if
+ the load was sufficiently low. Now, the min interval option works
+ the way the old max interval did (it will trigger a scrub after this
+ amount of time if the load is low), while the max interval will
+ force a scrub regardless of load. The default options have been
+ adjusted accordingly. If you have customized these in ceph.conf,
+ please review their values when upgrading.
+
+* CRUSH maps that are generated by default when calling ``ceph-mon
+ --mkfs`` directly now distribute replicas across hosts instead of
+ across OSDs. Any provisioning tools that are being used by Ceph may
+ be affected, although probably for the better, as distributing across
+ hosts is a much more commonly sought behavior. If you use
+ ``mkcephfs`` to create the cluster, the default CRUSH rule is still
+ inferred by the number of hosts and/or racks in the initial ceph.conf.
+
+Notable changes
+---------------
+
+* osd: snapshot trimming fixes
+* osd: scrub snapshot metadata
+* osd: fix osdmap trimming
+* osd: misc peering fixes
+* osd: stop heartbeating with peers if internal threads are stuck/hung
+* osd: PG removal is friendlier to other workloads
+* osd: fix recovery start delay (was causing very slow recovery)
+* osd: fix scheduling of explicitly requested scrubs
+* osd: fix scrub interval config options
+* osd: improve recovery vs client io tuning
+* osd: improve 'slow request' warning detail for better diagnosis
+* osd: default CRUSH map now distributes across hosts, not OSDs
+* osd: fix crash on 32-bit hosts triggered by librbd clients
+* librbd: fix error handling when talking to older OSDs
+* mon: fix a few rare crashes
+* ceph command: ability to easily adjust CRUSH tunables
+* radosgw: object copy does not copy source ACLs
+* rados command: fix omap command usage
+* sysvinit script: set ulimit -n properly on remote hosts
+* msgr: fix narrow race with message queuing
+* fixed compilation on some old distros (e.g., RHEL 5.x)
+
+For more detailed information, see :download:`the complete changelog <../changelog/v0.56.2.txt>`.
+
+
+v0.56.1 "bobtail"
+=================
+
+This release has two critical fixes. Please upgrade.
+
+Upgrading
+---------
+
+* There is a protocol compatibility problem between v0.56 and any
+ other version that is now fixed. If your radosgw or RBD clients are
+ running v0.56, they will need to be upgraded too. If they are
+ running a version prior to v0.56, they can be left as is.
+
+Notable changes
+---------------
+* osd: fix commit sequence for XFS, ext4 (or any other non-btrfs) to prevent data loss on power cycle or kernel panic
+* osd: fix compatibility for CALL operation
+* osd: process old osdmaps prior to joining cluster (fixes slow startup)
+* osd: fix a couple of recovery-related crashes
+* osd: fix large io requests when journal is in (non-default) aio mode
+* log: fix possible deadlock in logging code
+
+For more detailed information, see :download:`the complete changelog <../changelog/v0.56.1.txt>`.
+
+v0.56 "bobtail"
+===============
+
+Bobtail is the second stable release of Ceph, named in honor of the
+`Bobtail Squid`: https://en.wikipedia.org/wiki/Bobtail_squid.
+
+Key features since v0.48 "argonaut"
+-----------------------------------
+
+* Object Storage Daemon (OSD): improved threading, small-io performance, and performance during recovery
+* Object Storage Daemon (OSD): regular "deep" scrubbing of all stored data to detect latent disk errors
+* RADOS Block Device (RBD): support for copy-on-write clones of images.
+* RADOS Block Device (RBD): better client-side caching.
+* RADOS Block Device (RBD): advisory image locking
+* Rados Gateway (RGW): support for efficient usage logging/scraping (for billing purposes)
+* Rados Gateway (RGW): expanded S3 and Swift API coverage (e.g., POST, multi-object delete)
+* Rados Gateway (RGW): improved striping for large objects
+* Rados Gateway (RGW): OpenStack Keystone integration
+* RPM packages for Fedora, RHEL/CentOS, OpenSUSE, and SLES
+* mkcephfs: support for automatically formatting and mounting XFS and ext4 (in addition to btrfs)
+
+Upgrading
+---------
+
+Please refer to the document `Upgrading from Argonaut to Bobtail`_ for details.
+
+.. _Upgrading from Argonaut to Bobtail: ../install/upgrading-ceph/#upgrading-from-argonaut-to-bobtail
+
+* Cephx authentication is now enabled by default (since v0.55).
+ Upgrading a cluster without adjusting the Ceph configuration will
+ likely prevent the system from starting up on its own. We recommend
+ first modifying the configuration to indicate that authentication is
+ disabled, and only then upgrading to the latest version::
+
+ auth client required = none
+ auth service required = none
+ auth cluster required = none
+
+* Ceph daemons can be upgraded one-by-one while the cluster is online
+ and in service.
+
+* The ``ceph-osd`` daemons must be upgraded and restarted *before* any
+ ``radosgw`` daemons are restarted, as they depend on some new
+ ceph-osd functionality. (The ``ceph-mon``, ``ceph-osd``, and
+ ``ceph-mds`` daemons can be upgraded and restarted in any order.)
+
+* Once each individual daemon has been upgraded and restarted, it
+ cannot be downgraded.
+
+* The cluster of ``ceph-mon`` daemons will migrate to a new internal
+ on-wire protocol once all daemons in the quorum have been upgraded.
+ Upgrading only a majority of the nodes (e.g., two out of three) may
+ expose the cluster to a situation where a single additional failure
+ may compromise availability (because the non-upgraded daemon cannot
+ participate in the new protocol). We recommend not waiting for an
+ extended period of time between ``ceph-mon`` upgrades.
+
+* The ops log and usage log for radosgw are now off by default. If
+ you need these logs (e.g., for billing purposes), you must enable
+ them explicitly. For logging of all operations to objects in the
+ ``.log`` pool (see ``radosgw-admin log ...``)::
+
+ rgw enable ops log = true
+
+ For usage logging of aggregated bandwidth usage (see ``radosgw-admin
+ usage ...``)::
+
+ rgw enable usage log = true
+
+* You should not create or use "format 2" RBD images until after all
+ ``ceph-osd`` daemons have been upgraded. Note that "format 1" is
+ still the default. You can use the new ``ceph osd ls`` and
+ ``ceph tell osd.N version`` commands to doublecheck your cluster.
+ ``ceph osd ls`` will give a list of all OSD IDs that are part of the
+ cluster, and you can use that to write a simple shell loop to display
+ all the OSD version strings: ::
+
+ for i in $(ceph osd ls); do
+ ceph tell osd.${i} version
+ done
+
+
+Compatibility changes
+---------------------
+
+* The 'ceph osd create [<uuid>]' command now rejects an argument that
+ is not a UUID. (Previously it would take an optional integer
+ OSD id.) This correct syntax has been 'ceph osd create [<uuid>]'
+ since v0.47, but the older calling convention was being silently
+ ignored.
+
+* The CRUSH map root nodes now have type ``root`` instead of type
+ ``pool``. This avoids confusion with RADOS pools, which are not
+ directly related. Any scripts or tools that use the ``ceph osd
+ crush ...`` commands may need to be adjusted accordingly.
+
+* The ``ceph osd pool create <poolname> <pgnum>`` command now requires
+ the ``pgnum`` argument. Previously this was optional, and would
+ default to 8, which was almost never a good number.
+
+* Degraded mode (when there fewer than the desired number of replicas)
+ is now more configurable on a per-pool basis, with the min_size
+ parameter. By default, with min_size 0, this allows I/O to objects
+ with N - floor(N/2) replicas, where N is the total number of
+ expected copies. Argonaut behavior was equivalent to having min_size
+ = 1, so I/O would always be possible if any completely up to date
+ copy remained. min_size = 1 could result in lower overall
+ availability in certain cases, such as flapping network partitions.
+
+* The sysvinit start/stop script now defaults to adjusting the max
+ open files ulimit to 16384. On most systems the default is 1024, so
+ this is an increase and won't break anything. If some system has a
+ higher initial value, however, this change will lower the limit.
+ The value can be adjusted explicitly by adding an entry to the
+ ``ceph.conf`` file in the appropriate section. For example::
+
+ [global]
+ max open files = 32768
+
+* 'rbd lock list' and 'rbd showmapped' no longer use tabs as
+ separators in their output.
+
+* There is configurable limit on the number of PGs when creating a new
+ pool, to prevent a user from accidentally specifying a ridiculous
+ number for pg_num. It can be adjusted via the 'mon max pool pg num'
+ option on the monitor, and defaults to 65536 (the current max
+ supported by the Linux kernel client).
+
+* The osd capabilities associated with a rados user have changed
+ syntax since 0.48 argonaut. The new format is mostly backwards
+ compatible, but there are two backwards-incompatible changes:
+
+ * specifying a list of pools in one grant, i.e.
+ 'allow r pool=foo,bar' is now done in separate grants, i.e.
+ 'allow r pool=foo, allow r pool=bar'.
+
+ * restricting pool access by pool owner ('allow r uid=foo') is
+ removed. This feature was not very useful and unused in practice.
+
+ The new format is documented in the ceph-authtool man page.
+
+* 'rbd cp' and 'rbd rename' use rbd as the default destination pool,
+ regardless of what pool the source image is in. Previously they
+ would default to the same pool as the source image.
+
+* 'rbd export' no longer prints a message for each object written. It
+ just reports percent complete like other long-lasting operations.
+
+* 'ceph osd tree' now uses 4 decimal places for weight so output is
+ nicer for humans
+
+* Several monitor operations are now idempotent:
+
+ * ceph osd pool create
+ * ceph osd pool delete
+ * ceph osd pool mksnap
+ * ceph osd rm
+ * ceph pg <pgid> revert
+
+Notable changes
+---------------
+
+* auth: enable cephx by default
+* auth: expanded authentication settings for greater flexibility
+* auth: sign messages when using cephx
+* build fixes for Fedora 18, CentOS/RHEL 6
+* ceph: new 'osd ls' and 'osd tell <osd.N> version' commands
+* ceph-debugpack: misc improvements
+* ceph-disk-prepare: creates and labels GPT partitions
+* ceph-disk-prepare: support for external journals, default mount/mkfs options, etc.
+* ceph-fuse/libcephfs: many misc fixes, admin socket debugging
+* ceph-fuse: fix handling for .. in root directory
+* ceph-fuse: many fixes (including memory leaks, hangs)
+* ceph-fuse: mount helper (mount.fuse.ceph) for use with /etc/fstab
+* ceph.spec: misc packaging fixes
+* common: thread pool sizes can now be adjusted at runtime
+* config: $pid is now available as a metavariable
+* crush: default root of tree type is now 'root' instead of 'pool' (to avoid confusiong wrt rados pools)
+* crush: fixed retry behavior with chooseleaf via tunable
+* crush: tunables documented; feature bit now present and enforced
+* libcephfs: java wrapper
+* librados: several bug fixes (rare races, locking errors)
+* librados: some locking fixes
+* librados: watch/notify fixes, misc memory leaks
+* librbd: a few fixes to 'discard' support
+* librbd: fine-grained striping feature
+* librbd: fixed memory leaks
+* librbd: fully functional and documented image cloning
+* librbd: image (advisory) locking
+* librbd: improved caching (of object non-existence)
+* librbd: 'flatten' command to sever clone parent relationship
+* librbd: 'protect'/'unprotect' commands to prevent clone parent from being deleted
+* librbd: clip requests past end-of-image.
+* librbd: fixes an issue with some windows guests running in qemu (remove floating point usage)
+* log: fix in-memory buffering behavior (to only write log messages on crash)
+* mds: fix ino release on abort session close, relative getattr path, mds shutdown, other misc items
+* mds: misc fixes
+* mkcephfs: fix for default keyring, osd data/journal locations
+* mkcephfs: support for formatting xfs, ext4 (as well as btrfs)
+* init: support for automatically mounting xfs and ext4 osd data directories
+* mon, radosgw, ceph-fuse: fixed memory leaks
+* mon: improved ENOSPC, fs error checking
+* mon: less-destructive ceph-mon --mkfs behavior
+* mon: misc fixes
+* mon: more informative info about stuck PGs in 'health detail'
+* mon: information about recovery and backfill in 'pg <pgid> query'
+* mon: new 'osd crush create-or-move ...' command
+* mon: new 'osd crush move ...' command lets you rearrange your CRUSH hierarchy
+* mon: optionally dump 'osd tree' in json
+* mon: configurable cap on maximum osd number (mon max osd)
+* mon: many bug fixes (various races causing ceph-mon crashes)
+* mon: new on-disk metadata to facilitate future mon changes (post-bobtail)
+* mon: election bug fixes
+* mon: throttle client messages (limit memory consumption)
+* mon: throttle osd flapping based on osd history (limits osdmap ΄thrashing' on overloaded or unhappy clusters)
+* mon: 'report' command for dumping detailed cluster status (e.g., for use when reporting bugs)
+* mon: osdmap flags like noup, noin now cause a health warning
+* msgr: improved failure handling code
+* msgr: many bug fixes
+* osd, mon: honor new 'nobackfill' and 'norecover' osdmap flags
+* osd, mon: use feature bits to lock out clients lacking CRUSH tunables when they are in use
+* osd: backfill reservation framework (to avoid flooding new osds with backfill data)
+* osd: backfill target reservations (improve performance during recovery)
+* osd: better tracking of recent slow operations
+* osd: capability grammar improvements, bug fixes
+* osd: client vs recovery io prioritization
+* osd: crush performance improvements
+* osd: default journal size to 5 GB
+* osd: experimental support for PG "splitting" (pg_num adjustment for existing pools)
+* osd: fix memory leak on certain error paths
+* osd: fixed detection of EIO errors from fs on read
+* osd: major refactor of PG peering and threading
+* osd: many bug fixes
+* osd: more/better dump info about in-progress operations
+* osd: new caps structure (see compatibility notes)
+* osd: new 'deep scrub' will compare object content across replicas (once per week by default)
+* osd: new 'lock' rados class for generic object locking
+* osd: optional 'min' pg size
+* osd: recovery reservations
+* osd: scrub efficiency improvement
+* osd: several out of order reply bug fixes
+* osd: several rare peering cases fixed
+* osd: some performance improvements related to request queuing
+* osd: use entire device if journal is a block device
+* osd: use syncfs(2) when kernel supports it, even if glibc does not
+* osd: various fixes for out-of-order op replies
+* rados: ability to copy, rename pools
+* rados: bench command now cleans up after itself
+* rados: 'cppool' command to copy rados pools
+* rados: 'rm' now accepts a list of objects to be removed
+* radosgw: POST support
+* radosgw: REST API for managing usage stats
+* radosgw: fix bug in bucket stat updates
+* radosgw: fix copy-object vs attributes
+* radosgw: fix range header for large objects, ETag quoting, GMT dates, other compatibility fixes
+* radosgw: improved garbage collection framework
+* radosgw: many small fixes, cleanups
+* radosgw: openstack keystone integration
+* radosgw: stripe large (non-multipart) objects
+* radosgw: support for multi-object deletes
+* radosgw: support for swift manifest objects
+* radosgw: vanity bucket dns names
+* radosgw: various API compatibility fixes
+* rbd: import from stdin, export to stdout
+* rbd: new 'ls -l' option to view images with metadata
+* rbd: use generic id and keyring options for 'rbd map'
+* rbd: don't issue usage on errors
+* udev: fix symlink creation for rbd images containing partitions
+* upstart: job files for all daemon types (not enabled by default)
+* wireshark: ceph protocol dissector patch updated
+
+
+v0.54
+=====
+
+Upgrading
+---------
+
+* The osd capabilities associated with a rados user have changed
+ syntax since 0.48 argonaut. The new format is mostly backwards
+ compatible, but there are two backwards-incompatible changes:
+
+ * specifying a list of pools in one grant, i.e.
+ 'allow r pool=foo,bar' is now done in separate grants, i.e.
+ 'allow r pool=foo, allow r pool=bar'.
+
+ * restricting pool access by pool owner ('allow r uid=foo') is
+ removed. This feature was not very useful and unused in practice.
+
+ The new format is documented in the ceph-authtool man page.
+
+* Bug fixes to the new osd capability format parsing properly validate
+ the allowed operations. If an existing rados user gets permissions
+ errors after upgrading, its capabilities were probably
+ misconfigured. See the ceph-authtool man page for details on osd
+ capabilities.
+
+* 'rbd lock list' and 'rbd showmapped' no longer use tabs as
+ separators in their output.