commit 2dab17a455c09584f2a85e6b10888337d1ec8949 Author: Jenkins Build Slave User Date: Mon Feb 19 23:14:46 2018 +0000 12.2.3 commit c0429254cec6fdfa1089f3dc205a7a6199d2e40a Author: Kefu Chai Date: Tue Feb 20 06:40:14 2018 +0800 cmake: disable DOWNLOAD_NO_PROGRESS if cmake ver is lower than 3.1 see https://cmake.org/cmake/help/v3.1/module/ExternalProject.html and https://cmake.org/cmake/help/v3.0/module/ExternalProject.html. the former has DOWNLOAD_NO_PROGRESS, while the latter does not. Signed-off-by: Kefu Chai (cherry picked from commit d051bf0441c933e27fb0f54bc6ed999d1e8eca89) commit 74c89b975f786ca2bf6de7310f191e92faeaca6c Merge: 76df814ac4 7e33d67f1b Author: Alfredo Deza Date: Fri Feb 16 08:06:00 2018 -0500 Merge pull request #20452 from ceph/backport-wip-rm22785 luminous ceph-volume consume mount/format options from ceph.conf Reviewed-by: Andrew Schoen commit 7e33d67f1b143eff9c63af5d44d2d2dd480ed2f3 Author: Alfredo Deza Date: Fri Feb 16 06:51:12 2018 -0500 ceph-volume tests remove unused import Signed-off-by: Alfredo Deza (cherry picked from commit ad49283b2ee1e4a1a8a54d0c71039a2afc778b29) commit 76df814ac485df9f72e58081f8d6bb17151cc18b Merge: 3e86319b8f cdc27208aa Author: Abhishek L Date: Fri Feb 16 11:15:57 2018 +0100 Merge pull request #20451 from ceph/wip-yuri-PR20053-luminous qa/tests: Applied PR 20053 to stress-split tests Reviewed-By: Jason Dillaman commit cdc27208aa170061ef798b9457d6e85294e9238b Author: Yuri Weinstein Date: Thu Feb 15 10:26:46 2018 -0800 qa/tests: Applied PR 20053 to stress-split tests Signed-off-by: Yuri Weinstein commit 18a0ee570ffb456053e1712881ee61f8b2072f87 Author: Alfredo Deza Date: Mon Feb 12 16:47:36 2018 -0500 ceph-volume util.prepare extend flags when found Signed-off-by: Alfredo Deza (cherry picked from commit 287c952bc8c3422c6eed72a1e0b22612d0728314) commit cb5da5fffe625aa6b930bdf4a77869716b77caff Author: Alfredo Deza Date: Mon Feb 12 16:47:17 2018 -0500 ceph-volume util.constants update mount flags to include rw, normalize on list Signed-off-by: Alfredo Deza (cherry picked from commit 457a5173d2e08192273c22f1abe52a64773188b4) commit 33f7da3f165be7bbc4a56613078d95d6fa6e79cb Author: Alfredo Deza Date: Mon Feb 12 16:46:47 2018 -0500 ceph-volume configuration normalize underscore vs. whitespace named options Signed-off-by: Alfredo Deza (cherry picked from commit b80e3bf1175a65181d1f2ed38b6a416c6b730d82) commit 93e41ef6434daea37df3322e87e0c6e367de1741 Author: Alfredo Deza Date: Mon Feb 12 16:46:06 2018 -0500 ceph-volume tests check for mount/format usage from ceph.conf Signed-off-by: Alfredo Deza (cherry picked from commit 676c93fcf3041de07e8125d36d8dc7ad474db9b1) commit c586a24e91ca627449452103d562b6be1fbfcb4d Author: Alfredo Deza Date: Mon Feb 12 16:44:45 2018 -0500 ceph-volume tests add ceph.conf stub for custom conf files Signed-off-by: Alfredo Deza (cherry picked from commit a8eecc64716564479308fafba5159e271c42feac) commit 3e86319b8fa5f85a078441b88935f812c8039580 Merge: dbe215d69d 599869d7f2 Author: Andrew Schoen Date: Wed Feb 14 16:59:11 2018 +0100 Merge pull request #20429 from ceph/backport-wip-rm22988 luminous ceph-volume use realpath when checking mounts Reviewed-by: Andrew Schoen commit 599869d7f2df5d207ba9afa8915ce4980ff65137 Author: Alfredo Deza Date: Tue Feb 13 13:22:41 2018 -0500 ceph-volume tests verify realpath usage in mounts Signed-off-by: Alfredo Deza (cherry picked from commit 16eabbc3abb7d03a0808963a9aae6b2ffb3a875b) commit 565c044d556eaf741d79e7abbf5f472ee4366b44 Author: Alfredo Deza Date: Tue Feb 13 12:59:48 2018 -0500 ceph-volume util.system optionally use realpath on devices To check a mounted device it is needed to verify on a combination of realpath and plain devices against realpath and plain paths. In LVM, two different paths might refer to the same devices Signed-off-by: Alfredo Deza (cherry picked from commit 53e691e1d5122ff533e69ae08c73e18c1ed74765) commit dbe215d69d8ccf0956f197cef88f50f6518bbfa0 Merge: 2fc8d6c4ad dd94ac120e Author: Andrew Schoen Date: Wed Feb 14 14:13:59 2018 +0100 Merge pull request #20438 from ceph/backport-wip-rm22876 luminous: ceph-volume: lvm zap will unmount osd paths used by zapped devices Reviewed-by: Andrew Schoen commit dd94ac120ed337f0080dc776f3234eeaade46b54 Author: Andrew Schoen Date: Mon Feb 12 15:23:08 2018 -0600 ceph-volume: use terminal.MultiLogger in `lvm zap` Signed-off-by: Andrew Schoen (cherry picked from commit 3f2e7074ee42ff8248afaed2aa9bfd919096fc38) commit ec84f39a0573887e005fdb0bfe7d4e5e9457c51b Author: Andrew Schoen Date: Wed Feb 7 09:06:45 2018 -0600 ceph-volume: use api helper methods for pvs and lvs in lvm zap This is just a cleaner way of doing this instead of using PVolumes and Volumes directly. Signed-off-by: Andrew Schoen (cherry picked from commit c8a6da4e1bb645ca5ee8cb8f5d861584fe70210d) commit 19e02e6c433f718ce638624a803105525168cb52 Author: Andrew Schoen Date: Wed Feb 7 08:35:47 2018 -0600 ceph-volume: fix documentation typos for lvm zap Signed-off-by: Andrew Schoen (cherry picked from commit 5552cefc4ce6d6d819f43671cec83ca23b119665) commit 1de5592d4a13a0c0475f723fc02fb8e1c70b482e Author: Andrew Schoen Date: Thu Feb 1 14:52:46 2018 -0600 ceph-volume: when zapping unmount osd directories If you zap an lv, device or partition and it's currently mounted as a ceph osd directory then it will be unmounted so the zap can complete. Signed-off-by: Andrew Schoen (cherry picked from commit d4639ac116407f71ba6bff06a3202c1490d57dee) commit 13f42da7ae449dde84a94f93ac22f8bf0a8c7f79 Author: Andrew Schoen Date: Thu Feb 1 14:29:30 2018 -0600 ceph-volume: adds a util.system.unmount function This function will unmount the given path Signed-off-by: Andrew Schoen (cherry picked from commit 6909baef542f3cd9b7d97113a6152280e72013f6) commit 2fc8d6c4adfe32586b9d177ed6ab7b46c6b2b110 Merge: 22c6b1d3d6 24f423fa0b Author: Casey Bodley Date: Tue Feb 13 16:43:14 2018 -0500 Merge pull request #20407 from cbodley/wip-qa-multisite-trim-luminous luminous: qa/rgw: disable log trim in multisite suite Reviewed-by: Yuri Weinstein commit 24f423fa0b937fc8df8f7c48fdc1d83510d0a44f Author: Casey Bodley Date: Mon Dec 11 16:14:05 2017 -0500 qa/rgw: disable log trim in multisite suite the multisite tests run manual trim operations with radosgw-admin, which can race with internal log trimming to produce tests failures Signed-off-by: Casey Bodley (cherry picked from commit f8909bb6a6320822da7a4c4dab242f84732ebf5d) commit 22c6b1d3d6bbc7ee466fddd3af13bcdb7f0c9a55 Merge: fda9e4a3e2 01f79c086a Author: Andrew Schoen Date: Thu Feb 8 18:56:12 2018 +0100 Merge pull request #20368 from ceph/backport-20367 luminous: ceph-volume: adds custom cluster name support to simple Reviewed-by: Andrew Schoen commit fda9e4a3e2ab4dd91430d9197792ed15e8ad256d Merge: faefe0b68f 29c89c4964 Author: Abhishek L Date: Thu Feb 8 18:08:49 2018 +0100 Merge pull request #20357 from pdvian/wip-22938-luminous luminous: rgw: unlink deleted bucket from bucket's owner Reviewed-By: Orit Wasserman Reviewed-By: Abhishek Lekshmanan commit 01f79c086a2d6e1fd10f3d6283c6f12ef03991a2 Author: Andrew Schoen Date: Thu Feb 8 07:09:57 2018 -0600 ceph-volume: use a custom cluster name in simple functional tests Signed-off-by: Andrew Schoen (cherry picked from commit 7f1dc6b3ab885253d28c223bda7e6e1232a84609) commit 8d50005d3efddde777916df214ace36178f4741f Author: Andrew Schoen Date: Wed Feb 7 14:42:24 2018 -0600 ceph-volume: set conf.cluster name during simple activate Signed-off-by: Andrew Schoen (cherry picked from commit b1cee74fa2225fe10a024c7291bd23aadcc23441) commit faefe0b68fa748e8cb50ffb97b34d39ace65f35c Merge: 13f400f801 bf3ee8b22b Author: Orit Wasserman Date: Thu Feb 8 12:43:44 2018 +0200 Merge pull request #20338 from cbodley/wip-22930 luminous: rgw: fix for empty query string in beast frontend Reviewed-by: Orit Wasserman commit 29c89c496424dbaf66ab8771d1e9c9578eb092a2 Author: Casey Bodley Date: Thu Jan 18 14:53:35 2018 -0500 rgw: unlink deleted bucket from bucket's owner if a bucket is deleted by an admin/system user instead of its owner, the unlink would fail and the deleted bucket remained visible to the original owner Fixes: http://tracker.ceph.com/issues/22248 Signed-off-by: Casey Bodley (cherry picked from commit 4701e85a3ea72852a61a106724fa22497f3d5789) commit 13f400f80182c4fe0ebe8f9be19fa63898003f0b Merge: 609f2a88e0 b8b3eee73d Author: Yuri Weinstein Date: Wed Feb 7 14:49:36 2018 -0800 Merge pull request #20204 from tchaikov/wip-pr-18614-luminous luminous: osd,mgr: report pending creating pgs to mgr Reviewed-by: Josh Durgin Reviewed-by: Sage Weil commit 609f2a88e0e8b0cb40ac7853d474a175f4c9699e Merge: 2cef7159fe d93eef3127 Author: Yuri Weinstein Date: Wed Feb 7 14:45:33 2018 -0800 Merge pull request #20249 from pdvian/wip-22864-luminous luminous: mds: fix scrub crash Reviewed-by: Patrick Donnelly commit 2cef7159fe89a428256910d90dbfb0d42924d0cf Merge: 8bca8fb027 7756a44529 Author: Yuri Weinstein Date: Wed Feb 7 14:45:07 2018 -0800 Merge pull request #20256 from pdvian/wip-22860-luminous luminous: osdc: "FAILED assert(bh->last_write_tid > tid)" in powercycle-wip-yuri-master-1.19.18-distro-basic-smithi Reviewed-by: Patrick Donnelly commit 8bca8fb0274c5f3690262400a92eb23bb87a4012 Merge: 16d85f9673 042144a281 Author: Yuri Weinstein Date: Wed Feb 7 14:44:39 2018 -0800 Merge pull request #20299 from pdvian/wip-22859-luminous luminous: mds: set higher priority for some perf counters Reviewed-by: Amit Kumar Reviewed-by: Patrick Donnelly commit 16d85f9673931596017aeb3eb23132b5f3ef3500 Merge: 43a4e40916 5282a60c8f Author: Yuri Weinstein Date: Wed Feb 7 14:44:03 2018 -0800 Merge pull request #20300 from pdvian/wip-22867-luminous luminous: MDS : Avoid the assert failure when the inode for the cap_export from other… Reviewed-by: Patrick Donnelly Reviewed-by: Amit Kumar commit 43a4e40916159f347bca12ff02d2b64947d2a4e4 Merge: 1133c9bba1 c63fc964ce Author: Yuri Weinstein Date: Wed Feb 7 14:41:45 2018 -0800 Merge pull request #20339 from batrick/i22242 luminous: mds: limit size of subtree migration Reviewed-by: Yuri Weinstein commit 1133c9bba1750c8f861293f44ed9e99f679fd2cb Merge: a8cdfea4b3 87991643b3 Author: Yuri Weinstein Date: Wed Feb 7 14:41:14 2018 -0800 Merge pull request #20340 from batrick/i22240 luminous: mds: prevent filelock from being stuck at XSYN state Reviewed-by: Yuri Weinstein commit a8cdfea4b34e46a59fdf023cbeb57b9dfc0db2ee Merge: e2e6d30f3f 655d266944 Author: Yuri Weinstein Date: Wed Feb 7 14:40:33 2018 -0800 Merge pull request #20341 from batrick/i22089 luminous: mds: don't report repaired backtraces in damagetable, write back after repair, clean up scrub log Reviewed-by: Yuri Weinstein commit e2e6d30f3f00af38c49aa9845eaac940680f37cd Merge: 33ebb80c91 6caae86b7c Author: Yuri Weinstein Date: Wed Feb 7 14:38:58 2018 -0800 Merge pull request #20337 from liewegas/wip-bluestore-aio-read luminous: os/bluestore: propagate read EIO errors up the stack Reviewed-by: Josh Durgin commit 33ebb80c915c66dfc4d64b2ce85f6fc690f4326a Merge: b99521accd fc5807003b Author: Andrew Schoen Date: Wed Feb 7 19:02:31 2018 +0100 Merge pull request #20350 from ceph/backport-20264 luminous: ceph-volume dmcrypt support for simple Reviewed-by: Andrew Schoen commit fc5807003b01e6d009c392e0577765582db53d64 Author: Andrew Schoen Date: Tue Feb 6 11:36:03 2018 -0600 ceph-volume: do not test custom cluster names with simple Signed-off-by: Andrew Schoen (cherry picked from commit 63046e0d7bd05a42477dd1fbd4af11f81046b08d) commit c1eb7b9915a61d93a5b09ff588a5906e35a2b208 Author: Andrew Schoen Date: Tue Feb 6 10:46:44 2018 -0600 ceph-volume: sleep 2 minutes after reboot in simple tests This will allow for some time for OSD services to start back up after reboot. If not we see race conditions where tests fail because they were run before the OSD finished startup. Signed-off-by: Andrew Schoen (cherry picked from commit 85b319a2c577c1fa86c6ccbbac61f015dd24ad21) commit a7ecc1c91a9492337736c9674b4242317a4e526a Author: Andrew Schoen Date: Mon Feb 5 14:18:28 2018 -0600 ceph-volume: adds the simple dmcrypt_plain and dmcrypt_luks tests Signed-off-by: Andrew Schoen (cherry picked from commit 3052010a80a1783b6ebd629a603638ab5806f8fd) commit aab4961b5e597614480d23efcc88008cb0d600c4 Author: Andrew Schoen Date: Mon Feb 5 13:53:13 2018 -0600 ceph-volume: fix ceph-volume simple scan help menu test Signed-off-by: Andrew Schoen (cherry picked from commit 2ee70bf2d1958c0655ddb5b7cc4d2ed3f4bc58b8) commit c1035af7a6a669c5f88ef7f30942a2b9ed347c1c Author: Alfredo Deza Date: Fri Feb 2 09:11:09 2018 -0500 ceph-volume tests.functional add simple xenial filestore dmcrypt plain support Signed-off-by: Alfredo Deza (cherry picked from commit 00b14177bdbe6d66424001b1c35ca8fa7b46977f) commit 55def6eb6c463c08cd1c24839b6891395425372a Author: Alfredo Deza Date: Fri Feb 2 09:10:56 2018 -0500 ceph-volume tests.functional add simple xenial filestore dmcrypt luks support Signed-off-by: Alfredo Deza (cherry picked from commit 0b2f869ccbb30c27bd19f2a05bb893957912142e) commit 591796d073d6032a39a7e7b70cd56965fc8320f0 Author: Alfredo Deza Date: Fri Feb 2 09:10:36 2018 -0500 ceph-volume tests.functional add simple xenial bluestore dmcrypt plain support Signed-off-by: Alfredo Deza (cherry picked from commit 0174f9e1646f4b51c712506f629b48c329f20ea1) commit 31665eac381cf78cd4fdc8993b75dc291121a095 Author: Alfredo Deza Date: Fri Feb 2 09:10:12 2018 -0500 ceph-volume tests.functional add simple centos7 filestore dmcrypt plain support Signed-off-by: Alfredo Deza (cherry picked from commit 4c26162002709ccdd1b48c05126befdab7f269da) commit 4de4d69bf0e260c38177b32ba4f199dd13b9983f Author: Alfredo Deza Date: Fri Feb 2 09:09:57 2018 -0500 ceph-volume tests.functional add simple centos7 filestore dmcrypt luks support Signed-off-by: Alfredo Deza (cherry picked from commit a911d7c6a99e2ea235400614568d9cd362470cac) commit eb97511c3fb9a33df50bf35ace515a068211df7c Author: Alfredo Deza Date: Fri Feb 2 09:09:41 2018 -0500 ceph-volume tests.functional add simple centos7 bluestore dmcrypt plain support Signed-off-by: Alfredo Deza (cherry picked from commit 8e0213ab053e80038bdae8d1b06d448fd397e3bc) commit badd73df5d30f034c5ceee37fdb479bba17fffa6 Author: Alfredo Deza Date: Fri Feb 2 09:09:21 2018 -0500 ceph-volume tests.functional add simple centos7 bluestore dmcrypt luks support Signed-off-by: Alfredo Deza (cherry picked from commit 76151c461bfe1e5ae4dc277c9d9357efb81e8c92) commit 5d6aa430a51a481cae400b8fa9d8ee3fbc5fa392 Author: Alfredo Deza Date: Fri Feb 2 09:08:23 2018 -0500 ceph-volume tests.functional add simple bluestore dmcrypt luks support Signed-off-by: Alfredo Deza (cherry picked from commit d6e24022e8dfe845823fa0e87f3fba028c4b25b5) commit 8fcd52e88a54959dde41333ed440b88d78a021a3 Author: Alfredo Deza Date: Fri Feb 2 08:36:45 2018 -0500 doc/ceph-volume lvm prepare fully supports encryption now Signed-off-by: Alfredo Deza (cherry picked from commit 0b2075759e8788b2c4ae46f94fdee953cc82b21e) commit 476eac136a7aee3485b91b81108bc1d05d0d6698 Author: Alfredo Deza Date: Fri Feb 2 08:36:02 2018 -0500 doc/ceph-volume scan update for encryption support Signed-off-by: Alfredo Deza (cherry picked from commit e0d3bb82255a937cd937d33f966979fb541d1443) commit 141279aeff3892f1778fddb373d2e2180e402950 Author: Alfredo Deza Date: Fri Feb 2 08:28:40 2018 -0500 doc/ceph-volume remove notice that dmcrypt is not supported Signed-off-by: Alfredo Deza (cherry picked from commit a7d11ca27df1d22c8109cad5bf7be8fd23395c71) commit 2f7d03418783b33eaf0cab1fd22ff95f94861d3e Author: Alfredo Deza Date: Fri Feb 2 08:26:58 2018 -0500 doc/man/ceph-volume add simple documentation Signed-off-by: Alfredo Deza (cherry picked from commit 9524021aa15280d98f2b04d1fd66552733adf8a2) commit 1adafb88b6e458527759a13afe2362aa79c4adaa Author: Alfredo Deza Date: Fri Feb 2 08:15:36 2018 -0500 ceph-volume simple.scan update help menu to indicate device support Signed-off-by: Alfredo Deza (cherry picked from commit 09f35cb249674eda044b8fceea6f402ffac7731d) commit d015b1933920a855a6a2cafdda863162b804ecbe Author: Alfredo Deza Date: Fri Feb 2 08:13:12 2018 -0500 ceph-volume tests for keyring parsing Signed-off-by: Alfredo Deza (cherry picked from commit b90c1a86843d8f0c8c7fb68fe059bdd72f3730fd) commit 77d40d0713385dcb54b8aa1d0d5a2e1f8386609c Author: Alfredo Deza Date: Fri Feb 2 08:12:51 2018 -0500 ceph-volume tests for validate_devices Signed-off-by: Alfredo Deza (cherry picked from commit bece7af0523b04d39c26011990cb1fe323aecdd8) commit 70b5a325ffe93cb55eaf0f6debfc4f5dd305c24b Author: Alfredo Deza Date: Fri Feb 2 08:12:05 2018 -0500 ceph-volume simple.activate b64decode keys for activation as well Signed-off-by: Alfredo Deza (cherry picked from commit 636ebc127797c8020cfb7aff09955b72beac8bdf) commit baf12acff49e0e2129e0a7b63d10a497929f4992 Author: Alfredo Deza Date: Thu Feb 1 16:17:19 2018 -0500 ceph-volume tests add validation for lsblk parsers Signed-off-by: Alfredo Deza (cherry picked from commit 03102e4417194e16deb8b738fcdaf87e699e6073) commit 44cbcdee151adc302bd77ca2377032ed1bdb1bfd Author: Alfredo Deza Date: Thu Feb 1 15:51:41 2018 -0500 ceph-volume tests validate parsing of cryptsetup Signed-off-by: Alfredo Deza (cherry picked from commit cce6a12f4a0d8d9a3d7dcb509b42ed2536d9257f) commit 941fbbf29ba893e0fe7657669e969c7d784e08d3 Author: Alfredo Deza Date: Thu Feb 1 15:51:12 2018 -0500 ceph-volume tests add a stub for process.call Signed-off-by: Alfredo Deza (cherry picked from commit b80b61b90f8539b363c03c96e63f31c38c707fc8) commit 2ce659749487f60c4012207aaae6665e889f52dc Author: Alfredo Deza Date: Thu Feb 1 15:35:11 2018 -0500 ceph-volume util.encryption add notes about extra b64decode call for ceph-disk Signed-off-by: Alfredo Deza (cherry picked from commit c2367ef4a4db1e9c62e33dc211474f652d6bc6af) commit 36d81bdf2ab24b52db2808ca8cf91ebd864eddb1 Author: Alfredo Deza Date: Thu Feb 1 15:34:09 2018 -0500 ceph-volume util.encryption parse legacy encrypted with dirs too Signed-off-by: Alfredo Deza (cherry picked from commit 32cb810f8ab7e4a710bfb2590a065dea41369fce) commit de3013287fe9c6d879cc4fbca6c06437e5989e21 Author: Alfredo Deza Date: Thu Feb 1 15:32:54 2018 -0500 ceph-volume terminal create a logger to get terminal+log messages in one call Signed-off-by: Alfredo Deza (cherry picked from commit f6dd0ff15f0aeda9f660e923153f1d3dae68e583) commit c25ba9528e66ab7522d2452fc14c935924767839 Author: Alfredo Deza Date: Thu Feb 1 15:32:16 2018 -0500 ceph-volume simple.activate support dmcrypted devices for both plain and luks Signed-off-by: Alfredo Deza (cherry picked from commit 5ece73b533ee4b577c0ce61e3685a880e1de4851) commit 8a19b31c162ae08913136895144303d8999600dd Author: Alfredo Deza Date: Thu Feb 1 15:31:28 2018 -0500 ceph-volume simple.scan parse the keyring out of the keyring file Signed-off-by: Alfredo Deza (cherry picked from commit a1124362816cbaaad7255b7ffc3698b68601ffe5) Conflicts: src/ceph-volume/ceph_volume/devices/simple/scan.py commit 1d94a5f8e2d9d5e314463cbbb7cf7fc704d3ffc2 Author: Alfredo Deza Date: Tue Jan 30 16:23:06 2018 -0500 ceph-volume simple.scan support dmcrypt OSDs when scanning Signed-off-by: Alfredo Deza (cherry picked from commit 6c87d66d76ee1fc70f8dc6211d89dbd489c6ed3a) commit c3aa6e6b90a73d4754ae28270b650ca9a79809eb Author: Alfredo Deza Date: Tue Jan 30 09:35:53 2018 -0500 ceph-volume util.encryption add a utilty to open plain encrypted devices Signed-off-by: Alfredo Deza (cherry picked from commit e5759e4fc9485e92a2faec277c8a920295751844) commit b2bd65505df108420ee79c93d8da6ffdfa7f7675 Author: Alfredo Deza Date: Tue Jan 30 09:34:53 2018 -0500 ceph-volume util.system tmp mounts can now remove a dmcrypt mapper Signed-off-by: Alfredo Deza (cherry picked from commit 60e8699650a491013d51239b9dc58160946d3bf9) commit 520200ceea0070a1a14976d06befc72217ed438f Author: Alfredo Deza Date: Tue Jan 30 09:22:19 2018 -0500 ceph-volume util.system do not 'translate' using realpath Using realpath always means that device mapper paths, like for dmcrypt or LVM will get mangled and will return something like `/dev/dm-1` which is not useful for anything in ceph-volume Signed-off-by: Alfredo Deza (cherry picked from commit 469d01dc1aee8d6528944ef0acec58df868a9da7) commit abf21f28612c03ce82465b1bb816d477b07da702 Author: Alfredo Deza Date: Tue Jan 30 09:19:06 2018 -0500 ceph-volume util.disk add a PART_ENTRY_TYPE detection utility Signed-off-by: Alfredo Deza (cherry picked from commit cdb812438d6064089bac964cf38817d877afca38) commit c177b491758db2e7d81de100a9e52821f38b748a Author: Alfredo Deza Date: Fri Jan 26 10:44:14 2018 -0500 ceph-volume util.encryption add helpers for legacy devices Signed-off-by: Alfredo Deza (cherry picked from commit d59b087e50e4e2076c59f6316cc89eb2a070aa52) commit cc9840140e7a7fe385fcd2cbaeef7351cfd79ba8 Author: Alfredo Deza Date: Fri Jan 26 10:25:31 2018 -0500 ceph-volume util.constants add ceph-disk partition labels Signed-off-by: Alfredo Deza (cherry picked from commit 548a74bd239abe6b1b7a5a27fb2779800cfa7ac1) commit 69d142adb8d70f1b5c9f55d72395313fd1e80048 Author: Alfredo Deza Date: Thu Jan 25 11:04:15 2018 -0500 ceph-volume util.disk support PKNAME and absolute paths in lsblk Signed-off-by: Alfredo Deza (cherry picked from commit b90044c2ffff38d0dc36d598d237fcb066efe400) commit b99521accd04a9761fea8d9a927442ae51bf18a5 Merge: 62337485fa 40a16ac6cc Author: Alfredo Deza Date: Wed Feb 7 08:19:29 2018 -0500 Merge pull request #20323 from ceph/backport-20203 luminous: ceph-volume: fix usage of the --osd-id flag Reviewed-by: Alfredo Deza commit 62337485faf13bd1caa02adec877fd9a5de1bd78 Merge: 3d84e35378 b47837100f Author: Alfredo Deza Date: Wed Feb 7 08:18:32 2018 -0500 Merge pull request #20322 from ceph/backport-20059 luminous: doc/ceph-volume OSD use the fsid file, not the osd_fsid Reviewed-by: Alfredo Deza commit 3d84e3537826d1dcffe8a82195b93a8488d71b8b Merge: 1c847a1fc8 a26b4ae765 Author: Yuri Weinstein Date: Tue Feb 6 14:21:16 2018 -0800 Merge pull request #20162 from tchaikov/wip-pr-20130-luminous mgr: balancer: fixed mistype "AttributeError: 'Logger' object has no ... Reviewed-by: Josh Durgin Reviewed-by: Sage Weil commit 1c847a1fc88bb23126c77ac09a38dde29a5500e4 Merge: 6b5ae18b13 03569abb40 Author: Yuri Weinstein Date: Tue Feb 6 14:20:18 2018 -0800 Merge pull request #20325 from jcsp/wip-22851-luminous luminous: pybind/mgr/dashboard: fix duplicated slash in html href Reviewed-by: Josh Durgin commit 6b5ae18b133a260ef50f39652b64fd8ea208722a Merge: ac0e193f40 91673b1edf Author: Yuri Weinstein Date: Tue Feb 6 14:19:14 2018 -0800 Merge pull request #20326 from theanalyst/wip-22892-luminous luminous: _read_bdev_label unable to decode label at offset Reviewed-by: Josh Durgin commit 655d266944374eea2a846f182cc00193ef4cf4b8 Author: Yan, Zheng Date: Sun Nov 19 20:25:22 2017 +0800 mds: properly execute scrub finish context Bug was introduced by commit 7e52729699 (mds: flush after scrub repairs) Fixes: http://tracker.ceph.com/issues/22058 Signed-off-by: "Yan, Zheng" (cherry picked from commit ff71fbdcbb87ffc9baa02319ff0f336c173c5616) commit 798cc37c7a2acb4af182bdc7b214c4c7445645d0 Author: John Spray Date: Wed Oct 25 09:39:15 2017 -0400 mds: clean up clog damage message when repairing If we've repaired then we don't need to be at WRN and we don't need to advise the user to look up the damage detail. Signed-off-by: John Spray (cherry picked from commit 3160ef2c0a66324621a95926bed0416bbd537353) commit 6e544ac967b533f47ce2708b1757cd048ea2354c Author: John Spray Date: Wed Oct 25 06:30:57 2017 -0400 mds: flush after scrub repairs Otherwise, if we restart the MDS right after the scrub, then scrub again, it will see the same inconsistency when it looks at the on-disk state. This involves adapting the use of ScrubHeader to be something writeable during scrub to record whether repair happened. Signed-off-by: John Spray (cherry picked from commit 7e52729699e87dd75022433153450fd4ba71b40e) commit f19f1e1cbc5b23eeb028c015c77ffe32306aba64 Author: John Spray Date: Wed Oct 25 05:24:52 2017 -0400 mds: don't report repaired backtraces in damagetable Fixes: http://tracker.ceph.com/issues/18743 Signed-off-by: John Spray (cherry picked from commit 444382c3370bafa21564bb1e6a5c80f3e4825f6a) commit 162e25ec04e5288608718dcbd9d57b8234c17fb9 Author: John Spray Date: Wed Oct 25 05:24:03 2017 -0400 mds: cleaner scrub complete log msg People often are not using a tag with their scrub, let's not have the ugly "with tag ''" in that case. Signed-off-by: John Spray (cherry picked from commit 5fd00a6e1b914880ab9f401faf591e73024f18ef) commit 87991643b38b1a0e526b599b65bb010e66c08374 Author: Yan, Zheng Date: Tue Nov 7 12:39:49 2017 +0800 mds: prevent filelock from being stuck at XSYN state Lock:file_eval() does not properly handle the condition: filelock is in XSYN state, both loner_cap and want_loner_cap are >= 0, and loner_cap != want_loner_cap. To set loner to the wanted one, mds need to revoke caps from the old loner. Changing lock state to MIX can do the job. Fixes: http://tracker.ceph.com/issues/22008 Signed-off-by: "Yan, Zheng" Date: Tue Nov 7 10:50:14 2017 +0800 mds: implement LOCK_XSYN_LOCK and LOCK_XSYN_MIX Signed-off-by: "Yan, Zheng" (cherry picked from commit c7261e1a6d95d1ed98a1d9066905aa75af4c2886) commit f354347debffec93910e0db0be4e165c00778d83 Author: Yan, Zheng Date: Tue Nov 7 10:37:44 2017 +0800 mds: cleanup set/drop loner code integrate functionalities of try_drop_loner and try_set_loner into CInode::choose_ideal_loner Signed-off-by: "Yan, Zheng" (cherry picked from commit d4cbf3aeeada57a88b794d3d6d5fb1cd891e31cd) commit ac0e193f402a90202fafa3f1bfff4f99697babba Merge: bd75db92c6 06a1c3d1e4 Author: Yuri Weinstein Date: Tue Feb 6 09:40:01 2018 -0800 Merge pull request #19157 from pdvian/wip-22237-luminous luminous: mds: don't delay processing completed requests in replay queue Reviewed-by: Patrick Donnelly commit bd75db92c630fc6636a357697f01784c9b2aa98e Merge: 4974d42eea 024e11dbfc Author: Yuri Weinstein Date: Tue Feb 6 09:37:50 2018 -0800 Merge pull request #20327 from theanalyst/wip-22922-luminous luminous: rgw: bucket resharding should not update bucket ACL or user stats Reviewed-by: Casey Bodley commit c63fc964cef1d45fd0a4de9c6715bcd16badc597 Author: Yan, Zheng Date: Fri Nov 3 14:14:07 2017 +0800 mds: limit size of MExportDir message Only export port of subtree if whole subtree tree is too large for sinlge MExportDir message. The un-exported portion are added to export queue. Fixes: http://tracker.ceph.com/issues/21892 Signed-off-by: "Yan, Zheng" (cherry picked from commit ca5c987860821a7784db1c6a74d1c661cdf427cc) commit a7fd4f95d2d4462f4e382cd85f0b011786319ff9 Author: Yan, Zheng Date: Fri Nov 3 16:13:32 2017 +0800 mds: optimize MDCache::try_subtree_merge Signed-off-by: "Yan, Zheng" (cherry picked from commit 1f27ebbef95b39f7fa3c94d5dc9f191a58c9f9b4) commit 45f4b1a141fe3c3caafaece0e82f2f35d0586030 Author: Yan, Zheng Date: Fri Nov 3 12:03:44 2017 +0800 mds: optimize import/export state access Signed-off-by: "Yan, Zheng" (cherry picked from commit fb9277c14314aaa6a97ee23da9057ddc3c750408) commit bf3ee8b22bf1c2242189107cc662a28f5ea7c219 Author: Casey Bodley Date: Thu Jan 25 12:09:20 2018 -0500 rgw: fix for empty query string in beast frontend when the target does not contain a ?, the QUERY_STRING was being set to the same value as REQUEST_URI. this QUERY_STRING is included in the signature, and caused SignatureDoesNotMatch failures Fixes: http://tracker.ceph.com/issues/22797 Signed-off-by: Casey Bodley (cherry picked from commit ab9e79684ac7ae33b78522df6732b99271d10016) commit 6caae86b7c7591b1e2902a9002b7649fe99022e5 Author: xie xingguo Date: Fri Sep 15 22:28:40 2017 +0800 os/bluestore: propagate read-EIO for aio Signed-off-by: xie xingguo (cherry picked from commit 301912603789d78e3560da84df1e337edd046e46) commit e57274ee2a51e100579dadae88eb3c61e4fb1642 Author: Pan Liu Date: Wed Sep 6 18:09:55 2017 +0800 os/bluestore: report error and quit correctly when disk error happens. Signed-off-by: Pan Liu (cherry picked from commit ed3175895fa92929089f7507e7d173f557d61e27) commit 68180fafd60681a2ea385f0785463792586f01d5 Author: Pan Liu Date: Wed Sep 6 18:06:06 2017 +0800 os/filestore: use __func__ instead of hardcode function name Signed-off-by: Pan Liu (cherry picked from commit fdb4048a92fad40beab1350a68b29681bd4b6131) commit 9b93ff70a329e3a66b918d8194f851a2c8a5e3d1 Author: xie xingguo Date: Fri Sep 15 14:30:47 2017 +0800 os/bluestore: add "bluestore_read_eio" counter To track down how many EIO errors have been propagated to high level callers. Signed-off-by: xie xingguo (cherry picked from commit d2d989fcc6a194b9374aeea64e45969b08990262) commit 902c01792228beb2a539095fec5f24bd293814b5 Author: xie xingguo Date: Fri Sep 15 10:30:04 2017 +0800 os/bluestore: propagate read-EIO to high level callers E.g., we can let auto-repair to properly handle this instead of crashing the whole osd. Observe this once on one of our test cluster: /clove/vm/clove/ceph/rpmbuild/BUILD/ceph-12.2.0/src/os/bluestore/BlueStore.cc: 6604: FAILED assert(r == 0) Signed-off-by: xie xingguo (cherry picked from commit a51d9e979fbbe49917712889efb102806479fad2) commit 024e11dbfcb7077ce2956969f2242bb9bc814d9d Author: Orit Wasserman Date: Sun Jan 21 10:12:43 2018 +0200 rgw: bucket resharding should not update bucket ACL or user stats bucket link op resets bucket ACL using rgw_link_bucket instead Fixes: http://tracker.ceph.com/issues/22124 Fixes: http://tracker.ceph.com/issues/22742 Signed-off-by: Orit Wasserman (cherry picked from commit 1223baf76b45559f171c67c5e737f8e5f7d843e0) commit 91673b1edfd3d4023d77eb95801db6acf7f900f9 Author: Sage Weil Date: Tue Jan 23 21:07:08 2018 -0600 os/bluestore: change bdev parse error to ENOENT If there is not a valid label, then the label is not found. This is a more reasonable error code than "Invalid argumnet". Signed-off-by: Sage Weil (cherry picked from commit 5cd98b0dfdc7758761ff789b12faca2266a7e128) commit 9c083ea16b02c4daef444629439247a415c557c7 Author: Sage Weil Date: Tue Jan 23 21:06:27 2018 -0600 os/bluestore: make bdev label parse error less noisy This happens during the normal initialization of a new bluestore osd and it is confusing for users. Make it less noisy. Signed-off-by: Sage Weil (cherry picked from commit 14c498dd22d1e89d30a0896e89fe83e8adf6ac76) commit 40a16ac6cc1550a7d8631c10cbc523a089d143a2 Author: Andrew Schoen Date: Wed Jan 31 15:45:44 2018 -0600 ceph-volume: adds tests for zap --destroy and --osd-id These are not added for dmcrypt right now because there is a bug related to zapping encrypted disks that needs fixed first. Signed-off-by: Andrew Schoen (cherry picked from commit 8d38932c211c589b6171d9e17b100869c23890ac) commit 97592c255a6df1caeb97e41f88160e17c6e3878b Author: Andrew Schoen Date: Wed Jan 31 09:04:09 2018 -0600 ceph-volume: no need to return osd_id from util.prepare.check_id Now that osd_id is passed to util.prepare.create_id it doesn't make any sense to return osd_id from check_id anymore as it's not being used. Signed-off-by: Andrew Schoen (cherry picked from commit f88f2d3bc1f5dcae1e9b2be62dab20942374fc29) commit c01a2bf515c08ae0a058fdf0fd2ca5f469ed5880 Author: Andrew Schoen Date: Tue Jan 30 15:00:04 2018 -0600 ceph-volume: when reusing an osd ID you must give that id to 'osd new' If you do not then auth will not be created for the new OSD and the daemon will not be able to start. Signed-off-by: Andrew Schoen (cherry picked from commit 43f699fcbc9f30875f70295e086a4e423008b266) commit 7d0c8833a9b91a6a9da41f4b97938c8dbd244a99 Author: Andrew Schoen Date: Tue Jan 30 11:39:05 2018 -0600 ceph-volume: prepare_* methods do not need to recreate fsid or osd_id The prepare_filestore and prepare_bluestore methods do not need to attempt to recreate osd_id or fsid because this happens in prepare already. Signed-off-by: Andrew Schoen (cherry picked from commit 7c809166aed82298b1fe1581f0e28174a535574f) commit eab0d1e6cd760e3ec854febae3988a8024d136d4 Author: Andrew Schoen Date: Tue Jan 30 11:32:53 2018 -0600 ceph_volume: modify util.prepare.check_id to handle stdout as a list process.call returns stdout as a list, not a string Signed-off-by: Andrew Schoen (cherry picked from commit a08fc639613b3d9bcb3a09875e535287dd58870a) commit d670b7b3fcb191b8e9f415f261b029689087d116 Author: Andrew Schoen Date: Tue Jan 30 09:22:58 2018 -0600 ceph-volume: tests for util.prepare.check_id Signed-off-by: Andrew Schoen (cherry picked from commit 0d27fff319f1c4705648fdaff18ee47441f26a24) commit 55c0eda3f31f1327578daa11435720d1e513a250 Author: Andrew Schoen Date: Tue Jan 30 09:21:47 2018 -0600 ceph-volume: return osd_id from util.prepare.check_id if it exists This also changes this so the osd_id is returned as a string so an ID of 0 would evaluate to True. Signed-off-by: Andrew Schoen (cherry picked from commit 699aa65878ca18f2e2cd70d7444cad18b4a3fd98) commit 5b04509b61dcfa4b920ef16d62a93934df1c363b Author: Andrew Schoen Date: Mon Jan 29 10:48:09 2018 -0600 ceph-volume: check to see if an OSD id exists before reusing it Signed-off-by: Andrew Schoen (cherry picked from commit 548eadeec6fb42a18c3e84dbbd8164862f8836e6) commit 7e8dbc50129b7729036974bf3f20cfbfad2794bb Author: Andrew Schoen Date: Mon Jan 29 10:43:04 2018 -0600 ceph-volume: adds a prepare util for checking OSD ID existance Signed-off-by: Andrew Schoen (cherry picked from commit 4ac99de6d98b646bf160230584f6532775cdc9cb) commit 03569abb408e603e4799f97eac333d118ddf23c7 Author: Shengjing Zhu Date: Thu Feb 1 16:47:26 2018 +0800 pybind/mgr/dashboard: fix duplicated slash in html href The url_prefix val in html templates is expected not containing '/' in the end. Otherwise the href will be rendered as '//static/', which causes browser treating it as http://static/. Fixes: http://tracker.ceph.com/issues/22851 Signed-off-by: Shengjing Zhu (cherry picked from commit 24fe1cae636bf00e3d5923c7e826992d42d92946) commit b47837100f3eae46a2c983f0299e40bf7944c9f8 Author: Alfredo Deza Date: Mon Jan 22 15:16:40 2018 -0500 doc/ceph-volume OSD use the fsid file, not the osd_fsid Signed-off-by: Alfredo Deza (cherry picked from commit a5f26c622b0f11b7d8179da22d18f719b9febe0a) commit 4974d42eea073bd1f2f27cdd3b5a35be789450a7 Merge: 239b1ae5e1 f4d5757fb5 Author: Andrew Schoen Date: Mon Feb 5 16:08:27 2018 +0100 Merge pull request #20241 from ceph/backport-wip-rm22619 luminous ceph-volume: dmcrypt support for lvm Reviewed-by: Andrew Schoen commit 5282a60c8fa50db6fff957673dbd73ec1c22a59a Author: Jianyu Li Date: Fri Jan 5 00:54:30 2018 +0800 Avoid the assert failure when the inode for the cap_export from other MDS happened not in MDCache Signed-off-by: Jianyu Li (cherry picked from commit 82b9b838ae0bd17247f6296f9d7001331f50663c) commit 042144a281073c4cd3e80e969f7eaf7f2b289de5 Author: Shangzhong Zhu Date: Tue Jan 23 20:35:36 2018 +0800 mds: set higher priority for some perf counters Fixes: http://tracker.ceph.com/issues/22776 Signed-off-by: Shangzhong Zhu (cherry picked from commit 05c496d444b388ec505df4598abd3fc6309ed99f) commit 239b1ae5e19c16e976c2045fef5ad65f1f727278 Merge: 982bd23865 7a8e915483 Author: Patrick Donnelly Date: Sat Feb 3 11:31:34 2018 -0800 Merge PR #18930 into luminous * refs/pull/18930/head: mds: additional damage handling case in EImportStart commit 982bd2386543be41eb3d67f9207971d285b08807 Merge: b8392683a1 127236f676 Author: Josh Durgin Date: Fri Feb 2 15:47:11 2018 -0800 Merge pull request #20098 from ovh/bp-luminous-snaptrimq-visibility luminous: osd, pg, mgr: make snap trim queue problems visible Reviewed-by: Josh Durgin commit b8392683a1cc9ab489d3c77f8edac395469b2ce8 Merge: 9083f7db62 5cec46600d Author: Yuri Weinstein Date: Fri Feb 2 15:10:18 2018 -0800 Merge pull request #19267 from tchaikov/wip-ceph-disk-fsid-luminous luminous: ceph_disk: allow "no fsid" on activate Reviewed-by: Josh Durgin Reviewed-by: Amit Kumar commit 9083f7db62b49037d1b6d8c1a092f0028b5882fb Merge: 119c9f0363 bd558d7e72 Author: Yuri Weinstein Date: Fri Feb 2 15:09:49 2018 -0800 Merge pull request #19387 from ceph/wip-cd-luminous-upgrade luminous: tests: ceph-deploy: use wip branch for final upgrade step Reviewed-by: Josh Durgin commit 119c9f03637bcb2e7236b8010fb038f0c3235bad Merge: 5b9c671928 5a40ef2244 Author: Yuri Weinstein Date: Fri Feb 2 15:08:49 2018 -0800 Merge pull request #19610 from ukernel/luminous-22306 luminous: osdc/Journaler: add 'stopping' check to various finish callbacks Reviewed-by: Josh Durgin commit 5b9c6719287aa9e2783cdbec9be95955f877f3bb Merge: faf8e1cc56 44643cae8d Author: Yuri Weinstein Date: Fri Feb 2 15:08:10 2018 -0800 Merge pull request #19967 from smithfarm/wip-22707-luminous luminous: ceph_objectstore_tool: no flush before collection_empty() calls; ObjectStore/StoreTest.SimpleAttrTest/2 fails Reviewed-by: Josh Durgin commit faf8e1cc5654abea8efcc7d05804fdd2db8f5803 Merge: b7728d23be 654678ab2d Author: Yuri Weinstein Date: Fri Feb 2 15:04:29 2018 -0800 Merge pull request #19047 from shinobu-x/wip-22193-luminous luminous: bluestore: OSD crash on boot with assert caused by Bluefs on flush write Reviewed-by: Kefu Chai commit b7728d23bec0fb8af5d10d8af6c48d325331818c Merge: f185e67496 e56106cfc1 Author: Yuri Weinstein Date: Fri Feb 2 15:01:50 2018 -0800 Merge pull request #19388 from ceph/wip-cv-luminous-bp luminous: qa: ceph-volume updates Reviewed-by: Alfredo Deza commit f185e67496721fe04cef4d99d097242a8277b2bd Merge: 9ed49bdb3e e621e9d7f1 Author: Yuri Weinstein Date: Fri Feb 2 15:01:04 2018 -0800 Merge pull request #19487 from smithfarm/wip-22389-luminous luminous: ceph-objectstore-tool: Add option dump-import to examine an export Reviewed-by: David Zafman commit 9ed49bdb3ed0953e348bcde7cc73d39022b5a759 Merge: 2d92e6a4b8 5e5d5c8196 Author: Yuri Weinstein Date: Fri Feb 2 15:00:26 2018 -0800 Merge pull request #19501 from shinobu-x/wip-22399-luminous luminous: Manager daemon x is unresponsive. No standby daemons available Reviewed-by: John Spray commit 2d92e6a4b89d5bd12aef5f7049f94a63ca1228af Merge: e579ed377d 6cc4a33933 Author: Yuri Weinstein Date: Fri Feb 2 14:59:49 2018 -0800 Merge pull request #19865 from smithfarm/wip-22634-luminous luminous: build/ops: ceph-mgr dashboard has dependency on python-jinja2 Reviewed-by: John Spray commit e579ed377daf35679c551a254ea0770806499e03 Merge: 595a5984db d2a68571a8 Author: Yuri Weinstein Date: Fri Feb 2 14:59:08 2018 -0800 Merge pull request #19966 from smithfarm/wip-22706-luminous luminous: tests: force backfill test can conflict with pool removal Reviewed-by: Kefu Chai commit 595a5984db75025018ab0f740e1e2de7eebf7931 Merge: 241573d06c cd9e6788cb Author: Yuri Weinstein Date: Fri Feb 2 14:58:23 2018 -0800 Merge pull request #20116 from smithfarm/wip-22266-luminous tools/ceph_monstore_tool: include mgrmap in initial paxos epoch Reviewed-by: Kefu Chai commit 241573d06ca41144e1674f418ab7ae923485e20d Merge: 2d6afdca94 f454c293d1 Author: Yuri Weinstein Date: Fri Feb 2 14:57:20 2018 -0800 Merge pull request #20215 from smithfarm/wip-22701-luminous luminous: build/ops: ceph-volume fails when centos7 image doesn't have lvm2 installed Reviewed-by: Alfredo Deza commit 2d6afdca9449c94d81aa1c10266894c94f192f2b Merge: fcfd31a5eb f2f00cac71 Author: Yuri Weinstein Date: Fri Feb 2 14:56:30 2018 -0800 Merge pull request #20219 from tchaikov/wip-pr-20128-luminous luminous: common/pick_address: wrong prefix_len in pick_iface() Reviewed-by: Nathan Cutler commit fcfd31a5ebc7cad1a1a4304a007cf48374a4f90c Merge: 77c4502c8d 9d617b4ecc Author: Yuri Weinstein Date: Fri Feb 2 14:38:23 2018 -0800 Merge pull request #19849 from liewegas/wip-monlog-luminous mon: allow cluster and debug logs to both go to stderr with different prefixes Reviewed-by: Kefu Chai Reviewed-by: Joao Eduardo Luis commit 77c4502c8d566cb1e63d8f42b0088f790922e61e Merge: f24fb875a8 f59380e159 Author: Yuri Weinstein Date: Fri Feb 2 14:36:55 2018 -0800 Merge pull request #19888 from pdvian/wip-22633-luminous luminous: OSD crushes with FAILED assert(used_blocks.size() > count) during the first start after upgrade 12.2.1 -> 12.2.2 Reviewed-by: Josh Durgin commit f24fb875a8313d0965dce7dad05d988f6df118d7 Merge: db33f288c0 8b13643b32 Author: Yuri Weinstein Date: Fri Feb 2 14:36:03 2018 -0800 Merge pull request #19928 from jdurgin/wip-luminous-omap-recovery config: lower default omap entries recovered at once Reviewed-by: Yuri Weinstein commit db33f288c036bf0ebdf540eb607745d53ad4589c Merge: 9379ed806a d5e2e43de8 Author: Yuri Weinstein Date: Fri Feb 2 14:35:38 2018 -0800 Merge pull request #19938 from liewegas/wip-list-size-luminous luminous: common: compute SimpleLRU's size with contents.size() instead of lru.size() Reviewed-by: Josh Durgin commit 9379ed806ae419871cd535ce067e7e7adc3eec8a Merge: 5879a59eb6 e9f5612bd7 Author: Yuri Weinstein Date: Fri Feb 2 14:34:27 2018 -0800 Merge pull request #19947 from tchaikov/wip-pr-18191-luminous qa/standalone/osd/osd-mark-down: create pool to get updated osdmap faster Reviewed-by: Josh Durgin commit 5879a59eb65b6d5277658a8a130d57be00c60d4b Merge: 8f212ed733 8e3edae0c8 Author: Yuri Weinstein Date: Fri Feb 2 14:33:48 2018 -0800 Merge pull request #19969 from smithfarm/wip-22691-luminous luminous: ceph-base symbols not stripped in debs Reviewed-by: Josh Durgin commit 8f212ed733baa63f0d971f00e23b8b92cd416fe8 Merge: d9542bcf02 dc96e3c1f8 Author: Yuri Weinstein Date: Fri Feb 2 14:33:13 2018 -0800 Merge pull request #19976 from pdvian/wip-22690-luminous luminous: tests: increase osd count for ec testing Reviewed-by: Josh Durgin commit d9542bcf02dd3574d27eca04fb7ab799d06b8452 Merge: 29ffc82d0b 91be5317da Author: Yuri Weinstein Date: Fri Feb 2 14:32:39 2018 -0800 Merge pull request #19977 from pdvian/wip-22692-luminous luminous: common: compute SimpleLRU's size with contents.size() instead of lru.… Reviewed-by: Josh Durgin commit 29ffc82d0bdb34a3c3e6b89df87b95b3bc6361e5 Merge: 6b80c14af1 07a122a3fd Author: Yuri Weinstein Date: Fri Feb 2 14:31:56 2018 -0800 Merge pull request #19995 from pdvian/wip-22698-luminous luminous: common/throttle: start using 64-bit values Reviewed-by: Josh Durgin Reviewed-by: Nathan Cutler commit f4d5757fb574411e8e47055f2aaaf54bce076501 Author: Alfredo Deza Date: Mon Jan 22 14:21:37 2018 -0500 doc/ceph-volume add lvm/encryption to the index Signed-off-by: Alfredo Deza (cherry picked from commit 163ee6a61235e97c341b2817114d98b576697096) commit 4d237d7418a7c1a6f7bce80d978ee5fae8e2d22a Author: Alfredo Deza Date: Mon Jan 22 12:11:32 2018 -0500 doc/man/ceph-volume add dmcrypt support Signed-off-by: Alfredo Deza (cherry picked from commit babcdf2fa59baab6bc2cd8a1ac190b8d02fa3fce) commit 5a7b50f121aee3fc13474b42e11b5d19a36d7764 Author: Alfredo Deza Date: Mon Jan 22 12:06:25 2018 -0500 doc/man/ceph-volume add docs for zap sub-command Signed-off-by: Alfredo Deza (cherry picked from commit ad73be2c2a6ee273d37b706626974d6900742256) commit 3c8abd24b2801a8e76f4b327a1b029cb0eecfdd7 Author: Alfredo Deza Date: Mon Jan 22 12:03:18 2018 -0500 doc/man/ceph-volume add docs for list sub-command Signed-off-by: Alfredo Deza (cherry picked from commit 498d7a06a1f17c769c11da7ba9f83bcf2c0e433d) commit e51d7dd4da3a156250489f33deddcbe8c208d2aa Author: Alfredo Deza Date: Mon Jan 22 11:41:32 2018 -0500 doc/dev/ceph-volume update internal LVM metadata information Includes dmcrypt information, and updated key names to reflect current code Signed-off-by: Alfredo Deza (cherry picked from commit 46babe80b638eb607e2af6f0c8bb77e675d4d63b) commit 8b938c42f76b95425476c7ee2df275fa721d8de9 Author: Alfredo Deza Date: Fri Jan 19 11:12:52 2018 -0500 ceph-volume tests.functional add xenial filestore tests for dmcrypt Signed-off-by: Alfredo Deza (cherry picked from commit f0f0c1f8c4dc433d8978de134c120328842a5d7f) commit 5c405a05fcb07b8abd81a11c152f197e3596db78 Author: Alfredo Deza Date: Fri Jan 19 11:12:40 2018 -0500 ceph-volume tests.functional add xenial bluestore tests for dmcrypt Signed-off-by: Alfredo Deza (cherry picked from commit be0dfe76535fa135168763275513d7fafba6fe4c) commit 558b5c62fcc1dd6ac7627e11523c60b0bfab3787 Author: Alfredo Deza Date: Fri Jan 19 11:08:25 2018 -0500 ceph-volume tests.functional add dmcrypt support in tox.ini Signed-off-by: Alfredo Deza (cherry picked from commit d1ef5c487ba1ea868fd85b8c425986a2cf435519) commit 95c279b4363a3630422f15a3bd5ad33bd10204e3 Author: Alfredo Deza Date: Fri Jan 19 11:03:58 2018 -0500 ceph-volume tests.functional add centos7 filestore tests for dmcrypt Signed-off-by: Alfredo Deza (cherry picked from commit 2477ac805e172a10ae3214151a78b4601cf88021) commit 9b9ed02de83fa4397f776427fa37a21fdb16f4b9 Author: Alfredo Deza Date: Fri Jan 19 10:45:32 2018 -0500 ceph-volume tests.functional add centos7 bluestore tests for dmcrypt Signed-off-by: Alfredo Deza (cherry picked from commit 18972322567e158a56d0e71ca4f2afbed413c855) commit 1c560cd8053ab5e6b921b3c1365cb70a1c71e811 Author: Alfredo Deza Date: Fri Jan 19 09:58:36 2018 -0500 ceph-volume tests add util.prepare unit tests Signed-off-by: Alfredo Deza (cherry picked from commit a774eae43a089798f895507f18b0c5d68a49fbb4) commit 43fb4561d8c44ddd5d246e601671d91606a09ca3 Author: Alfredo Deza Date: Fri Jan 19 09:58:18 2018 -0500 ceph-volume tests add fixture helpers for call and run Signed-off-by: Alfredo Deza (cherry picked from commit d51973a64d9f18e0dfde7601b3ffbc57571ea653) commit 22c10490150426a45c832a59cf6613d331f7cb19 Author: Alfredo Deza Date: Thu Jan 18 16:37:00 2018 -0500 ceph-volume util allow prepare module to work with encryption keys Signed-off-by: Alfredo Deza (cherry picked from commit a36dcdf9cfbc3ca012c5cb022e0d7dbcd95984f4) commit b5f674daff7113a7c17b10cd213734140545b018 Author: Alfredo Deza Date: Thu Jan 18 16:22:06 2018 -0500 ceph-volume lvm.activate allow encryption setups for luks Signed-off-by: Alfredo Deza (cherry picked from commit 3476e87cec353b100cf8c117d74708f51dd6de2c) commit a26321213b8d77084345af306d7ee8a73b238255 Author: Alfredo Deza Date: Thu Jan 18 16:19:26 2018 -0500 ceph-volume lvm.prepare allow encryption setups for luks Signed-off-by: Alfredo Deza (cherry picked from commit be3300455e479b03ac072d28bb05ccc7eaddb2e9) commit fde4817d87132d5dade973fba36743f8b3c37230 Author: Alfredo Deza Date: Thu Jan 18 14:46:42 2018 -0500 ceph-volume process allow silencing output on file logging for sensitive data Signed-off-by: Alfredo Deza (cherry picked from commit 0e2190c7a22b61aeecdb60b6c8bfe5ba6aa873a4) commit f357d5db65d427c1184169e588c475742ac1a45d Author: Alfredo Deza Date: Thu Jan 18 14:29:53 2018 -0500 ceph-volume util create a helper module for encryption Signed-off-by: Alfredo Deza (cherry picked from commit dc34a32675a870da98931adffacb2d3db4f74c6c) commit 38c8a384b22da1de36b567d2b3816ba936083216 Author: Alfredo Deza Date: Tue Jan 16 09:02:31 2018 -0500 ceph-volume lvm.common add dmcrypt flag Signed-off-by: Alfredo Deza (cherry picked from commit 88ff873e9a60bd811356f7e15c8ef7f826d09337) commit 65a38c7855f3e1ce7788d25d02ac7ca779b917a0 Author: Alfredo Deza Date: Thu Jan 11 09:02:03 2018 -0500 doc/ceph-volume create an encryption doc explaining dmcrypt workflows Signed-off-by: Alfredo Deza (cherry picked from commit 477670a0912c9e01fe59155bf135eaf92bdc3318) commit 6b80c14af1f36dbb0bd5c73d9935d41bf7d53350 Merge: a219bf4727 0aa31d29d5 Author: Sage Weil Date: Fri Feb 2 13:19:51 2018 -0600 Merge pull request #20244 from ceph/backport-19276 luminous: ceph-volume: do not use --key during mkfs Reviewed-by: Sage Weil Reviewed-by: Kefu Chai commit 7756a44529dd27e0960393e098104c7c1808bb36 Author: Yan, Zheng Date: Thu Jan 25 12:08:47 2018 +0800 osdc/ObjectCacher: don't merge TX buffer heads Signed-off-by: "Yan, Zheng" (cherry picked from commit 0b1e515d1c5dddf3c11eddbe6d53167d2fb6aaab) commit 43b37b83df897b57c10cea2ae307897f1f7fb96c Author: Yan, Zheng Date: Thu Jan 25 10:25:25 2018 +0800 osdc/ObjectCacher: fix off-by-one error in bh_write_commit() Fixes: https://tracker.ceph.com/issues/22741 Signed-off-by: "Yan, Zheng" (cherry picked from commit cc4c6dcc2ffdc482f1915db569949e674691c372) commit d93eef31277d30020b016d74e8d6b7e93e850fd9 Author: dongdong tao Date: Fri Jan 19 00:08:44 2018 +0800 mds: fix scrub crash Fixes: http://tracker.ceph.com/issues/22730 Signed-off-by: dongdong tao (cherry picked from commit e741c1a6e16cb5179d623ef491f66d31b0ffdfd6) commit a219bf47277d923db18dd221660a04bb6bac3580 Merge: e62f21ca34 f57a80f0a8 Author: Josh Durgin Date: Thu Feb 1 15:40:51 2018 -0800 Merge pull request #20247 from ceph/revert-19552-wip-22452-luminous Revert " luminous: msg/async: unregister connection failed when racing happened" Reviewed-by: Josh Durgin commit f57a80f0a8f4ad4d7d482c05ac8b25fd160f55df Author: Sage Weil Date: Thu Feb 1 17:28:06 2018 -0600 Revert " luminous: msg/async: unregister connection failed when racing happened" commit e62f21ca347750d3289bc1f66f5066a09c3445f8 Merge: 419ed62db3 82c9e3dbe1 Author: Yuri Weinstein Date: Thu Feb 1 15:10:13 2018 -0800 Merge pull request #19187 from liewegas/wip-trim-mgrmap-luminous mon/MgrMonitor: limit mgrmap history Reviewed-by: Josh Durgin Reviewed-by: Kefu Chai commit 419ed62db3d8a6aed94c1598e6dadaddbac7b9ab Merge: 11c3f8f3dc 3bfb493fa0 Author: Yuri Weinstein Date: Thu Feb 1 15:09:29 2018 -0800 Merge pull request #19269 from tchaikov/wip-pr-19202-luminous luminous: cmake,common/RWLock: check for libpthread extensions Reviewed-by: Amit Kumar Reviewed-by: Josh Durgin Reviewed-by: Casey Bodley commit 11c3f8f3dc179d34f3197ce7ffd07731b5e652c9 Merge: 4106dbddea c22f06b3c5 Author: Yuri Weinstein Date: Thu Feb 1 15:08:45 2018 -0800 Merge pull request #19499 from shinobu-x/wip-22402-luminous luminous: osd: replica read can trigger cache promotion Reviewed-by: Josh Durgin Reviewed-by: Amit Kumar commit 4106dbddeab069e816219c1fc775852eab8fd1ff Merge: 6dcfa33858 9a9ed272a7 Author: Yuri Weinstein Date: Thu Feb 1 15:08:07 2018 -0800 Merge pull request #19552 from shinobu-x/wip-22452-luminous luminous: msg/async: unregister connection failed when racing happened Reviewed-by: Josh Durgin commit 6dcfa3385836357f556e5d45e97a0b05eda5df58 Merge: 589342d932 47f74bcd13 Author: Yuri Weinstein Date: Thu Feb 1 15:07:36 2018 -0800 Merge pull request #19628 from shinobu-x/wip-22501-luminous luminous: tests: do not configure ec data pool with memstore Reviewed-by: Josh Durgin commit 589342d932d3e7310167842dd22c1303bf1156d5 Merge: 37ce36b6d0 ce54fd9b99 Author: Yuri Weinstein Date: Thu Feb 1 15:06:56 2018 -0800 Merge pull request #19630 from shinobu-x/wip-22507-luminous luminous: bluestore: do not crash on over-large objects Reviewed-by: Josh Durgin commit 37ce36b6d0e630b0c527ead826d9761fd4cb9622 Merge: bb9964ae21 6791e67faf Author: Yuri Weinstein Date: Thu Feb 1 15:06:17 2018 -0800 Merge pull request #19742 from tchaikov/wip-luminous-pr-19649-and-19650 luminous: common/dns, erasure-code: fix mem leaks Reviewed-by: Josh Durgin commit bb9964ae2163eeff35538e4df897317c1e477fa8 Merge: 6f0167b181 ac064e1fc8 Author: Yuri Weinstein Date: Thu Feb 1 15:02:15 2018 -0800 Merge pull request #20049 from liewegas/wip-zero-length-luminous os: fix 0-length zero semantics, test Reviewed-by: Josh Durgin commit 6f0167b181a5850b9c40128b682ab210b217824a Merge: 742b7ba11f a0c0e637fe Author: Yuri Weinstein Date: Thu Feb 1 15:01:35 2018 -0800 Merge pull request #20068 from smithfarm/wip-22761-luminous luminous: osd: do not check out-of-date osdmap for DESTROYED flag on start Reviewed-by: Josh Durgin commit 742b7ba11fae67fe806306284b9ae76fcb8160a0 Merge: c8082bda93 238ddccbbc Author: Yuri Weinstein Date: Thu Feb 1 15:01:11 2018 -0800 Merge pull request #20069 from smithfarm/wip-22770-luminous luminous: tools: ceph-objectstore-tool set-size should clear data-digest Reviewed-by: Josh Durgin commit c8082bda938d6db1c4def4617875ab99429d78aa Merge: df3088dbfd 8950736b8e Author: Yuri Weinstein Date: Thu Feb 1 15:00:41 2018 -0800 Merge pull request #20150 from pdvian/wip-22807-luminous luminous: mon: do not use per_pool_sum_delta to show recovery summary Reviewed-by: Josh Durgin commit 0aa31d29d5eb3b2071dbdb678fc3f1dc303a64ea Author: Kefu Chai Date: Sat Dec 2 00:43:21 2017 +0800 ceph-volume: process: disable stdin param of run() we cannot use process.communicate() to feed the Popen with input, because, upon return of process.communicate() the stdout,stderr are closed. see https://docs.python.org/2/library/subprocess.html#subprocess.Popen.communicate . Signed-off-by: Kefu Chai (cherry picked from commit 6df444cf33a251f5dcb536606b44af696cdb4ed5) Conflicts: src/ceph-volume/ceph_volume/process.py commit a67d46b63228517cd9ce37617a7e0f5f79ab16f1 Author: Sage Weil Date: Thu Nov 30 08:33:21 2017 -0600 ceph-volume: use --keyfile instead of --key We do not want the key to show up on the command line (it may appear in the process list or sudo log file). Fixes: http://tracker.ceph.com/issues/22283 Signed-off-by: Sage Weil (cherry picked from commit 46abd50f8bcbd347d577581ae42156b718556158) commit 5717860f38cc79e165bd5bf791064177f8a4200d Author: Sage Weil Date: Thu Nov 30 08:32:47 2017 -0600 osd: accept 'keyfile' as well as 'key' during mkfs Also, don't print the raw key to the log. Signed-off-by: Sage Weil Signed-off-by: Kefu Chai (cherry picked from commit 18aff37ee3a2c2d3016f08c98161a5acc41ff8e5) commit df3088dbfd021393106a5fba93bd722a5e0f540a Merge: d43ec551af cf6799e4b5 Author: Yuri Weinstein Date: Thu Feb 1 13:15:14 2018 -0800 Merge pull request #19370 from shinobu-x/wip-22339-luminous luminous: client: quit on failed remount during dentry invalidate test #19370 Reviewed-by: Patrick Donnelly commit d43ec551af92b7c205f80c1c6617a9b258a98697 Merge: 685e526cc5 4943899005 Author: Yuri Weinstein Date: Thu Feb 1 13:14:47 2018 -0800 Merge pull request #19585 from ukernel/luminous-22460 luminous: mds: handle client session messages when mds is stopping Reviewed-by: Patrick Donnelly commit 685e526cc5d601f21292f13f56cfdcafd5c3c9b4 Merge: 5ca708cd9a d0d66c73b0 Author: Yuri Weinstein Date: Thu Feb 1 13:14:18 2018 -0800 Merge pull request #19626 from shinobu-x/wip-22499-luminous luminous: cephfs-journal-tool: tool would miss to report some invalid range Reviewed-by: Patrick Donnelly commit 5ca708cd9ac00e63557648ba14b4ef6b369dbaa1 Merge: b5c189b84e 6eea75200f Author: Yuri Weinstein Date: Thu Feb 1 13:13:48 2018 -0800 Merge pull request #19627 from shinobu-x/wip-22500-luminous luminous: cephfs: potential adjust failure in lru_expire Reviewed-by: Patrick Donnelly commit b5c189b84e7a98ece7c0d485ee7ec87bb2cdc0cf Merge: a6189cc37f 08edb16311 Author: Yuri Weinstein Date: Thu Feb 1 13:13:16 2018 -0800 Merge pull request #19776 from ukernel/luminous-22492 luminous: mds: respect mds_client_writeable_range_max_inc_objs config Reviewed-by: Patrick Donnelly commit a6189cc37f169ecdce05576fd88362f7b2871927 Merge: b4bc724549 dd6b6626cc Author: Yuri Weinstein Date: Thu Feb 1 13:12:48 2018 -0800 Merge pull request #19829 from pdvian/wip-22573-luminous luminous: vstart_runner: fixes for recent cephfs changes Reviewed-by: Patrick Donnelly commit b4bc7245497d2550c9fc2fd047ac9a185c334b7b Merge: 66189e1656 98e3e2a7c8 Author: Yuri Weinstein Date: Thu Feb 1 13:12:20 2018 -0800 Merge pull request #19830 from shinobu-x/wip-22579-luminous luminous: mds: check for CEPH_OSDMAP_FULL is now wrong; cluster full flag is obsolete Reviewed-by: Patrick Donnelly commit 66189e16560485cdea9b97b46e47371cf134b249 Merge: b35c39f6db 5dcd2a5977 Author: Yuri Weinstein Date: Thu Feb 1 13:11:52 2018 -0800 Merge pull request #19871 from smithfarm/wip-21948-luminous luminous: mon: MDSMonitor: reject misconfigured mds_blacklist_interval Reviewed-by: Patrick Donnelly commit b35c39f6db246463045a5eb6b5c477b99fdeab77 Merge: 243f187c02 5fd01ca888 Author: Yuri Weinstein Date: Thu Feb 1 13:11:28 2018 -0800 Merge pull request #19959 from smithfarm/wip-22694-luminous luminous: mds: fix dump last_sent Reviewed-by: Patrick Donnelly commit 243f187c023962f17fa7715c196c45e712ea51a1 Merge: b1c27aedb9 80ea9ed3b3 Author: Yuri Weinstein Date: Thu Feb 1 13:11:02 2018 -0800 Merge pull request #19962 from smithfarm/wip-22580-luminous luminous: tests: full flag not set on osdmap for tasks.cephfs.test_full Reviewed-by: Patrick Donnelly commit b1c27aedb92d79f507b4fa0bc514b2d173d87296 Merge: f60c2df92d cc76ab1eef Author: Yuri Weinstein Date: Thu Feb 1 13:10:29 2018 -0800 Merge pull request #19968 from ukernel/luminous-22699 luminous: ceph-fuse: ::rmdir() uses a deleted memory structure of dentry leads … Reviewed-by: Patrick Donnelly commit f60c2df92d021f4eb82da5e74de06464818e30c8 Merge: 69c87bedcc 5353569eea Author: Yuri Weinstein Date: Thu Feb 1 13:09:07 2018 -0800 Merge pull request #19982 from ukernel/luminous-22719 luminous: mds: handle 'inode gets queued for recovery multiple times' Reviewed-by: Patrick Donnelly commit 69c87bedcc0814e10db6d38b50f836be6c0528c9 Merge: d81ce5c0ed 2260f03b36 Author: Yuri Weinstein Date: Thu Feb 1 13:05:39 2018 -0800 Merge pull request #20042 from tchaikov/wip-pr-19998-luminous luminous: fix broken use of streamstream::rdbuf() Reviewed-by: Sage Weil commit d81ce5c0ed368d2ec01d539b282966e25b4138e7 Merge: 2b0599d308 3ccf7e0bc6 Author: Yuri Weinstein Date: Thu Feb 1 13:04:16 2018 -0800 Merge pull request #20106 from smithfarm/wip-22502-luminous luminous: osd: Pool Compression type option doesn't apply to new OSDs Reviewed-by: Kefu Chai commit 2b0599d308ecf7e7d637caf1932eef8020ff49fc Merge: 2ac9bbdd2e b12e00097b Author: Alfredo Deza Date: Thu Feb 1 15:07:48 2018 -0500 Merge pull request #20240 from ceph/backport-20010 luminous: ceph-volume: adds a --destroy flag to ceph-volume lvm zap Reviewed-by: Alfredo Deza commit b12e00097b7dc9ad322abc5f4e53d7367b3ee37c Author: Andrew Schoen Date: Thu Jan 18 14:54:55 2018 -0600 ceph-volume: clean up docstrings for lvm zap Signed-off-by: Andrew Schoen (cherry picked from commit 57471fe7e74c94351e713ff99cbcbaa7dc6a0f52) commit d06e7d52d27f19ec1ab9d39af4f45733fbe21246 Author: Andrew Schoen Date: Thu Jan 18 14:41:53 2018 -0600 ceph-volume: print a message to terminal if --destroy is skipped Signed-off-by: Andrew Schoen (cherry picked from commit d2529ee5dd9a4dc6a81eeb2d5e54c0bdd6beeb2f) commit aba99e70d9f3f555b7d7e183d2ea9e506ca2ca40 Author: Andrew Schoen Date: Thu Jan 18 14:34:22 2018 -0600 ceph-volume: no need to set PVolume.vg_name explicitly This will be set automatically in __init__ by looping through kw.items and creating properties on PVolume with them. Signed-off-by: Andrew Schoen (cherry picked from commit 129182645c4a7e5e42500d3ef2083cfdfb0137e4) commit 039d1875e4e5ca52e2d54c35ff2dbfea25202d49 Author: Andrew Schoen Date: Thu Jan 18 14:31:20 2018 -0600 ceph-volume: switch remove_vg and remove_pv to process.run Signed-off-by: Andrew Schoen (cherry picked from commit 5a1be9a73919e13fe603861588bcc20872a4c133) commit 5f98bc964b2ef05110363c70507048c85e33fa8c Author: Andrew Schoen Date: Thu Jan 18 14:25:46 2018 -0600 ceph-volume: allow passing a fail_msg param to process.run This will allow the user to provide a more detailed error message on why the command failed. Signed-off-by: Andrew Schoen (cherry picked from commit 7204851042fa42799e1003ef57d57a512d4a7f28) Conflicts: src/ceph-volume/ceph_volume/process.py commit 2ac9bbdd2ee98f0b07f61d507fbf56f5bfbe213e Merge: 3f0d1f7166 ccc6b55463 Author: Alfredo Deza Date: Thu Feb 1 15:01:04 2018 -0500 Merge pull request #20239 from ceph/backport-19949 luminous: ceph-volume: adds --crush-device-class flag for lvm prepare and create #19949 Reviewed-by: Alfredo Deza commit 8d7552bfff302728669b51df41c2334e2520e2e8 Author: Andrew Schoen Date: Thu Jan 18 12:04:38 2018 -0600 ceph-volume: tests for lvm zap --destroy Signed-off-by: Andrew Schoen (cherry picked from commit fafa183d8f13fa769eaab5f865f6592b89225901) commit 4f3ebfe4e1f008230bad5df7b42e6c103c75e79d Author: Andrew Schoen Date: Thu Jan 18 11:10:48 2018 -0600 ceph-volume: also remove pvs with lvm zap --destroy Leaving the pv around doesn't keep us from redeploying on that device because we also wipefs, but explicitly destroying it makes the output read better. Signed-off-by: Andrew Schoen (cherry picked from commit cb748ccb2264aea93dd4717a46d7f833c5b1eccd) commit 2a42491c65bc49bbcd7ecd86ca61878dde5dc73d Author: Andrew Schoen Date: Thu Jan 18 11:10:15 2018 -0600 ceph-volume: adds an api.lvm.remove_pv method Used to remove physical volumes Signed-off-by: Andrew Schoen (cherry picked from commit 0811c02b118c270b2bec87cd26a877b26bc780b1) commit 8fd6f29ee3988d3158467e9cf447e63c7d3485e5 Author: Andrew Schoen Date: Thu Jan 18 11:05:32 2018 -0600 ceph-volume: remove the -a flag from api.lvm.get_pvs When we run `pvs -a` it returns all pvs, even ones that have been recently deleted. Because of this pvs that have just been zapped with `lvm zap` show up and can cause issues if a device is zapped more than once. Signed-off-by: Andrew Schoen (cherry picked from commit 10a349742cae909d1ebf0a0b3b49c88774929f53) commit 4d23258e19b56d04f2d6f108ad808b01bc38b831 Author: Andrew Schoen Date: Thu Jan 18 08:26:07 2018 -0600 ceph-volume: docs for the --destroy flag of lvm zap Signed-off-by: Andrew Schoen (cherry picked from commit 708cb43c503e962304211041570d6ca660bca1fb) commit 89a0aa94c9b5becee205a1803541ceb2aad961ad Author: Andrew Schoen Date: Wed Jan 17 15:11:19 2018 -0600 ceph-volume: adds the --destroy flag to ceph-volume lvm zap If you use the --destroy flag and are zapping a raw device or parition then zap will destroy any vgs or lvs it finds on that device. Signed-off-by: Andrew Schoen (cherry picked from commit aea326b772395c076e0d75f7c2f591a46aef7f57) commit 3002953e4f506850339c676a85dc5402def7423d Author: Andrew Schoen Date: Wed Jan 17 13:12:17 2018 -0600 ceph-volume: adds an api.lvm.remove_vg method This method can be used to remove volume groups Signed-off-by: Andrew Schoen (cherry picked from commit 50a6a543a956d133b428cf36ad629c2131be5490) commit 9146305907b1c8d9f0162b44c76fd6133b7cb094 Author: Andrew Schoen Date: Wed Jan 17 12:58:50 2018 -0600 ceph-volume: expose vg_name in api.lvm.get_api_pvs Signed-off-by: Andrew Schoen (cherry picked from commit b7f24ec451e167d81a83c1e21597a9fece4f0684) commit 3f0d1f7166e20defae5ed8783f7f49f7a8b461d1 Merge: 5250230360 4bd607b2ae Author: Alfredo Deza Date: Thu Feb 1 14:55:57 2018 -0500 Merge pull request #20238 from ceph/backport-19875 luminous: ceph-volume: adds success messages for lvm prepare/activate/create Reviewed-by: Alfredo Deza commit ccc6b55463335895b5896f4e772f6c0e412d7522 Author: Andrew Schoen Date: Wed Jan 17 11:44:10 2018 -0600 ceph-volume: print the lvm activate success message correctly Move the terminal.success to the methods that have access to the correct osd_id so that they do not print None for the osd_id in the log message. Signed-off-by: Andrew Schoen (cherry picked from commit c9f7689a7a71a6b2321301b84599f319e8ae8eba) commit 0e47c46fef88b3dc4ee9c7849da67f24db375044 Author: Andrew Schoen Date: Mon Jan 15 11:55:44 2018 -0600 ceph-volume: docs for --crush-device-class Signed-off-by: Andrew Schoen (cherry picked from commit 21f459437bd1208812504a20e5a50f7a47bfa979) commit 73e714357d898a06b7bb172691a0dbe360f0f535 Author: Andrew Schoen Date: Thu Jan 11 13:11:31 2018 -0600 ceph-volume: adds crush_device_class to json metadata for 'osd new' Signed-off-by: Andrew Schoen (cherry picked from commit 3404d8bba503f3f74f411b11108d39cb626d3bed) commit 5d9b454522128c0ba0ee1c93485accc48e91e7a4 Author: Andrew Schoen Date: Thu Jan 11 13:04:07 2018 -0600 ceph-volume: adds crush_device_class to the functional tests Signed-off-by: Andrew Schoen (cherry picked from commit 36a388d72acb049b5ee25ceea6a8d341aa58e31f) commit 00efa1a1beac442fb008f2ccb2e26939eab2cc10 Author: Andrew Schoen Date: Wed Jan 10 12:19:54 2018 -0600 ceph-volume: add crush_device_class to lvm metadata Signed-off-by: Andrew Schoen (cherry picked from commit 11da2185635ce14103471d851f1e8c84199d3745) commit 48591335c3903cb5446208da9940752fb8b390c7 Author: Andrew Schoen Date: Wed Jan 10 09:29:15 2018 -0600 ceph-volume: adds a --crush-device-class flag to lvm create and prepare Signed-off-by: Andrew Schoen (cherry picked from commit bf468bc3737c15a8f15ec19ebf81c42672e28aaf) commit 4bd607b2ae7519414e8dd5c92e7522c8c5024060 Author: Andrew Schoen Date: Tue Jan 9 12:21:35 2018 -0600 ceph-volume: adds success message to ceph-volume simple activate This used to only show the message if systemd was not being used, now it will show this always if the command is successful. Signed-off-by: Andrew Schoen (cherry picked from commit 87c6ed14e7e0b528a1fe107e2e531908cdea0f47) commit ad73312ff0d376b8e055936e2f9c6d02ec90d0f4 Author: Andrew Schoen Date: Tue Jan 9 09:52:23 2018 -0600 ceph-volume: add success message for ceph-volume lvm create Signed-off-by: Andrew Schoen (cherry picked from commit 0185d93cc1be1e7a6d3a803631ff92df9787754a) commit 4709380f6b23d7bb5e6c0b98f7745d2495f2af07 Author: Andrew Schoen Date: Tue Jan 9 09:48:09 2018 -0600 ceph-volume: add success message for ceph-volume lvm activate Signed-off-by: Andrew Schoen (cherry picked from commit d30928d7c7d7cfb7e465ec2e7e46dc48407436f0) commit ce5df11e06c85e2d00ee1bdd98a0386212effce6 Author: Andrew Schoen Date: Tue Jan 9 09:45:40 2018 -0600 ceph-volume: add success message for ceph-volume lvm prepare Signed-off-by: Andrew Schoen (cherry picked from commit 75c460f6e1864f15e1d23146c5740ce43c000e42) commit 5250230360a7d91d6cee1a425e183c8220684889 Merge: 4e9092b194 25fbed91ec Author: Andrew Schoen Date: Thu Feb 1 20:43:41 2018 +0100 Merge pull request #20237 from ceph/backport-wip-rm22281 luminous: ceph-volume rollback on failed OSD prepare/create Reviewed-by: Andrew Schoen commit 25fbed91ec23260cba944411929055a75d151f96 Author: Alfredo Deza Date: Tue Dec 5 15:31:55 2017 -0500 ceph-volume lvm.create rollback osd creation when prepare or activate fails Signed-off-by: Alfredo Deza (cherry picked from commit 07be6fe0ab7981ebdc34779a8ccd6220d640a549) commit d6598635b89eb3dd11389c4fae322b8f3934b5c8 Author: Alfredo Deza Date: Tue Dec 5 10:22:05 2017 -0500 ceph-volume lvm.create fallback to bluestore when objectstore is not specified Signed-off-by: Alfredo Deza (cherry picked from commit 6781d6498e01bfc89189c235540e5c2a2e6bc8ca) commit c49473556746a0e7f0fe9b7a25e0a2cd056281d5 Author: Alfredo Deza Date: Tue Dec 5 10:21:20 2017 -0500 ceph-volume lvm.prepare fallback to bluestore when objectstore is not specified Signed-off-by: Alfredo Deza (cherry picked from commit 9b17ad2f8fe617d219fdb2f46dd2675669f32f08) commit 14e16a236ae6c5fad0f9e25a70a2c2d74eee5546 Author: Alfredo Deza Date: Tue Dec 5 10:11:03 2017 -0500 ceph-volume lvm.prepare rollback osd when prepare fails and an osd id was generated Signed-off-by: Alfredo Deza (cherry picked from commit 579d12806b2ffc693a4e70ba3eaf663fa281d35e) commit 2d23836024ee796f9212bb44ef9f1fb18d77eef4 Author: Alfredo Deza Date: Tue Dec 5 10:08:35 2017 -0500 ceph-volume lvm.common create a rollback_osd utility to cleanup failed osd prepare/create calls Signed-off-by: Alfredo Deza (cherry picked from commit e533792e0dbafc160238c0f5783f4430e2705571) commit 4e9092b1949d635599a1d47ab8ccedfa69d77db5 Merge: 041dd60315 26b7337583 Author: Jason Dillaman Date: Thu Feb 1 12:30:26 2018 -0500 Merge pull request #19483 from smithfarm/wip-22375-luminous luminous: build/ops: luminous build fails with --without-radosgw Reviewed-by: Jason Dillaman commit 041dd603153c5270ebeb4091b537ae88b5941f2d Merge: c727db5bb9 445583a922 Author: Jason Dillaman Date: Thu Feb 1 12:10:34 2018 -0500 Merge pull request #20211 from dillaman/wip-22198-luminous luminous: librbd: compare and write against a clone can result in failure Reviewed-by: Mykola Golub commit 445583a922c9eecf1eba60df4f9da49fb7bc8926 Author: Jason Dillaman Date: Wed Nov 15 08:24:50 2017 -0500 test/librbd: disable compare and write if skipping partial discard Signed-off-by: Jason Dillaman (cherry picked from commit 4e8b0b60a72fe1e1ac10a2beb46b77d4b58c91c4) commit e41f815e6b4e96ba7155202a2ce3e046370c0d56 Author: Jason Dillaman Date: Thu Nov 9 12:10:30 2017 -0500 librbd: refactor object write request state machines Fixes: http://tracker.ceph.com/issues/20789 Signed-off-by: Jason Dillaman (cherry picked from commit 0e643fb926f0484bb4f860740c7f85b692de6737) commit 11f21becff13e1f27f758500816cb9643cd010d0 Author: Jason Dillaman Date: Thu Nov 9 10:24:08 2017 -0500 librbd: consolidate all object discard-related logic into single state machine Signed-off-by: Jason Dillaman (cherry picked from commit 318797f59adcedcc386e01f1975011e0086434ac) commit 4f77368ab08df27417cc5c5f3c3200a5f68733ca Author: Jason Dillaman Date: Wed Nov 8 12:31:28 2017 -0500 librbd: simplify interface between object IO and copyup state machines The initial copyup was not receiving a write hint and the code for hints was duplicated multiple times. Additionally, the object map state should match the last executed write op. Signed-off-by: Jason Dillaman (cherry picked from commit 7117aa4ecaf0b619b527b2783fa1d09b11f3dd55) Conflicts: src/librbd/io/ObjectRequest.h: trivial resolution commit 7807083ea27ccb4b60dbc050e9d135464756fdb1 Author: Jason Dillaman Date: Thu Nov 9 15:15:58 2017 -0500 librbd: copyup state machine needs to handle empty write ops The compare-and-write object operation cannot be executed concurrently within a copyup operation since the object might not exist yet (if not performing a deep-copy). Signed-off-by: Jason Dillaman (cherry picked from commit f6db9b8027b6978b4d28fcf9f0389c88f9109e75) commit 487c41e71460610ff3833a6cc6f3c21ab345a2d4 Author: Jason Dillaman Date: Tue Nov 7 14:36:10 2017 -0500 librbd: object cacher should re-use read state machine This adds support for sparse-reads and ensures all object reads utilize a single, tested code path. Signed-off-by: Jason Dillaman (cherry picked from commit e79c0cf2009c69681ed4c70f52ade2f5fd570567) commit b12810056dd74d0b6367123a4a7be6599b93f727 Author: Jason Dillaman Date: Tue Nov 7 12:24:44 2017 -0500 librbd: refactor io::ObjectReadRequest Signed-off-by: Jason Dillaman (cherry picked from commit 62c3282b875f7345d7b71228d1e35e0cc87d44fa) commit f7b21684671b0d3df6db2bca9507921181a008e4 Author: Jason Dillaman Date: Tue Nov 7 14:17:25 2017 -0500 librbd: default template parameter for C_SparseReadRequest Signed-off-by: Jason Dillaman (cherry picked from commit ab0e558961ed9b63c00ad1d09ee0f9f79fd62295) commit 7b20cc9b519707e047bee6e6166daea232b35f5c Author: Jason Dillaman Date: Tue Nov 7 13:42:19 2017 -0500 librbd: reduce lock scope when assembling read results Signed-off-by: Jason Dillaman (cherry picked from commit 284feb6a32795c64b1c5ec3b3bf0d959e72dd940) commit ba3b6099945ae6e4d3781eee13027a6bcea2e61f Author: Jason Dillaman Date: Tue Nov 7 13:33:12 2017 -0500 librbd: ObjectReadRequest does not require buffer extents The ReadResult wrapper requires the buffer extents to know how to properly reconstruct the out buffer. Signed-off-by: Jason Dillaman (cherry picked from commit 35ce1c2d39302011adbf494ada6248495681a436) commit b77b7218c4087555997872661fb0d08f737406a4 Author: Jason Dillaman Date: Mon Nov 6 18:35:42 2017 -0500 librbd: templatize IO object request state machines Signed-off-by: Jason Dillaman (cherry picked from commit 654a78fa54520370f5466f9d282d4e9f8365ad48) commit 1fe2fe8d899930feecf6deee01d594691cab3f90 Author: Jason Dillaman Date: Fri Nov 10 21:45:13 2017 -0500 test/librados_test_stub: mocked remainder of IO ops used by librbd Signed-off-by: Jason Dillaman (cherry picked from commit c4e5613e4a74316ba9667654e887a835b240609a) commit 36f7a543252761f356aa60acc8d499142fed4966 Author: Jason Dillaman Date: Fri Nov 10 16:26:19 2017 -0500 test/librados_test_stub: simulate set_alloc_hint creating a non-existent file Signed-off-by: Jason Dillaman (cherry picked from commit 4b851ce0fa534ca043aca835057b9d68660971a4) commit 1581b9e3d3cd5f3c90e3883709d5db947501bd40 Author: Mykola Golub Date: Wed Aug 23 09:46:42 2017 +0200 test/librados_test_stub: pass snap context to zero op Signed-off-by: Mykola Golub (cherry picked from commit 78ca0fde22076e8b156feb8dfd60aaee316a1895) commit c727db5bb921bd68cbc7e23c9c49c50c401a8669 Merge: a3647582de 092e6cf29a Author: Jason Dillaman Date: Thu Feb 1 11:15:26 2018 -0500 Merge pull request #20209 from dillaman/wip-22033-luminous luminous: mgr/dashboard: added iSCSI IOPS/throughput metrics Reviewed-by: Mykola Golub commit a3647582deb09ead251562afa6e92d970473f564 Merge: 2f887e044e 436d89b9d5 Author: Jason Dillaman Date: Thu Feb 1 11:14:48 2018 -0500 Merge pull request #20210 from dillaman/wip-22169-luminous luminous: librbd: set deleted parent pointer to null Reviewed-by: Mykola Golub commit 2f887e044ec10bb018ab8d6284be1a367769fbaf Merge: 402b8e0885 c1c43110ef Author: Jason Dillaman Date: Thu Feb 1 11:13:36 2018 -0500 Merge pull request #20208 from dillaman/wip-21920-luminous luminous: librbd: default to sparse-reads for any IO operation over 64K Reviewed-by: Mykola Golub commit 402b8e08854c3436f92b9c8af01a429caee79da8 Merge: d2dbebf90b c22e49f321 Author: Jason Dillaman Date: Thu Feb 1 11:12:42 2018 -0500 Merge pull request #20207 from dillaman/wip-21793-luminous luminous: rbd-mirror: primary image should register in remote, non-primary image's journal Reviewed-by: Mykola Golub commit d2dbebf90b52ec4b23bcb1dd7bcd49e49c6414d9 Merge: eaedbcd0e1 1a0e9498a6 Author: Jason Dillaman Date: Thu Feb 1 11:12:11 2018 -0500 Merge pull request #20206 from dillaman/wip-21694-luminous luminous: librbd: journal should ignore -EILSEQ errors from compare-and-write Reviewed-by: Mykola Golub commit eaedbcd0e11ac833c833840a8a132d16f6f39cba Merge: 672ef6752b 2691ada5ac Author: Jason Dillaman Date: Thu Feb 1 11:11:39 2018 -0500 Merge pull request #20205 from dillaman/wip-22577-luminous luminous: qa/workunits/rbd: simplify split-brain test to avoid potential race Reviewed-by: Mykola Golub commit 672ef6752bc1dd754cc74dabb0dd64b0ac5096f6 Merge: 381c044d77 cfd05be6ac Author: Jason Dillaman Date: Thu Feb 1 11:11:04 2018 -0500 Merge pull request #20153 from pdvian/wip-22809-luminous luminous: librbd: fix snap create/rm may taking long time Reviewed-by: Jason Dillaman commit 381c044d7726983a459103bcd0b29f08ea0944c5 Merge: b272904f57 37067a6f12 Author: Jason Dillaman Date: Thu Feb 1 11:09:15 2018 -0500 Merge pull request #20135 from dillaman/wip-22806-luminous luminous: librbd: force removal of a snapshot cannot ignore dependent children Reviewed-by: Mykola Golub commit b272904f57eb1daa8769737bed268fccb4c9070f Merge: cf8fd69d16 2e69cd92d0 Author: Jason Dillaman Date: Thu Feb 1 11:00:25 2018 -0500 Merge pull request #19123 from pdvian/wip-22174-luminous luminous: possible deadlock in various maintenance operations Reviewed-by: Jason Dillaman commit f2f00cac71c5ae1b6b19706b5a77dfa1bbf11ff7 Author: Gu Zhongyan Date: Fri Jan 26 16:19:50 2018 +0800 common/pick_address: wrong prefix_len in pick_iface() With prefix_len initialized as zero, mask in netmask_ipv4/6() will always be zero, so find_ip_in_subnet() always return the first interface. Set prefix_len to the right value to fetch the right interface. Signed-off-by: Gu Zhongyan guzhongyan@360.cn Signed-off-by: yupeng chen chenyupeng-it@360.cn (cherry picked from commit b0d8043a72af22648c058382c46a132264736aaa) commit 06a1c3d1e4cdb0f267dfc1189534ef747ac38521 Author: Yan, Zheng Date: Mon Nov 20 08:29:12 2017 +0800 mds: don't delay processing completed requests in replay queue Completed requests can also be in replay queue, they need to be processed (treated as lookup request) in clientreplay stage. MDS will be stuck at clientreplay stage if it delays processing requests of this type. the bug was introduced by commit 0afbc033 (mds: don't rdlock locks in replica object while auth mds is recovering) Fixes: http://tracker.ceph.com/issues/22163 Signed-off-by: "Yan, Zheng" (cherry picked from commit 959fad28b2a1b6ad1e5f8597d04ceb9b1761b98a) commit f454c293d178f108fa0d795d8f2d4487942c71b7 Author: Theofilos Mouratidis Date: Thu Dec 14 17:43:59 2017 +0100 ceph-volume: Require lvm2, move to osd package Fixes: http://tracker.ceph.com/issues/22443 Signed-off-by: Theofilos Mouratidis (cherry picked from commit 02bc369e052125f50c7d3a7fe9b311215291c84d) commit 93f13ab24cf2c54a183e90e23a4bddb4fe028da3 Author: Nathan Cutler Date: Tue Nov 21 21:32:57 2017 +0100 build/ops: rpm: fix systemd macros for ceph-volume@.service Fixes: http://tracker.ceph.com/issues/22217 Signed-off-by: Nathan Cutler (cherry picked from commit d1f2df37dd03d77132e711423aacda46768d1e02) commit e621e9d7f1447f796b3f80475e38bc7de3f9b6a6 Author: David Zafman Date: Fri Dec 8 18:48:21 2017 -0800 test: ceph_objectstore_tool.py: Perform dump-import Signed-off-by: David Zafman (cherry picked from commit c4602c9ac8f7819abc89c8668fbdc4572341ac5e) commit 24b0482add051f0e0bf5e40490a001540b29e074 Author: David Zafman Date: Tue Dec 5 20:57:47 2017 -0800 ceph-objectstore-tool: Add option "dump-import" to examine an export Fixes: http://tracker.ceph.com/issues/22086 Signed-off-by: David Zafman (cherry picked from commit a8b8d541ddcc8085a9e7e301948665f3d6c17f0c) Conflicts: (trivial resolution: list of subcommands is slightly different in luminous, compared to master) qa/standalone/special/ceph_objectstore_tool.py src/tools/ceph_objectstore_tool.cc commit a5282f55df1fcc55ad7ec140bde87001f2145d58 Author: David Zafman Date: Tue Dec 5 21:04:16 2017 -0800 ceph-objectstore-tool: Remove unused map section from log dump Caused by: 44cb6d30640954c66fc03d4ce58b583ffd614951 Signed-off-by: David Zafman (cherry picked from commit b20eeada312c95713cf2ea032fd7437d2b282558) commit 99d740322e231b3c1ddc8a8c6517baca71c14a01 Author: David Zafman Date: Tue Dec 5 18:25:19 2017 -0800 ceph-objectstore-tool: Fix output of section type number Signed-off-by: David Zafman (cherry picked from commit 508328784516136fe189581bfbfe5a791db8e9fd) commit faef2777ffe0c6548616f1691c72470096e96fb4 Author: David Zafman Date: Tue Dec 5 17:53:01 2017 -0800 ceph-objectstore-tool: Improve ceph-objectstore-tool usage output Remove unused argument test-align Caused by: 1c8731c31eb53e17d8f363906438963ab5688fe0 Positional arguments are described by usage() because the parsing code creates descriptions like "--object" for the object positional argument. Signed-off-by: David Zafman (cherry picked from commit c6896cccc35bc9a159bbe8c3bbb6a0c9ab08fcac) commit cf8fd69d16b786caa178ca412c509425766856b4 Merge: 427bad3aae dbf54df879 Author: Yuri Weinstein Date: Wed Jan 31 14:10:45 2018 -0800 Merge pull request #17875 from smithfarm/wip-21359-luminous luminous: cephfs: racy is_mounted() checks in libcephfs Reviewed-by: Patrick Donnelly commit cf6799e4b5323641502e6da9f546c9a6cee3cb21 Author: Abhishek Lekshmanan Date: Wed Jan 31 22:57:42 2018 +0100 client: quit on failed remount during dentry invalidate test Fixes: http://tracker.ceph.com/issues/22269 Signed-off-by: Patrick Donnelly (cherry picked from commit 5b2b1d14468c290c56ee6c95ea557c99464e0098) Conflicts: PendingReleaseNotes: trivial conflict src/ceph_fuse.cc src/client/Client.cc: i chosed to pick code from 5b2b1d1 because, to fixthe issue, we need to call _do_remount which was introduced in 5b2b1d1. commit 427bad3aae9b427fe7308e91e965e57a29964918 Merge: 9e0eea2be2 d4f5dc2780 Author: Yuri Weinstein Date: Wed Jan 31 13:21:50 2018 -0800 Merge pull request #20121 from batrick/i21252p2 luminous: mds: fix return value of MDCache::dump_cache Reviewed-by: Patrick Donnelly commit 5dcd2a59770887a3560d24712904c57f39dd3742 Author: John Spray Date: Wed Oct 18 11:34:53 2017 +0100 mon: s/mds_blacklist_interval/mon_mds_blacklist_interval/ We can rename LEVEL_DEV options at will. Signed-off-by: John Spray (cherry picked from commit ed7fc5370c45deb4c80a36e9166e6dd84d4a2b99) commit d8f9aec74259620bafcde7fa673eea52777d8348 Author: John Spray Date: Tue Oct 17 23:06:07 2017 +0100 mon: tidy up mds_blacklist_interval Remove the misleading reference to this from the client eviction page, it was never the right option to mention there (my mistake). Demote the option from LEVEL_ADVANCED to LEVEL_DEV as it is hard to imagine a good reason for the user to change it. Set a hard minimum of one hour, to make it harder to corrupt` a system by setting it close to zero. Remove the legacy definition of the field while we're at it. Fixes: http://tracker.ceph.com/issues/21821 Signed-off-by: John Spray (cherry picked from commit 0d68d3513e03cc2400facf421e08c8a92cb2886a) Conflicts: doc/cephfs/eviction.rst (trivial resolution of conflict resulting from 0e43f0f01cd800fee4cd800f1545405b449fa55b being merged to luminous) commit 9e0eea2be2d135c3c55fb402c64db171a70022a1 Merge: 6ef48aeb55 075e3b7506 Author: Jason Dillaman Date: Wed Jan 31 13:27:40 2018 -0500 Merge pull request #20213 from dillaman/wip-21868-luminous luminous: doc/rbd: tweaks for the LIO iSCSI gateway Reviewed-by: Jason Dillaman commit 075e3b75063b0aaf04adb9f451651ab2290c0d15 Author: Jason Dillaman Date: Fri Nov 3 10:09:38 2017 -0400 doc/rbd: tweaks to the Windows iSCSI initiator directions Signed-off-by: Jason Dillaman (cherry picked from commit 997bb80ba231c4a21237485ac6780d573be992c2) commit ba26353aae43cf01d9ee66e2d23d8c0ba9a0c9d9 Author: Ashish Singh Date: Fri Oct 20 20:45:35 2017 +0530 doc: Added CHAP cofiguration instructions for iSCSI Added details to specify the CHAP username and password while discovering/login the iSCSI target. Signed-off-by: Ashish Singh (cherry picked from commit 2a5755e0767108aefa6ccfb8a85e5d63e8c04fe8) commit a3101dd478baedbc701d548c2752bdba0556f4a2 Author: Mike Christie Date: Mon Jan 22 17:08:55 2018 -0600 doc: fix ceph-iscsi-config version number Signed-off-by: Mike Christie (cherry picked from commit 3ab71020a5e420a333cb4a6b0be67ade185c1593) commit 3560c6de1892998cf848f799f312fea258dc8b01 Author: Mike Christie Date: Sat Jan 20 16:06:42 2018 -0600 doc: add ceph iscsi test kernel info Signed-off-by: Mike Christie (cherry picked from commit 5c30318660eebc882326d87eec39b4fbe4cc85d7) commit 4bc150c8280fe94f1c4525d2e58ba55cd1229e82 Author: Mike Christie Date: Tue Jan 16 12:46:09 2018 -0600 doc: add rbd pool requirement for gwcli Signed-off-by: Mike Christie (cherry picked from commit b56d994f3addcd9c6a7848dec9be93568cdb2d0e) commit 30c93cdb409e64baa9f11f4e23717c5529f09b20 Author: Mike Christie Date: Mon Jan 15 12:59:37 2018 -0600 doc: add daemon-reload for rbd-target-api setup Signed-off-by: Mike Christie (cherry picked from commit 4ca0d508bb9b68c791abc5799d0195f4baace455) commit 46bf788c310031739016cf07a5be8c23170e6f90 Author: Mike Christie Date: Tue Jan 16 12:04:28 2018 -0600 doc: add gwcli manual install instructions Signed-off-by: Mike Christie (cherry picked from commit f45aafed1c7a5393e712f9fbfd7479fc444ad3d8) commit efdb585141bfebfaab38dc0049905c6c3a5baae1 Author: Mike Christie Date: Thu Jan 11 14:54:06 2018 -0600 doc: separate gwcli install from post install setup steps Signed-off-by: Mike Christie (cherry picked from commit f80f083d0a982d7e89c77e3a6f8d71c2871a8f2c) commit f399a3399cb210ef76fa17c8a31178a2327aaa25 Author: Mike Christie Date: Thu Jan 11 14:53:16 2018 -0600 doc: add note to clarify trusted_ip_list use Signed-off-by: Mike Christie (cherry picked from commit 3cc367dc6ebca18e884d7c75d2cd1fa7907b7139) commit 2eda81b9245c340be76decd4728ccb5a3d950ac9 Author: Mike Christie Date: Wed Jan 10 22:14:18 2018 -0600 doc: update esx instructions Signed-off-by: Mike Christie (cherry picked from commit 3806f9750d29c6d930a7c8a003902e79b99766f6) Conflicts: doc/rbd/iscsi-initiator-esx.rst: trivial resolution commit f4b1f3974b26b52d0cfd54f6c1237bccf5cb9921 Author: Mike Christie Date: Wed Jan 10 20:33:03 2018 -0600 doc: add warning about SCSI PGRs/reservations Signed-off-by: Mike Christie (cherry picked from commit 62661a2e23ce98432f3c037fa9a8155a633ee182) commit 723d5fabfabfff2ad517f67a8fd1bdf1c2380bfc Author: Mike Christie Date: Wed Jan 10 18:51:51 2018 -0600 doc: add esx web based client images Signed-off-by: Mike Christie (cherry picked from commit 78e12135eeb5829cad65cebd3c2be9df4d582b1f) commit 0f937ed3d67103e5874bd8862d36ab53177f4677 Author: Mike Christie Date: Thu Jan 11 19:36:06 2018 -0600 doc: remove windows based vsphere client images Signed-off-by: Mike Christie (cherry picked from commit 426b562fdc1b373b72de2d69c4851be22ec12008) commit f8ab982b72f2a8f27bc034945685dc55ef9222bf Author: Mike Christie Date: Thu Jan 11 19:35:22 2018 -0600 doc: fix gwcli examples Signed-off-by: Mike Christie (cherry picked from commit 480f6a82a2b9dcd0179951ea16432d8d32f2c96d) commit 335fa90e0db06a2b6b74ac756f0d935ea4e67648 Author: Mike Christie Date: Wed Jan 10 16:38:13 2018 -0600 doc: Update ceph iscsi kernel/distro requirements. Signed-off-by: Mike Christie (cherry picked from commit f436e993d359dfdad26b7d569d555a3b3a5b798d) commit 0363593523c1b95233a23529ae1a8910088f0fd2 Author: Jason Dillaman Date: Wed Oct 11 12:02:55 2017 -0400 doc/rbd: tweaks for the LIO iSCSI gateway Fixes: http://tracker.ceph.com/issues/21763 Signed-off-by: Jason Dillaman (cherry picked from commit 4f227e108d40271d5d6b2b1e88659d86cbc6c914) commit 092e6cf29a33d49ed1abddb2558695bd6de4ae5b Author: Jason Dillaman Date: Tue Oct 31 11:05:47 2017 -0400 mgr/dashboard: added iSCSI IOPS/throughput metrics Fixes: http://tracker.ceph.com/issues/21391 Signed-off-by: Jason Dillaman (cherry picked from commit e55ef24c9140c23c832a2205a2154e68452545a3) commit 451ae9ed812481aedb372ddb97d0e01e1c711ca7 Author: Jason Dillaman Date: Tue Oct 31 10:41:36 2017 -0400 mgr/dashboard: include A/O start relative timestamp for iSCSI Signed-off-by: Jason Dillaman (cherry picked from commit e62f186613a1de64a155a852b21ec9033c6e73ea) commit 076ee1a247a6a65e88f2b62b1742be3197f81387 Author: Jason Dillaman Date: Tue Oct 17 15:40:55 2017 -0400 librbd: track image open and lock acquire time via perf counter Signed-off-by: Jason Dillaman (cherry picked from commit 8583270fd2d0930c1fccd2285378de0844b1d928) commit 30ff1fbec70cb7140ffc66a4cb25060795550fea Author: Jason Dillaman Date: Tue Oct 17 15:21:42 2017 -0400 librbd: export read and writes performance counters for daemons Signed-off-by: Jason Dillaman (cherry picked from commit 7a9d10a4e61ed168678f38f6a224859e0df44099) commit fad78966d49f01b6c3d3fa3aba20f314d1e758b6 Author: Jason Dillaman Date: Tue Oct 17 15:49:22 2017 -0400 librbd: moved performance counter enums to common location Signed-off-by: Jason Dillaman (cherry picked from commit 2224e5ca1dacef1e3f8ec4854f08b4abe86f5d75) Conflicts: src/librbd/internal.h: trivial resolution commit 436d89b9d538b7521993b0fd6d0aeca253827775 Author: Jason Dillaman Date: Sat Nov 18 08:30:27 2017 -0500 librbd: set deleted parent pointer to null Fixes: http://tracker.ceph.com/issues/22158 Signed-off-by: Jason Dillaman (cherry picked from commit 7b53256bb825383fc604a96d71bcd51c36668cc4) commit b8b3eee73d79345e28404281040f7bdd3c36d171 Author: Kefu Chai Date: Thu Nov 23 12:56:31 2017 +0800 qa: silence SLOW_OPS,PENDING_CREATING_PGS warnings this is an intermediate step to deprecate REQUEST_SLOW warnings. Signed-off-by: Kefu Chai (cherry picked from commit 4a1f2a5c78cc77eafe99f087d10c0d3daa7fca7b) Conflicts: qa/suites/rados/singleton/all/thrash-eio.yaml qa/suites/smoke/basic/tasks/mon_thrash.yaml qa/suites/smoke/basic/tasks/rados_bench.yaml qa/suites/smoke/basic/tasks/rados_cache_snaps.yaml qa/suites/smoke/basic/tasks/rados_ec_snaps.yaml qa/suites/smoke/basic/tasks/rbd_fsx.yaml: do not cherry-pick the SLOW_OPS related changes. commit 030d7e8c840386c366ca794ec86e14ddab44d6d5 Author: Kefu Chai Date: Wed Nov 1 23:17:57 2017 +0800 mgr: summarize osd metrics in MMgrReport and sent it to mon Signed-off-by: Kefu Chai (cherry picked from commit 7e7978732d20c506eca581f2b153ede70ceefa3d) Conflicts: src/mgr/OSDHealthMetricCollector.cc: only the change reporting pending-creating-pgs is cherry-picked. commit 687967459b0a929fb33cc689401475c6354ab391 Author: Kefu Chai Date: Thu Nov 9 21:37:11 2017 +0800 mon/health_check: s/std::list/list/ so it's self-contained, and .cc files including it are not obliged to `using namespace std`. Signed-off-by: Kefu Chai (cherry picked from commit 416082c1ce6c357943f4a6b9e081a3211b1110c0) commit bb5ee06baac34aff59578575e14fe2a6bb6ae0ab Author: Kefu Chai Date: Mon Oct 23 12:29:31 2017 +0800 osd: send health-checks to mgr Signed-off-by: Kefu Chai (cherry picked from commit f4b74125e44fe78154fb377fa06fc08b3325859d) Conflicts: src/osd/OSD.cc src/osd/OSD.h: only the changes related to reporting pending-creating-pgs are cherry-picked. because we want to minimize the impact to luminous. and to remove the slow ops from cluster log is not in the scope of this backport. also, with_unique_lock was introduced after luminous is branched, so use the plain lock_guard<> instead. src/osd/OSDHealthMetric.h: because denc.h does not support denc of enum types in luminous. so we need to case from/to osd_metric when necessary. commit 2472ec3b956b104717fa23ca5d0dc29cfbc43955 Author: Jason Dillaman Date: Tue Oct 17 14:57:08 2017 -0400 librbd: track the child of an image in-memory Signed-off-by: Jason Dillaman (cherry picked from commit ce2ae1d7c0f8818c86cbc746921e8eb6b0f89913) commit c22e49f3216c44bc26e029b05602a4b06e92c49d Author: Jason Dillaman Date: Fri Oct 27 16:02:47 2017 -0400 rbd-mirror: journal debug message should not be logged as error Signed-off-by: Jason Dillaman (cherry picked from commit 625222edbce66c98b86fc87fa3bd12ea40db17a2) commit 08ed96a7ddc1e587854a7da7edd6c350cc0a1fe5 Author: Jason Dillaman Date: Fri Oct 27 16:02:20 2017 -0400 rbd-mirror: avoid attempting to decode an empty client data buffer Fixes: http://tracker.ceph.com/issues/21961 Signed-off-by: Jason Dillaman (cherry picked from commit 1e1d15eb0c14a6be82a6d75ab0088a6f0f4f8125) commit c1c43110ef683b7a1f8dbb7136ad44b31943fe5a Author: Jason Dillaman Date: Thu Oct 19 13:52:56 2017 -0400 librbd: default to sparse-reads for any IO operation over 64K Testing BlueStore against both HDDs and OSDs with fully allocated and sparse-allocated objects shows a performance improvement with sparse-read between 32K and 64K. Fixes: http://tracker.ceph.com/issues/21849 Signed-off-by: Jason Dillaman (cherry picked from commit 251658471eab6d8cd968d678922bab437f72a9c7) commit cec71d94ffd0b86caee5524d2346b1d66f19869e Author: Jason Dillaman Date: Thu Oct 19 14:32:05 2017 -0400 librbd: templatize io::CopyupRequest and io::ObjectRequest Signed-off-by: Jason Dillaman (cherry picked from commit dd5452004047c31e0f3eb813d5dd0b50bd68e76f) commit ba74ae952cbc032689833ab60bb8c59df99a675e Author: Jason Dillaman Date: Thu Oct 5 16:38:32 2017 -0400 rbd-mirror: removed duplicate client registration logic The client will now be registered before the bootstrap state machine is invoked. Fixes: http://tracker.ceph.com/issues/21561 Signed-off-by: Jason Dillaman (cherry picked from commit d774d4702fbb9ff0b88797392b6673964d6b8c7c) commit f7d43cb0dc690dd0210090276514eaf257aa22d4 Author: Jason Dillaman Date: Thu Oct 5 15:56:19 2017 -0400 rbd-mirror: local primary images should still register to remote images Fixes: http://tracker.ceph.com/issues/21561 Signed-off-by: Jason Dillaman (cherry picked from commit d74b95dc10a5073ee7980583d94628ded7850ebb) commit 1a0e9498a6bd32987051fb02b3264bdb212a6310 Author: Jason Dillaman Date: Tue Oct 3 15:42:50 2017 -0400 librbd: journal should ignore -EILSEQ errors from compare-and-write Fixes: http://tracker.ceph.com/issues/21628 Signed-off-by: Jason Dillaman (cherry picked from commit c7a9b045808ba741f6aadd2c2e65bd84135347c4) commit 2691ada5ac3179d874381b44c1482c3dc85cce31 Author: Jason Dillaman Date: Tue Dec 19 16:29:18 2017 -0500 qa/workunits/rbd: simplify split-brain test to avoid potential race Fixes: http://tracker.ceph.com/issues/22485 Signed-off-by: Jason Dillaman (cherry picked from commit 8c2ee6dd0f4c77b9fbbc24a673841b04d3e175d6) Conflicts: qa/workunits/rbd/rbd_mirror.sh: trivial resolution commit 69e8a0ab28c53303030423ac795577564827dbaa Author: Kefu Chai Date: Sun Oct 29 19:07:54 2017 +0800 osd: remember is_primary when adding pending creating pgs so mgr can avoid calculating the same PG from different OSDs. Signed-off-by: Kefu Chai (cherry picked from commit 486515ae884b91571a9066296680fff78f93f66d) commit 530283690eb8b5016ad5ed92f9f44cdcb977e847 Author: Kefu Chai Date: Tue Oct 24 22:41:30 2017 +0800 mgr/MgrClient: send health_checks using MMgrReport Signed-off-by: Kefu Chai (cherry picked from commit 6fed1d21be1adf1464b5834787c95fae205a91b5) commit 6ef48aeb555790a6d5a8ef025a7ecf84dfe07de3 Merge: 0489bddb5d bb3d091b46 Author: John Spray Date: Wed Jan 31 12:09:08 2018 +0000 Merge pull request #19555 from shinobu-x/wip-22455-luminous luminous: balancer crush-compat sends "foo" command Reviewed-by: John Spray commit 0489bddb5db768d4034bbcfdbd75151cc1df8278 Merge: 8bbf1653ff f2a202775f Author: John Spray Date: Wed Jan 31 12:06:59 2018 +0000 Merge pull request #20156 from pdvian/wip-22811-luminous luminous: mon: fix mgr using auth_client_required policy Reviewed-by: John Spray commit 8bbf1653ff4fabdac6bc8e79c9913fed5c2727ab Merge: 1e3229418a 0aa3c5c577 Author: John Spray Date: Wed Jan 31 12:01:45 2018 +0000 Merge pull request #19481 from jan--f/wip-22421-luminous luminous: mon: reenable timer to send digest when paxos is temporarily inactive Reviewed-by: John Spray commit 1e3229418aa47cc1648121134e18d8f491c6cdac Merge: 2d64d41a39 c5cb0cab9f Author: John Spray Date: Wed Jan 31 11:59:04 2018 +0000 Merge pull request #19553 from shinobu-x/wip-22453-luminous luminous: mgr/balancer/upmap_max_iterations must be cast to integer Reviewed-by: John Spray commit 2d64d41a394b8dea69436c9b9c06d2a8aa3466f1 Merge: d351505965 40954a9c27 Author: John Spray Date: Wed Jan 31 11:49:41 2018 +0000 Merge pull request #19624 from shinobu-x/wip-22496-luminous luminous: KeyError: ('name',) in balancer rm Reviewed-by: John Spray commit d351505965654e1f8c4525ca7e9cfebd308828fc Merge: ea85e979b2 b5892a13b9 Author: John Spray Date: Wed Jan 31 11:48:50 2018 +0000 Merge pull request #19834 from pdvian/wip-22585-luminous luminous: pybind/mgr/prometheus: fix metric type undef -> untyped Reviewed-by: John Spray commit ea85e979b239127ee80373949dfa88a9c317d03f Merge: 81712618b4 1b2b589a12 Author: John Spray Date: Wed Jan 31 11:47:05 2018 +0000 Merge pull request #20084 from pdvian/wip-22760-luminous luminous: mgr: prometheus: added osd commit/apply latency metrics (#22718). Reviewed-by: John Spray commit 81712618b4ce36511674bdfb0720b4d7bb454c06 Merge: fab5aa92cb bb21f23731 Author: John Spray Date: Wed Jan 31 11:46:03 2018 +0000 Merge pull request #20089 from pdvian/wip-22768-luminous luminous: mgr: disconnect unregistered service daemon when report received Reviewed-by: John Spray commit fab5aa92cb2883b2aab88ffec769a57cc36fe115 Merge: 7eb2423bef 91bfcc73f6 Author: John Spray Date: Wed Jan 31 11:42:58 2018 +0000 Merge pull request #20182 from jcsp/wip-22831 luminous: pybind/mgr/dashboard: fix reverse proxy support Reviewed-by: John Spray commit 7eb2423befadb1bd04ba68ebdbb7a4734daff9e2 Merge: 3512fc40cb 52e59854c2 Author: Yuri Weinstein Date: Tue Jan 30 16:47:21 2018 -0800 Merge pull request #17869 from smithfarm/wip-21479-luminous luminous: mgr: Services reported with blank hostname Reviewed-by: John Spray Reviewed-by: Amit Kumar commit 3512fc40cb75f97ca66df1682c5873c5110a3b78 Merge: 25cf00a740 4db6b35548 Author: Yuri Weinstein Date: Tue Jan 30 16:46:34 2018 -0800 Merge pull request #18832 from shinobu-x/wip-22075-luminous luminous: mgr tests don't indicate failure if exception thrown from serve() Reviewed-by: John Spray commit 25cf00a740573d329a35cd050b2723070d8e0bad Merge: ef1038691d eeb12253d4 Author: Yuri Weinstein Date: Tue Jan 30 16:46:04 2018 -0800 Merge pull request #18842 from shinobu-x/wip-21863-luminous luminous: ceph-conf: dump parsed config in plain text or as json Reviewed-by: Kefu Chai commit ef1038691da028df20b4e22a6a628bf3e6a72586 Merge: a3091bdd20 de1d25c2dd Author: Yuri Weinstein Date: Tue Jan 30 16:45:40 2018 -0800 Merge pull request #19002 from tchaikov/wip-pr-18976-luminous luminous: udev: Fix typo in udev OSD rules file Reviewed-by: Nathan Cutler commit a3091bdd20a64c02dc423052015e46d42240adf3 Merge: 5563a65058 1bdd81e095 Author: Yuri Weinstein Date: Tue Jan 30 16:44:46 2018 -0800 Merge pull request #19023 from pdvian/wip-22164-luminous luminous: cluster [ERR] Unhandled exception from module 'balancer' while running on mgr.x: 'NoneType' object has no attribute 'iteritems'" in cluster log Reviewed-by: John Spray commit 5563a650587375640ecefca6643570d5366ce420 Merge: b0b09df8aa c21e1c7831 Author: Yuri Weinstein Date: Tue Jan 30 16:44:16 2018 -0800 Merge pull request #19031 from pdvian/wip-22167-luminous luminous: Various odd clog messages for mons Reviewed-by: Kefu Chai Reviewed-by: John Spray commit b0b09df8aa5150fcad9bfd2f200bd74b71f5faaf Merge: a35101135f 6f983c14d9 Author: Yuri Weinstein Date: Tue Jan 30 16:42:52 2018 -0800 Merge pull request #19039 from jan--f/wip-22199-luminous luminous: tools/crushtool: skip device id if no name exists Reviewed-by: Kefu Chai Reviewed-by: Amit Kumar Reviewed-by: Nathan Cutler commit a35101135f6fd8bb83e4a44baf7abe9e2c5b83d6 Merge: 5f175b5bea 488228e919 Author: Yuri Weinstein Date: Tue Jan 30 16:41:50 2018 -0800 Merge pull request #19046 from shinobu-x/wip-22189-luminous luminous: osdc/Objecter: objecter op_send_bytes perf counter always 0 Reviewed-by: Kefu Chai commit 5f175b5bea82c848a7c2bc9aef4860e5e8672310 Merge: a1636c348e d5f2e16600 Author: Yuri Weinstein Date: Tue Jan 30 16:41:25 2018 -0800 Merge pull request #19059 from pdvian/wip-22176-luminous luminous: osd: subscribe osdmaps if any pending pgs Reviewed-by: Kefu Chai commit a1636c348ebb1e8ec8a79e60cc22193701da84c2 Merge: 5c79793917 540f425f1a Author: Yuri Weinstein Date: Tue Jan 30 16:40:44 2018 -0800 Merge pull request #19083 from smithfarm/wip-22213-luminous luminous: On pg repair the primary is not favored as was intended Reviewed-by: David Zafman commit 5c7979391775529b5650afe18b25315bfc1e2c4a Merge: 525fe577b8 ed24c8ce8f Author: Yuri Weinstein Date: Tue Jan 30 16:40:04 2018 -0800 Merge pull request #19084 from shinobu-x/wip-22216-luminous luminous: "osd status" command exception if OSD not in pgmap stats Reviewed-by: John Spray commit 525fe577b803552eae568d765f34b4e9c98ea8c7 Merge: 91d1a7bed8 8f67e4553a Author: Yuri Weinstein Date: Tue Jan 30 16:39:36 2018 -0800 Merge pull request #19118 from tangwenjun3/wip-backport-19030 luminous: os/bluestore: fix the allocate in bluefs Reviewed-by: Sage Weil Reviewed-by: xie xingguo Reviewed-by: Shinobu Kinjo commit 91d1a7bed8254f54290ca089ccc39a41a5da962e Merge: 2c2e60681d bdb9d385a5 Author: Yuri Weinstein Date: Tue Jan 30 16:33:43 2018 -0800 Merge pull request #19133 from smithfarm/wip-22194-luminous luminous: Default kernel.pid_max is easily exceeded during recovery on high OSD-count system Reviewed-by: David Disseldorp Reviewed-by: Kefu Chai Reviewed-by: Amit Kumar commit 2c2e60681d93b0aa3b1b8d8da8d1f02b12bdc8ed Merge: 3937ae421a 7ffe8c41d3 Author: Yuri Weinstein Date: Tue Jan 30 16:30:53 2018 -0800 Merge pull request #19631 from shinobu-x/wip-22509-luminous luminous: osd: "sudo cp /var/lib/ceph/osd/ceph-0/fsid ..." fails Reviewed-by: Sage Weil commit 3937ae421a47d781af11860788f999fcaaeeade4 Merge: 74064cb9e4 0a347506af Author: Yuri Weinstein Date: Tue Jan 30 16:30:11 2018 -0800 Merge pull request #19839 from tchaikov/wip-22558-luminous luminous: rpm: adjust ceph-{osdomap,kvstore,monstore}-tool feature move Reviewed-by: Nathan Cutler commit 74064cb9e4d2731919374a28c030cda034d24907 Merge: f5dcf47a10 c86675dadc Author: Yuri Weinstein Date: Tue Jan 30 16:28:05 2018 -0800 Merge pull request #18838 from shinobu-x/wip-21973-luminous luminous: [test] UpdateFeatures RPC message should be included in test_notify.py Reviewed-by: Jason Dillaman commit f5dcf47a109c52bb4db7385d9ae6fd9ae53a5102 Merge: 40ca9d22de 8cb21e6ea6 Author: Yuri Weinstein Date: Tue Jan 30 16:27:34 2018 -0800 Merge pull request #18840 from shinobu-x/wip-21970-luminous luminous: [journal] tags are not being expired if no other clients are registered Reviewed-by: Jason Dillaman commit 40ca9d22de212ac111cd1d744106e21aa13d5615 Merge: f7778d4700 74a6592d7f Author: Yuri Weinstein Date: Tue Jan 30 16:26:21 2018 -0800 Merge pull request #18949 from jcsp/wip-luminous-rgw-config-desc luminous: options.cc: document rgw config options Reviewed-by: Casey Bodley commit f7778d4700e654c62b49158b30c541d6bf879523 Merge: c421d533a2 2d9aafe1bc Author: Yuri Weinstein Date: Tue Jan 30 16:25:18 2018 -0800 Merge pull request #19051 from shinobu-x/wip-22185-luminous luminous: rbd: abort in listing mapped nbd devices when running in a container Reviewed-by: Jason Dillaman commit 91bfcc73f68a869703eb2f6a5ee5f5962b9b727b Author: Nick Erdmann Date: Mon Jan 22 12:33:32 2018 +0000 pybind/mgr/dashboard: fix reverse proxy support This fixes http redirection for reverse http proxies Fixes: http://tracker.ceph.com/issues/22557 Signed-off-by: Nick Erdmann (cherry picked from commit 95e1963cb5327f0699081c4c4b0b355d109ff0e3) commit f0cae0bffced632fcc9f0dfec5d1c85424d9a42c Author: Nick Erdmann Date: Fri Jan 26 13:40:27 2018 +0000 pybind/mgr: fix typo Signed-off-by: Nick Erdmann (cherry picked from commit fdd64bb2dac33085b47c737032a525d89319c866) commit ba2c6015ebe0d2fd55e38f1a1472810551e8d0c1 Author: Kefu Chai Date: Mon Jan 15 11:49:08 2018 +0800 pybind/mgr/mgr_module: add default param for MgrStandbyModule.get_config() this matches its counterpart of MgrModule. Signed-off-by: Kefu Chai (cherry picked from commit bc7de8b124c32a2ee6910a3bac6a06f1c566a2ac) commit c421d533a258c8950205f59a032a3bdf9fd298fb Merge: d146d83076 660abda211 Author: Casey Bodley Date: Tue Jan 30 09:29:15 2018 -0500 Merge pull request #20062 from cbodley/wip-luminous-rgw-bilog-trim luminous: rgw multisite: automated trimming for bucket index logs Reviewed-by: Orit Wasserman commit 2e69cd92d07ca7f7957f67d40d46a04b94ccb0b4 Author: Jason Dillaman Date: Tue Nov 14 12:07:36 2017 -0500 librbd: added missing locks for snap rollback and rename Signed-off-by: Jason Dillaman (cherry picked from commit 0ccd26f9989b892be2ab620bcf7f4fbf14980e8e) commit 7df6f0602cef58ee9185814ca3363d5448bb739d Author: Jason Dillaman Date: Mon Nov 13 13:28:06 2017 -0500 librbd: possible deadlock with synchronous maintenance operations Fixes: http://tracker.ceph.com/issues/22120 Signed-off-by: Jason Dillaman (cherry picked from commit 90b7ecd8a8aac9a5c282d44004752ade0c195331) commit d146d830762414934375afe257c69a25f4e533f3 Merge: 20cfb9bda3 67fbd3830f Author: Sage Weil Date: Mon Jan 29 10:22:14 2018 -0600 Merge pull request #19848 from cbodley/wip-luminous-boost-166 luminous: backport beast frontend and boost 1.66 update commit 20cfb9bda35dea6ccd83f526a88a38edef210412 Merge: 9981b51339 7dabbe9131 Author: Sage Weil Date: Mon Jan 29 10:21:07 2018 -0600 Merge pull request #19500 from shinobu-x/wip-22401-luminous luminous: rgw: make HTTP dechunking compatible with Amazon S3 Reviewed-by: Casey Bodley commit 9981b513395e261b472ff516b17df419cfec714d Merge: abeb3c56c0 fc791b980d Author: Yuri Weinstein Date: Mon Jan 29 08:12:39 2018 -0800 Merge pull request #17816 from smithfarm/wip-hadoop-s3a luminous: tests: qa/rgw: hadoop-s3a suite targets centos_latest Reviewed-by: Casey Bodley Reviewed-by: Yuri Weinstein commit abeb3c56c074f9024565ed8d48dc0d89b4337084 Merge: 8fe88c45e3 0d18c24fce Author: Yuri Weinstein Date: Mon Jan 29 08:10:27 2018 -0800 Merge pull request #18972 from linuxbox2/luminous-22084 luminous: rgw: Fix swift object expiry not deleting objects Reviewed-by: Casey Bodley commit 8fe88c45e3eb9d985ee20cf49679f5db08faf3db Merge: d55eda8267 fe5c8b34a7 Author: Yuri Weinstein Date: Mon Jan 29 08:09:56 2018 -0800 Merge pull request #19086 from shinobu-x/wip-22210-luminous luminous: radosgw-admin zonegroup get and zone get should return defaults when there is no realm Reviewed-by: Casey Bodley commit d55eda826741653537f64c6cccd7464d7b82efa2 Merge: 7e84974d2d 14434beda6 Author: Yuri Weinstein Date: Mon Jan 29 08:09:25 2018 -0800 Merge pull request #19088 from pdvian/wip-22177-luminous luminous: rgw: check going_down() when lifecycle processing Reviewed-by: Casey Bodley Reviewed-by: Amit Kumar commit 7e84974d2d5b653776eb89c623130ef651108a0d Merge: 0f1f98ad44 e3bb21529b Author: Yuri Weinstein Date: Mon Jan 29 08:08:15 2018 -0800 Merge pull request #19636 from shinobu-x/wip-22506-luminous luminous: rgw usage trim only trims a few entries Reviewed-by: Casey Bodley commit 0f1f98ad449b73568077513196f0d8dc43a9b07f Merge: d51dbcaf03 ad0ea9fa53 Author: Yuri Weinstein Date: Mon Jan 29 08:06:43 2018 -0800 Merge pull request #19784 from linuxbox2/luminous-20201 luminous: RGW: S3 POST policy should not require Content-Type Reviewed-by: Casey Bodley Reviewed-by: Abhishek Lekshmanan commit d51dbcaf03a8c8a6c7a8330fb2f23a03d75a411a Merge: 1708c17f28 84fff7aca5 Author: Yuri Weinstein Date: Mon Jan 29 08:05:58 2018 -0800 Merge pull request #19799 from smithfarm/wip-22581-luminous luminous: rgw: multisite: 'radosgw-admin sync error list' contains temporary EBUSY errors Reviewed-by: Casey Bodley commit 1708c17f28dfc57462f9b05c1c9b6c73312193e5 Merge: 7c3e468476 2adc133b86 Author: Yuri Weinstein Date: Mon Jan 29 08:03:10 2018 -0800 Merge pull request #19867 from smithfarm/wip-22591-luminous luminous: rgw: refuses upload when Content-Type missing from POST policy Reviewed-by: Casey Bodley commit 7c3e46847667ad104b3b718b7f4460b2fc9f763f Merge: 915a8e1454 4318ac5e3f Author: Yuri Weinstein Date: Mon Jan 29 08:02:23 2018 -0800 Merge pull request #20051 from ceph/revert-19538-wip-22434-luminous Revert "luminous: rgw: user stats increased after bucket reshard" Reviewed-by: Casey Bodley commit 915a8e1454982e6b996e6757d13523fb4083cd2d Merge: 9882489f8a e0b8512419 Author: Yuri Weinstein Date: Mon Jan 29 08:01:56 2018 -0800 Merge pull request #20056 from linuxbox2/luminous-rgw-latchfix luminous: rgw_file: alternate fix deadlock on lru eviction Reviewed-by: Casey Bodley commit 9882489f8ae81104ae621cf9633f61b469640a8e Merge: 440fdfa438 35d2baf47b Author: Yuri Weinstein Date: Mon Jan 29 07:57:29 2018 -0800 Merge pull request #20073 from smithfarm/wip-22767-luminous luminous: rgw: librgw: fix shutdown error with resources uncleaned Reviewed-by: Casey Bodley commit 440fdfa438bf881a3e5e2c8d4379284dc70eb673 Merge: f83a246ba5 e3fbaa92ce Author: Yuri Weinstein Date: Mon Jan 29 07:56:53 2018 -0800 Merge pull request #20075 from smithfarm/wip-22773-luminous luminous: rgw: file deadlock on lru evicting Reviewed-by: Matt Benjamin commit f83a246ba550a7725438823c296cc18e56dea7ff Merge: e50835f80c b219b854b7 Author: Yuri Weinstein Date: Mon Jan 29 07:56:04 2018 -0800 Merge pull request #20107 from dreamhost/wip-22792 luminous: rgw: When a system object is created exclusively, do not distribute the Reviewed-by: Casey Bodley commit a26b4ae76521e4cd12a65e69c735676454ce6e4c Author: Konstantin Shalygin Date: Fri Jan 26 18:31:20 2018 +0700 mgr: balancer: fixed mistype "AttributeError: 'Logger' object has no attribute 'err'" Signed-off-by: Konstantin Shalygin (cherry picked from commit 2062e84c7a33fc5170740f2d60d07ddf62085457) commit f2a202775fdf1132891221fa58700b276da7429b Author: John Spray Date: Tue Jan 23 07:43:12 2018 -0500 mgr: apply auth_service_required to client conns Previously was using auth_cluster_required for all connections, which meant that if someone had disabled client cephx, they'd get BADAUTHORIZER from their CLI when it tried to load mgr command descriptions. Disabling cephx on the admin CLI is odd, but the mon tolerates it so the mgr should too. Fixes: https://tracker.ceph.com/issues/22096 Signed-off-by: John Spray (cherry picked from commit 86ee30c33a06a13e6bb360b80dd1e41ba928147b) commit 2852a5f533a5cf044517fa288e8076ac493b1da0 Author: John Spray Date: Mon Jan 22 13:42:20 2018 +0000 mon: fix mgr using auth_client_required policy This caused mgr daemons to fail to authenticate when auth_client_required was set to something different to auth_cluster_required. Fixes: https://tracker.ceph.com/issues/22096 Signed-off-by: John Spray (cherry picked from commit 1e06fe003e50d360b5c5ce9824bba044b6406ec5) commit cfd05be6ace8fb3145068c464bc0a73d487836a1 Author: Song Shun Date: Wed Jan 17 09:32:45 2018 +0800 librbd: fix snap create/rm may taking long time fix snap create/rm may taking long time http://tracker.ceph.com/issues/22716 Signed-off-by: Song Shun (cherry picked from commit d04ed348a1b20e5ea5bedada2462cb41f0f1d85a) Conflicts: test_mock_SnapshotRemoveRequest.cc: define om as a ObjectMap obj commit 8950736b8e3c2837f201b4eba4fdebcc97fbd9be Author: Chang Liu Date: Tue Jan 23 15:39:56 2018 +0800 mon: use pool_sum as param name in PGMapDigest::recovery_summary the param name of delta_sum of PGMapDigest::recovery_summary() is misleading too. delta_sum fits the recovery_rate_summary() well, but not recovery_summary(). Signed-off-by: Chang Liu (cherry picked from commit 73ff298b898ca76e3f9b615449b4e4c0f2d0c017) commit e94900c7caa2bd920684073704bc688249bc0591 Author: Chang Liu Date: Thu Jan 18 22:50:25 2018 +0800 mon: do not use per_pool_sum_delta to show recovery summary Fixes: http://tracker.ceph.com/issues/22727 Signed-off-by: Chang Liu (cherry picked from commit 5981ddc1992190431743ce1f0b834fb8899b6811) commit 540f425f1a86dabf746ac4bdd14c54b4ec76f664 Author: David Zafman Date: Fri Oct 20 19:50:32 2017 -0700 osd: Scrub of object with only object error(s) are not logged Signed-off-by: David Zafman (cherry picked from commit 1ad05b1068ddd5d3312af45af1a60587200ddcd7) commit 4c2a8f1dd08882eca83512152c8570f497081229 Author: David Zafman Date: Fri Oct 20 17:34:49 2017 -0700 osd: Primary shard needs to be at the front to be preferred Caused by: cd0d8b0714d8684cf61b4650e170027ef46f489b Fixes: http://tracker.ceph.com/issues/21907 Signed-off-by: David Zafman (cherry picked from commit b7d2f27da1a68046a688749f95093a88e6362b28) commit 17c64595a2f3481ab60ffe6e453a6dc879f1e01e Author: David Zafman Date: Thu Oct 19 12:56:58 2017 -0700 ceph-osdomap-tool: Add compact comand option Signed-off-by: David Zafman (cherry picked from commit b1f397f4fc3313ee0325722156759bfe32286c6b) commit 152272ce49cf2b32e4b3dbba5aa28019e9a183e0 Author: David Zafman Date: Wed Oct 18 18:07:41 2017 -0700 osd: Fixes for mark_unfound_lost Let recovery requeue operations and release backoff For "delete" mark object context, if present, to non-existent Signed-off-by: David Zafman (cherry picked from commit 0c106f1a46815bb769e7a503b1adb265480b6779) commit 339e99005691ebdcdac45ba249add4ce4d6a8125 Author: David Zafman Date: Wed Oct 18 16:07:16 2017 -0700 ceph-objectstore-tool: Fix debug output Signed-off-by: David Zafman (cherry picked from commit 090c773fc9b7f3316ed88edd24b85a703224ddc5) commit f23035fb17fb816c1b0f41ccf4072a14f50b1fa1 Author: David Zafman Date: Wed Oct 18 16:04:59 2017 -0700 test: Remove bogus check in ceph_objectstore_tool.py Signed-off-by: David Zafman (cherry picked from commit f918b1fac1f3cdb4cf316fb4c7afc90de49ce06e) commit dbf54df87941f538e73b672131018ce4baba338c Author: Jeff Layton Date: Fri Aug 25 08:31:47 2017 -0400 client: reset unmounting flag to false when starting a new mount Once the "unmounting" flag goes to true, it never flips back to false. We don't have a lot of checks for "unmounting" in the code, but they're in some subtle places today and I plan to add more later. It's not clear to me whether it's possible (or advisable) to reuse a Client that has previously been unmounted. It most certainly won't work as expected today, and the checks I'll be adding will break it for sure. Make sure we clear the unmounting flag when we go to re-mount the Client. Signed-off-by: Jeff Layton (cherry picked from commit 3bc61f6c23a9ada5e2250f66e4df04a66dcfbacf) commit 20164df443795c4bde6bfdbb9eeda34e915c7557 Author: Jeff Layton Date: Fri Aug 25 08:31:47 2017 -0400 client: rework Client::get_local_osd() return codes It currently returns -1 when there isn't one, but the C wrapper can also return -ENOTCONN if the cmount isn't mounted. Change it to return -ENXIO (No such device or address) in the case of "no local osd". Signed-off-by: Jeff Layton (cherry picked from commit 4285b970cb57c88deae0317be24afcf21a95fdc0) commit 12e93809aa1adbb7c3b112b4ba6b3a4374429163 Author: Jeff Layton Date: Fri Aug 25 08:31:47 2017 -0400 client: remove misleading comment in get_cap_ref That may have been true at one time, but it's not always a single bit now. Signed-off-by: Jeff Layton (cherry picked from commit 93f8cc1f71ffdc8dddca7efb56d82548a66ee3fe) commit e50835f80c63563a419ae47b05f72885e61e528e Merge: bba828b2d7 4efada35fd Author: Jason Dillaman Date: Fri Jan 26 08:15:17 2018 -0500 Merge pull request #20136 from dillaman/wip-22815-luminous luminous: qa/workunits/rbd: switch devstack to pike release Reviewed-by: Mykola Golub commit 4efada35fd6b2d9b24cd89ac1dc3aeb7c823cfbc Author: Jason Dillaman Date: Wed Jan 24 11:28:19 2018 -0500 qa/workunits/rbd: switch devstack to pike release Signed-off-by: Jason Dillaman (cherry picked from commit 7ff1e0bc2193acd009a74db81cf8e8cc8639ad12) commit 37067a6f1275ea62d40f398c7b5d8e76f08f527a Author: Jason Dillaman Date: Wed Jan 24 14:40:56 2018 -0500 librbd: force removal of a snapshot cannot ignore dependent children Fixes: http://tracker.ceph.com/issues/22791 Signed-off-by: Jason Dillaman (cherry picked from commit fcc58ecfeba6bbc72588e69dba35779f94d28ba5) commit bba828b2d761b1459b32588accb6838ceeb0cb40 Merge: f473d3e444 4dda1b6ead Author: Yuri Weinstein Date: Thu Jan 25 12:03:21 2018 -0800 Merge pull request #19105 from batrick/i22228 luminous: client: anchor Inode while trimming caps Reviewed-by: Yan, Zheng Reviewed-by: Yuri Weinstein commit f473d3e4446110f219f69a577a4e5392ec4b422d Merge: 646a2c4dd4 ee06a16ee8 Author: Yuri Weinstein Date: Thu Jan 25 12:02:32 2018 -0800 Merge pull request #19360 from ukernel/luminous-22219 luminous: mds: ignore export pin for unlinked directory Reviewed-by: Patrick Donnelly commit 646a2c4dd4e02a2f995b9b59d514cdef19065478 Merge: 1b1ce1326e 1643d5a4f8 Author: Yuri Weinstein Date: Thu Jan 25 12:01:35 2018 -0800 Merge pull request #19646 from pdvian/wip-22503-luminous luminous: mds: properly eval locks after importing inode Reviewed-by: Patrick Donnelly commit 1b1ce1326e991b2e2d65826848fb594d238b6815 Merge: 09baff4bbb 80de33b1ca Author: Josh Durgin Date: Thu Jan 25 11:46:32 2018 -0800 Merge pull request #20055 from dzafman/wip-22724 luminous: miscounting degraded objects and PG stuck in recovery_unfound Reviewed-by: Josh Durgin commit 80de33b1ca215046290160fcc2781f3b29cb5c35 Merge: 7522ddd978 09baff4bbb Author: David Zafman Date: Thu Jan 25 11:44:18 2018 -0800 Merge branch 'luminous' into wip-22724 commit 09baff4bbbaa80d57ecb130b7921f19e9df03e8f Merge: d7874dc154 0040c7bfe1 Author: Josh Durgin Date: Thu Jan 25 11:36:22 2018 -0800 Merge pull request #20081 from dzafman/wip-recovering-luminous luminous: osd/ReplicatedPG.cc: recover_replicas: object added to missing set for backfill, but is not in recovering, error! Reviewed-by: Josh Durgin commit d7874dc1543b26ea0ac86f93d70588ec2e394fd9 Merge: 2ecc4c326e a170e67760 Author: Yuri Weinstein Date: Thu Jan 25 11:12:20 2018 -0800 Merge pull request #18008 from ukernel/luminous-21584 luminous: mds: fix CDir::log_mark_dirty() Reviewed-by: Patrick Donnelly Reviewed-by: Jos Collin Reviewed-by: Nathan Cutler commit 2ecc4c326edaef0ab95ee43e69ddbc37b428840a Merge: 6097c4f10e ddba907279 Author: Yuri Weinstein Date: Thu Jan 25 11:11:17 2018 -0800 Merge pull request #18782 from ukernel/luminous-21985 luminous: mds: fix MDS_FEATURE_INCOMPAT_FILE_LAYOUT_V2 definition Reviewed-by: Patrick Donnelly commit 6097c4f10e5353ee73b40b7defb4c828aeb7b071 Merge: b18280dbd5 ab103b9fe3 Author: Yuri Weinstein Date: Thu Jan 25 11:09:39 2018 -0800 Merge pull request #18783 from ukernel/luminous-21975 luminous: mds: trim 'N' log segments according to how many log segments are there Reviewed-by: Patrick Donnelly commit b18280dbd572768b0b84edce034a2419ae181a0f Merge: 443bfb698f d73deb017c Author: Yuri Weinstein Date: Thu Jan 25 11:08:27 2018 -0800 Merge pull request #18831 from shinobu-x/wip-22076-luminous luminous: cephfs: "ceph tell mds" commands result in "File exists" errors on client admin socket Reviewed-by: Jos Collin Reviewed-by: Patrick Donnelly commit 443bfb698f96c2e339a793a2950aab7dcd03b232 Merge: 0e762253c6 efd4147fa8 Author: Yuri Weinstein Date: Thu Jan 25 11:07:27 2018 -0800 Merge pull request #18835 from shinobu-x/wip-22074-luminous luminous: mds: don't check gid when none specified in auth caps Reviewed-by: Patrick Donnelly commit 0e762253c6709631c984cfddd2a066bc87034255 Merge: 1412d885aa 1410b5a5f5 Author: Yuri Weinstein Date: Thu Jan 25 11:06:52 2018 -0800 Merge pull request #18869 from shinobu-x/wip-21952-luminous luminous: mds: no assertion on inode being purging in find_ino_peers() Reviewed-by: Patrick Donnelly commit 1412d885aab6a0c76566960e22109920f7580803 Merge: dbc7cf62ea e485b89ffd Author: Yuri Weinstein Date: Thu Jan 25 11:06:17 2018 -0800 Merge pull request #18871 from shinobu-x/wip-21947-luminous luminous: mds: preserve order of requests during recovery of multimds cluster Reviewed-by: Patrick Donnelly commit dbc7cf62ea5b46775c1b142523e9cb01d65a9dad Merge: 4a0835d0a4 1ec93753a3 Author: Yuri Weinstein Date: Thu Jan 25 11:05:28 2018 -0800 Merge pull request #18912 from shinobu-x/wip-22077-luminous luminous: src/mds/MDCache.cc: 7421: FAILED assert(CInode::count() == inode_map.size() + snap_inode_map.size()) Reviewed-by: Patrick Donnelly commit 4a0835d0a476aa7686ec449f2d8c7cb0b2ced4c3 Merge: 6fa7c82bca bd3e7795ca Author: Yuri Weinstein Date: Thu Jan 25 11:04:50 2018 -0800 Merge pull request #19055 from shinobu-x/wip-22192-luminous luminous: MDSMonitor: monitor gives constant "is now active in filesystem cephfs as rank" cluster log info messages Reviewed-by: Patrick Donnelly Reviewed-by: Yan, Zheng commit 6fa7c82bca25a0d28fd2104850aac3770d0243c7 Merge: cdfe41dde0 c514d3d427 Author: Yuri Weinstein Date: Thu Jan 25 11:02:59 2018 -0800 Merge pull request #19326 from ukernel/luminous-22263 luminous: mds: handle client reconnect gather race Reviewed-by: Patrick Donnelly commit cdfe41dde0e4ae2bc5d8d46a27595532db299a62 Merge: dc68871f08 35bb3d4d17 Author: Yuri Weinstein Date: Thu Jan 25 11:00:27 2018 -0800 Merge pull request #19480 from smithfarm/wip-22407-luminous luminous: cephfs: client: implement delegation support in userland cephfs Reviewed-by: Patrick Donnelly Reviewed-by: Jeff Layton commit dc68871f0850971c41434b1618e7662b436e4d18 Merge: 75d7ce3cb3 8d6c2c2dea Author: Yuri Weinstein Date: Thu Jan 25 10:58:46 2018 -0800 Merge pull request #19775 from ukernel/luminous-19578 luminous: mds: track dirty dentries in separate list Reviewed-by: Patrick Donnelly commit 75d7ce3cb3cea53340bd7738d1b9567f3c5e88d1 Merge: b2e38487d7 e97399d2ee Author: Yuri Weinstein Date: Thu Jan 25 10:58:18 2018 -0800 Merge pull request #19827 from pdvian/wip-22587-luminous luminous: mds: reduce debugging level for balancer messages Reviewed-by: Patrick Donnelly Reviewed-by: Amit Kumar commit b2e38487d78327c2c092e786842028e61177edbc Merge: af9499dba0 55f4642be2 Author: Yuri Weinstein Date: Thu Jan 25 10:57:05 2018 -0800 Merge pull request #20082 from smithfarm/wip-21525-luminous luminous: cephfs: client: dual client segfault with racing ceph_shutdown Reviewed-by: Patrick Donnelly Reviewed-by: Josh Durgin commit af9499dba0e1822b8cd56f3d533fdc0be88e0065 Merge: 832f228dc8 dc0e0250d1 Author: Yuri Weinstein Date: Thu Jan 25 10:55:53 2018 -0800 Merge pull request #20085 from pdvian/wip-22763-luminous luminous: cephfs-journal-tool: add "set pool_id" option Reviewed-by: Patrick Donnelly commit 832f228dc845129c95bbb688f70322ee636f7e74 Merge: e7113bca11 4c211d0d6c Author: Yuri Weinstein Date: Thu Jan 25 10:55:19 2018 -0800 Merge pull request #20086 from pdvian/wip-22765-luminous luminous: client: avoid recursive lock in ll_get_vino Reviewed-by: Patrick Donnelly commit 26b733758360f144008a4b48272bad45a7ac7673 Author: Jason Dillaman Date: Tue Dec 5 09:58:04 2017 -0500 ceph-dencoder: moved RBD types outside of RGW preprocessor guard Fixes: http://tracker.ceph.com/issues/22321 Signed-off-by: Jason Dillaman (cherry picked from commit b00ac12f4045cd39ac9a436cf32d233d16bc3485) commit d4f5dc27803d0f356c15ee2a4de4c22dbf943fe3 Author: Yan, Zheng Date: Wed Sep 6 17:06:44 2017 +0800 mds: fix return value of MDCache::dump_cache previous commit "mds: track snap inodes through sorted map" makes MDCache::dump_cache return 1 on success. Signed-off-by: "Yan, Zheng" (cherry picked from commit f519fca9dd958121a289676edf5175fb8be9894f) Backport note: the original cause was commit 46c829d23aa5379a973d2330b0551a5b92598133. The fix was part of snapshot fixes for Mimic in f519fca9dd958121a289676edf5175fb8be9894f which was not backported. commit fc791b980d2e9835e66fec8b39264be370d6a740 Author: Casey Bodley Date: Mon Sep 18 13:29:40 2017 -0400 qa/rgw: hadoop suite targets centos_latest Signed-off-by: Casey Bodley (cherry picked from commit a97d5bea42a40909aa4e9672505b9ad994cec184) Conflicts: qa/suites/rgw/hadoop-s3a/s3a-hadoop.yaml (changes already backported via 6b5f212b9170dbd99779527b41be233b9d40b5e4) commit 470c5f23a0b85fb8a71c98566ecb0f95e376ceef Author: Casey Bodley Date: Mon Sep 18 13:28:22 2017 -0400 qa/rgw: use overrides for hadoop version instead of having two separate tasks, add v27.yaml and v28.yaml with overrides for hadoop_version Signed-off-by: Casey Bodley (cherry picked from commit 4a953a16d7b80d343899de39a8cf55dc761847de) commit e7113bca11803206dd81b8c3a5e457c4941c845a Merge: 60c74451c3 22ff147734 Author: Yuri Weinstein Date: Thu Jan 25 09:11:10 2018 -0800 Merge pull request #18019 from ukernel/luminous-21091 luminous: mds: fix StrayManager::truncate() Reviewed-by: Patrick Donnelly Reviewed-by: John Spray commit 60c74451c36d1e20a79bcb25edc58a070c74f934 Merge: cea6b6e392 de05d2c819 Author: Jason Dillaman Date: Thu Jan 25 12:03:54 2018 -0500 Merge pull request #19485 from dillaman/wip-21788-luminous luminous: librbd: refresh image after applying new/removing old metadata Reviewed-by: Mykola Golub commit cea6b6e392dccd3793d6db2eeed50dd9969c80e8 Merge: fa97a4793b 369dcb36c7 Author: Jason Dillaman Date: Thu Jan 25 11:44:39 2018 -0500 Merge pull request #18834 from shinobu-x/wip-22073-luminous luminous: [api] compare-and-write methods not properly advertised Reviewed-by: Jason Dillaman commit fa97a4793bfb8ee092fc10b4961652465f5d686b Merge: 2d0a2df9b8 818c354a43 Author: Jason Dillaman Date: Thu Jan 25 11:27:18 2018 -0500 Merge pull request #19058 from shinobu-x/wip-22190-luminous luminous: class rbd.Image discard----OSError: [errno 2147483648] error discarding region Reviewed-by: Jason Dillaman commit 2d0a2df9b82412b1fd64ee8a6ce13a60ea59c56e Merge: 5bfd5cb595 2070fcf7d4 Author: Yuri Weinstein Date: Thu Jan 25 08:18:13 2018 -0800 Merge pull request #19066 from pdvian/wip-22172-luminous luminous: [rbd-nbd] Fedora does not register resize events Reviewed-by: Jason Dillaman Reviewed-by: Amit Kumar commit 5bfd5cb5953d2cc0524632b79ee07821b01b7fae Merge: 043a7a19e0 17ff1f1fd2 Author: Yuri Weinstein Date: Thu Jan 25 08:17:38 2018 -0800 Merge pull request #19107 from pdvian/wip-22208-luminous luminous: rbd: disk usage on empty pool no longer returns an error message Reviewed-by: Jason Dillaman commit 043a7a19e07b821693fce5d42317dbf9d825993e Merge: 588e575936 c3c3bb0bc9 Author: Yuri Weinstein Date: Thu Jan 25 08:17:01 2018 -0800 Merge pull request #19305 from pdvian/wip-21700-luminous luminous: rbd-mirror: Allow a different data-pool to be used on the secondary cluster Reviewed-by: Jason Dillaman commit 588e5759360fdeaf00ac6c3c901cf3d20a269191 Merge: ab6b45d687 08a2358f85 Author: Yuri Weinstein Date: Thu Jan 25 08:16:26 2018 -0800 Merge pull request #19447 from smithfarm/wip-21646-luminous luminous: librbd: Image-meta should be dynamically refreshed Reviewed-by: Jason Dillaman commit ab6b45d687b4a66e25833875c81d2f2dab8d2d95 Merge: 43333e0d5f 5e8c4f83f4 Author: Yuri Weinstein Date: Thu Jan 25 08:15:50 2018 -0800 Merge pull request #19479 from smithfarm/wip-22376-luminous luminous: rbd: Python RBD metadata_get does not work Reviewed-by: Jason Dillaman Reviewed-by: Amit Kumar commit 43333e0d5fd2949441d99e9683e6399afd67e5e3 Merge: 3ac673215f 235032ec64 Author: Yuri Weinstein Date: Thu Jan 25 08:14:53 2018 -0800 Merge pull request #19484 from dillaman/wip-21644-luminous luminous: rbd-mirror: sync image metadata when transfering remote image Reviewed-by: Mykola Golub commit 3ac673215f3310c12762ad1bc3e32948b220e741 Merge: 9560a8b2dc 1507015169 Author: Yuri Weinstein Date: Thu Jan 25 08:13:37 2018 -0800 Merge pull request #19503 from shinobu-x/wip-22395-luminous luminous: librbd: cannot clone all image-metas if we have more than 64 key/value pairs Reviewed-by: Jason Dillaman commit 9560a8b2dcb13cf774a94d97b7d3bc274f77c22c Merge: 9f48a65b50 feb4d7ba37 Author: Yuri Weinstein Date: Thu Jan 25 08:12:33 2018 -0800 Merge pull request #19504 from shinobu-x/wip-22393-luminous luminous: librbd: cannot copy all image-metas if we have more than 64 key/value pairs Reviewed-by: Jason Dillaman commit 9f48a65b50538c3925fc162f5e282c0a414c0a14 Merge: 49d5128b05 2b9bbf605e Author: Yuri Weinstein Date: Thu Jan 25 08:11:57 2018 -0800 Merge pull request #19554 from shinobu-x/wip-22454-luminous luminous: cluster resource agent ocf:ceph:rbd - wrong permissions Reviewed-by: Jason Dillaman commit 49d5128b055930859d4d8716ea94100e5cbc3878 Merge: 5051aa8646 3d0f4fa752 Author: Yuri Weinstein Date: Thu Jan 25 08:11:22 2018 -0800 Merge pull request #19625 from shinobu-x/wip-22497-luminous luminous: [rbd-mirror] new pools might not be detected Reviewed-by: Jason Dillaman commit 5051aa8646f9240043261e184baa5fd05bcedd23 Merge: 147d57b3a5 a15eb7df1e Author: Yuri Weinstein Date: Thu Jan 25 08:10:37 2018 -0800 Merge pull request #19800 from smithfarm/wip-21641-luminous luminous: rbd ls -l crashes with SIGABRT Reviewed-by: Mykola Golub commit 52e59854c21d4e29e49a16c6d2f73470e68c73cd Author: Chang Liu Date: Sat Oct 21 23:35:47 2017 +0800 mgr: request daemon's metadata when receiving a report message from an unknown server Fixes: http://tracker.ceph.com/issues/21687 Signed-off-by: Chang Liu (cherry picked from commit a39813837ca84434e57afa47138973268ff366d3) commit e97ad14cdc22c0da907fabc48d62698c94f6ba66 Author: liuchang0812 Date: Thu Aug 24 10:56:29 2017 +0800 mgr: kill MgrSession when MMgrReport come from daemon without metadata info Signed-off-by: liuchang0812 (cherry picked from commit 29080134339e5e64d50af1db9fe50df7ea55c1d0) commit 147d57b3a5e7c9dde57d8f7ccd4f947c92d31ad7 Merge: a69f543b70 88c987cdb5 Author: Yuri Weinstein Date: Thu Jan 25 08:09:49 2018 -0800 Merge pull request #19802 from smithfarm/wip-21690-luminous luminous: tests: rbd_mirror_helpers.sh request_resync_image function saves image id to wrong variable Reviewed-by: Jason Dillaman commit a69f543b70e554c88022dec020bd78d48da0445d Merge: 7ba84945e7 17aa16dc6b Author: Yuri Weinstein Date: Thu Jan 25 08:09:02 2018 -0800 Merge pull request #19853 from pdvian/wip-22593-luminous luminous: rbd: librbd: filter out potential race with image rename Reviewed-by: Jason Dillaman commit 4318ac5e3f36b9885953f03d30c6b6a72b62d7e6 Author: Orit Wasserman Date: Mon Jan 22 17:18:43 2018 +0200 Revert "luminous: rgw: user stats increased after bucket reshard" Signed-off-by: Orit Wasserman commit cd9e6788cbb8ead60c82d15c574c5a00f4c1d222 Author: Kefu Chai Date: Thu Jan 4 13:06:52 2018 +0800 tools/ceph_monstore_tool: include mgrmap in initial paxos epoch before this change, the "mgr" map is not included paxos, thus why the peon mon fails to have the mgr map after it is sync'ed with its leader. and hence there is chance that ceph-mgr is unable to get a mgr map from the monitor it is connected to. Fixes: http://tracker.ceph.com/issues/22266 Signed-off-by: Kefu Chai (cherry picked from commit f3f2a8bad4da7fda485eba7412d40b7b5758452d) commit b219b854b708c532bacb37ee8c8253c80586fef6 Author: J. Eric Ivancich Date: Wed Dec 6 16:36:28 2017 -0500 When a system object is created exclusively, do not distribute the cache information to the other rados gateways and instead let them lazily retrieve the information when they need it. Signed-off-by: J. Eric Ivancich (cherry picked from commit 647ce3387312fc683660c1f3c7571c577379be1c) Signed-off-by: Robin H. Johnson commit 7ba84945e7598333dca5f75dcbb7771f6b85fa13 Merge: 9297805666 9a7e1f6a44 Author: Yuri Weinstein Date: Wed Jan 24 13:19:46 2018 -0800 Merge pull request #18722 from shinobu-x/wip-luminous-21946 luminous: mds: set PRIO_USEFUL on num_sessions counter Reviewed-by: Patrick Donnelly Reviewed-by: Josh Durgin commit 92978056660f60e878a4a2aca377e2721c4d08e5 Merge: ae19fa733f 948ca443fc Author: Yuri Weinstein Date: Wed Jan 24 13:18:41 2018 -0800 Merge pull request #18730 from shinobu-x/wip-luminous-22030 luminous: List of filesystems does not get refreshed after a filesystem deletion Reviewed-by: Josh Durgin Reviewed-by: Patrick Donnelly commit ae19fa733f2fe6dc1feab1650851d559abb48791 Merge: fbb14a0e7b fd956acfd6 Author: Yuri Weinstein Date: Wed Jan 24 13:18:04 2018 -0800 Merge pull request #18754 from shinobu-x/wip-21636-luminous luminous: ceph-monstore-tool --readable mode doesn't understand FSMap, MgrMap Reviewed-by: Josh Durgin commit 3ccf7e0bc6ca51d0d1e54f5bb7558f9f7aa199f3 Author: Kefu Chai Date: Mon Jan 22 13:02:40 2018 +0800 osd/PG: print a space after __func__ in log message Signed-off-by: Kefu Chai (cherry picked from commit e7bdf589db2010ea82aa33d60e9c54c14c7a78f0) commit 32c0eab8749e813b44ffe5ecefa3e59c61cb90a7 Author: Kefu Chai Date: Mon Jan 22 13:08:48 2018 +0800 osd: update store with options after pg is created Fixes: http://tracker.ceph.com/issues/22419 Signed-off-by: Kefu Chai (cherry picked from commit 43b2e512e61319f14f7390d063d66f7a2a568b4d) Conflicts: src/osd/PG.cc (omitted immediately preceding write_if_dirty() call because fa7818aefa92b4e9e067945c33f96d520e885f4e not backported to luminous) commit fbb14a0e7bd76053f7347afb1c0460e4e984b768 Merge: 06b9f828cf 7d6984b554 Author: Abhishek L Date: Wed Jan 24 17:07:06 2018 +0100 Merge pull request #20053 from dillaman/wip-22740-luminous luminous: qa/suites/upgrade: disable broken kraken upgrade cls_rbd test Reviewed-By: Yuri Weinstein commit 06b9f828cfef1843e43852e2e95efcb6d0b6c4d7 Merge: 1898fb6f38 767ad8518d Author: Yuri Weinstein Date: Wed Jan 24 07:05:13 2018 -0800 Merge pull request #18564 from kmroz/wip-21688-luminous luminous: Possible deadlock in 'list_children' when refresh is required Reviewed-by: Jason Dillaman commit 1898fb6f38e5eaea16ff0b1d6007bfc605337136 Merge: b0a841fd7c 1d4435f0cd Author: Yuri Weinstein Date: Wed Jan 24 07:04:38 2018 -0800 Merge pull request #18698 from shinobu-x/wip-luminous-21808 luminous: rbd: fix crash during map Reviewed-by: Jason Dillaman commit b0a841fd7cc356b073bc38f4b66d55e741e5de68 Merge: 61ec339e6c c6bc756304 Author: Yuri Weinstein Date: Wed Jan 24 07:03:43 2018 -0800 Merge pull request #19038 from pdvian/wip-22183-luminous luminous: rgw: set sync_from_all as true when no value is seen Reviewed-by: Casey Bodley commit 61ec339e6cba3935f531886684d25cd5708fb67a Merge: 2d93e37609 34450ed6c0 Author: Abhishek L Date: Wed Jan 24 15:23:54 2018 +0100 Merge pull request #19446 from smithfarm/wip-22404-luminous luminous: doc: crush_ruleset is invalid command in luminous Reviewed-By: Abhishek Lekshmanan commit 2d93e37609a757f2d3e6e4870a8054eb36626ed8 Merge: c93fa099ee acccae56c4 Author: Alfredo Deza Date: Wed Jan 24 08:46:21 2018 -0500 Merge pull request #18747 from shinobu-x/wip-21794-luminous luminous: backoff causes out of order op Reviewed-by: Kefu Chai Reviewed-by: Yuri Weinstein commit 127236f6766c8539dbcf6a7957b4f3a04e7807db Author: Piotr Dałek Date: Wed Dec 13 16:13:33 2017 +0100 options, Mon: monitor the snap trim queues If new option "mon osd snap trim queue warn on" is set to value larger than 0 (32768 by default), cluster will go into HEALTH_WARN state once any pg has a snap trim queue larger than that value. This can be used as an indicator of snaptrimmer not keeping up and disk space not being reclaimed fast enough. Warning message will tell how many pgs are affected. Signed-off-by: Piotr Dałek (cherry picked from commit 8412a65e0e6f610fb39430d8f65e561b6dbda13f) commit ca4413dee9a148c5ffda235793b02499a3975e88 Author: Piotr Dałek Date: Wed Dec 13 15:23:55 2017 +0100 osd, PG: share the snaptrimq.size() within pg_stat_t That way it will be unnecessary to go through all pgs separately to find pgs with excessively long snap trim queues. And we don't need to share snap trim queues itself, which may be large by itself. As snap trim queues tend to be short and anything above 50 000 I consider absurdly large, the snaptrimq_len is capped at 2^32 to save space in pg_stat_t. Signed-off-by: Piotr Dałek (cherry picked from commit 022d246f17b31d309525395831c84e9856533adb) Conflicts: src/osd/osd_types.cc - needed a fix for snaptrimq_len that was placed after two other new fields commit faeb81cd4dc24dc087877644ee27b937d50b4108 Author: Piotr Dałek Date: Wed Dec 13 10:14:30 2017 +0100 osd/PrimaryLogPG: dump snap_trimq size One can just parse the snap_trimq string, but that's much more expensive than just reading an unsigned int. Signed-off-by: Piotr Dałek (cherry picked from commit dc7781cf17d11cb09067656cb25c0d710ab60d71) commit c93fa099ee173a0fea14bc8b2c36c2161fa0124b Merge: 6c5684c880 b97227d34e Author: Kefu Chai Date: Wed Jan 24 11:23:49 2018 +0800 Merge pull request #20087 from dzafman/wip-misc-luminous qa/standalone: Minor fixes for ceph_objectstore_tool.py test Reviewed-by: Kefu Chai commit 0040c7bfe1a47b73954c046cf460e2e96c66263a Author: David Zafman Date: Wed Oct 11 20:16:44 2017 -0700 test: Cleanup test-erasure-eio.sh code Signed-off-by: David Zafman (cherry picked from commit 69b5fc54feb59f8b0a26a3ca3e925980c91b5b75) commit 325eec1ba9c9b03d3ad6bcf56572be64d6c16c98 Author: David Zafman Date: Mon Oct 9 14:15:51 2017 -0700 test: Add replicated recovery/backfill test Signed-off-by: David Zafman (cherry picked from commit c2572bee3c6256419e0b265e9e2829e7f3afb76d) commit 3068ea1e93f1806b74b78c636693f7afcff95aea Author: David Zafman Date: Mon Oct 9 14:03:23 2017 -0700 osd: Better recovery/backfill restart for mark_unfound_lost delete/revert Signed-off-by: David Zafman (cherry picked from commit 689bff354a6bfa1cf47b0d0a04fa9f1b2ef68f75) commit 25a901baf7f5d8130b7ed3d6913608105ccd4de6 Author: David Zafman Date: Mon Oct 9 08:19:21 2017 -0700 osd, mon: Add new pg states recovery_unfound and backfill_unfound Signed-off-by: David Zafman (cherry picked from commit 7f8b0ce9e681f727d8217e3ed74a1a3355f364f3) Conflicts: src/osd/PG.h (trivial) src/osd/osd_types.h (trivial) commit e40e2aa53b192a86abaa95fc74f05b5575e16d62 Author: David Zafman Date: Mon Oct 9 08:17:29 2017 -0700 osd: Add new UnfoundBackfill and UnfoundRecovery pg transitions Signed-off-by: David Zafman (cherry picked from commit bb2bcb95f51abc206e005e44ef383ee45b8f2209) Conflicts: src/osd/PG.cc (trivial, no MBackfillReserve::CANCEL) src/osd/PG.h (trivial) fixup! osd: Add new UnfoundBackfill and UnfoundRecovery pg transitions commit bb21f23731027df30844236573648bca300b523e Author: Jason Dillaman Date: Thu Nov 30 10:05:14 2017 -0500 mgr: disconnect unregistered service daemon when report received This will allow the service daemon to reconnect and re-register itself as a service daemon without requiring the mgr client to subscribe to MgrStats and detect its removal. Fixes: http://tracker.ceph.com/issues/22286 Signed-off-by: Jason Dillaman (cherry picked from commit c97cca13ab066ed2306708986ff387d903b2486a) commit b97227d34e097e282aa51fe88a7549fe12caaad6 Author: David Zafman Date: Mon Sep 18 14:07:09 2017 -0700 test: Fix ceph-objectstore-tool usage check Caused by: c7b7a1f04f78fa62890c567d0ca53874c8d75eb7 Signed-off-by: David Zafman (cherry picked from commit 0364ae104afc15effadfe968b5f60221704d933c) commit fe956597f2a3f2e9e0b4c2eb80547d4e575f6ee3 Author: David Zafman Date: Tue Jan 23 18:16:32 2018 -0800 test: ceph_objectstore_tool.py bad --filestore option to vstart Luminous doesn't have --filestore option Caused by: fd7d53bacb10d5b2094f1f579c96899d8a0a4b4f Signed-off-by: David Zafman commit 4c211d0d6ce76e592eb9b70445fde6f572fdb4f2 Author: dongdong tao Date: Mon Jan 8 16:00:31 2018 +0800 client: avoid recursive lock in ll_get_vino Fixes: http://tracker.ceph.com/issues/22629 Signed-off-by: dongdong tao (cherry picked from commit fdfbe40c3f16c14dc9c3352f3eea66d34138ee9d) commit dc0e0250d1d5a261b2ee4a29eaa33246d7a7ebcb Author: dongdong tao Date: Tue Jan 9 17:07:11 2018 +0800 cephfs-journal-tool: add usage help info for header set Signed-off-by: dongdong tao (cherry picked from commit 2b06bff950a78b15aca083b20eaa812b4214e8d4) commit 61a64511f99c2d8f30ab046fa72e518fba727058 Author: dongdong tao Date: Tue Jan 9 16:54:40 2018 +0800 mds: add error log info Signed-off-by: dongdong tao (cherry picked from commit a7c2ecd6391d22c5312a9e568ff742cff252749e) commit 4d07eb8ba724c5bfcc7f8f12e6213282f497fb58 Author: dongdong tao Date: Tue Jan 9 16:31:07 2018 +0800 cephfs-journal-tool: add "set pool_id" option Fixes: http://tracker.ceph.com/issues/22631 Signed-off-by: dongdong tao (cherry picked from commit b3a5e313117028b23133f3e0d72e0b6b7159a178) commit 1b2b589a12c2e70c09c44e9f490819c833e1e69d Author: Konstantin Shalygin Date: Wed Jan 17 13:40:05 2018 +0700 mgr: prometheus: added osd commit/apply latency metrics (#22718). Signed-off-by: Konstantin Shalygin (cherry picked from commit 1adf4325455ccdbf03da63046c510bb041ac8520) commit 6c5684c880882e71e262aec4c8d7d4478edb6c98 Merge: f7bfda8fce 7012cf4b60 Author: Yuri Weinstein Date: Tue Jan 23 15:10:49 2018 -0800 Merge pull request #18742 from shinobu-x/wip-21924-luminous luminous: ceph_test_objectstore fails ObjectStore/StoreTest.Synthetic/1 (filestore) buffer content mismatch Reviewed-by: Josh Durgin commit f7bfda8fce31ebf186907400c50f8a13eb64088a Merge: 6bd8b907ae dfebcce5a4 Author: Yuri Weinstein Date: Tue Jan 23 15:10:19 2018 -0800 Merge pull request #18744 from shinobu-x/wip-21922-luminous luminous: Objecter::C_ObjectOperation_sparse_read throws/catches exceptions on -ENOENT Reviewed-by: Josh Durgin commit 6bd8b907aec6414fb04cbfd24ed1d01d66331109 Merge: f5e1d2b160 c281456346 Author: Yuri Weinstein Date: Tue Jan 23 15:09:41 2018 -0800 Merge pull request #18745 from shinobu-x/wip-21921-luminous luminous: Objecter::_send_op unnecessarily constructs costly hobject_t Reviewed-by: Josh Durgin commit 55f4642be2bbdc758378cb58a2d369c14e37bfb1 Author: Jeff Layton Date: Wed Oct 11 11:16:39 2017 -0400 test: make the LibCephFS.ShutdownRacer test even more thrashy Have each thread do the startup and shutdown in a loop for a specified number of times. Tracker: http://tracker.ceph.com/issues/21512 Signed-off-by: Jeff Layton (cherry picked from commit f877e365a42f8f34af6fc3382593ef09101a50d0) commit 963d6fb26565799a08e01577cf15adc0a2f1a956 Author: Jeff Layton Date: Wed Oct 11 11:16:38 2017 -0400 lockdep: free_ids and lock_ref hashes must be truly global It's possible for the teardown of g_lockdep_ceph_ctx to occur, followed by a new context being registered as the lockdep context. When that occurs, we can end up reusing lock id's that were previously handed out to consumers. We need for those IDs to be persistent across lockdep enablement and disablement. Make both the free_ids table, and the lock_refs map persistent across lockdep_unregister_ceph_context and lockdep_register_ceph_context cycles. Entries in those tables will only be deleted by the destruction of the associated mutex. When lockdep_unregister is called, do the refcounting like we normally would, but only clear out the state when the lockid is registered in the lock_names hash. Finally, we do still need to handle the case where g_lockdep has gone false even when there are outstanding references after the decrement. Only log the message if that's not the case. With this, we can deal with the case of multiple clients enabling and disabling lockdep in an unsynchronized way. Tracker: http://tracker.ceph.com/issues/21512 Signed-off-by: Jeff Layton (cherry picked from commit 82fe4e94bcb706434f9215bc8405ff60770cf14f) commit 42bc9058a77ff427dd1dc8a4ba8098c54782a918 Author: Jeff Layton Date: Wed Oct 11 11:16:38 2017 -0400 common: add a clear_g_str_vec() function to clear g_str_vec Prefix str_vec and str_vec_lock with "g_" to make it clear that they are truly global values. Add a new clear_g_str_vec function to allow it to be explicitly cleaned out by callers that need that functionality (mostly testcase for now). Tracker: http://tracker.ceph.com/issues/21512 Signed-off-by: Jeff Layton (cherry picked from commit f72ace0a5666db49bb390ff4ec6808e6e55a779f) commit f5e1d2b1608c17449002f9ab61e0021cf25b256f Merge: 2e0800bd68 5234ef2bca Author: Yuri Weinstein Date: Tue Jan 23 15:08:59 2018 -0800 Merge pull request #18746 from shinobu-x/wip-21916-luminous luminous: msg/async/AsyncConnection.cc: 1835: FAILED assert(state == STATE_CLOSED) Reviewed-by: Josh Durgin commit 8dec6175ab1167bab01af2594ac5d687072f3af2 Author: Jeff Layton Date: Wed Oct 11 11:16:38 2017 -0400 common: make it safe to call env_to_vec multiple times After it has been called once and we have outstanding CephContexts with pointers into str_vec, we can't call get_str_vec on it again. Add a static local mutex to protect access to str_vec. Tracker: http://tracker.ceph.com/issues/21512 Signed-off-by: Jeff Layton (cherry picked from commit 90e898de5f4b8d22f1a6d0e2aedf9e8c50cf72d5) commit eeb62a53c057bdb55e2da014dc7633312ab648b5 Author: Jeff Layton Date: Thu Sep 14 09:28:34 2017 -0400 lockdep: fix Mutex tests to disable lockdep properly ...and make g_lockdep a bool. Signed-off-by: Jeff Layton (cherry picked from commit 0cd0bd778a6149d3e2fe657060e223be3675aed7) commit 864a2c5a2a40806067fa09a1390c3af14c8dd0ef Author: Jeff Layton Date: Thu Sep 14 09:28:33 2017 -0400 client: test shutdown race Spawn threads that bring up a bunch of ceph_mounts with individual CephContext objects, and then tear them down in parallel. Tracker: http://tracker.ceph.com/issues/20988 Signed-off-by: Jeff Layton (cherry picked from commit 8252f3168a1bc90add7f7515c549e9eecffa54b7) commit 8db3ecd6c759251fa8105aeb676e74f9c9302027 Author: Jeff Layton Date: Thu Sep 14 12:22:52 2017 -0400 client: fix signed/unsigned comparison compiler warning The build says: src/client/Client.cc: In member function ‘void Client::trim_caps(MetaSession*, int)’: src/client/Client.cc:4121:22: warning: comparison between signed and unsigned integer expressions [-Wsign-compare] if (s->caps.size() > max) ~~~~~~~~~~~~~~~^~~~~ Signed-off-by: Jeff Layton (cherry picked from commit e057b6770b5545f327990cbaf17e7a391e3e8a50) commit 49ac68c3f95a0bdccb7a6c2f1706a3784ffc706a Author: Jeff Layton Date: Thu Sep 14 09:28:34 2017 -0400 lockdep: fix races with concurrent lockdep teardown If the cct is unregistered while other threads are flogging mutexes, then we can hit all sorts of bugs. Ensure that we handle that situation sanely, by checking that g_lockdep is still set after we take the lockdep_mutex. Also, remove an assertion from lockdep_unregister, and just turn it into an immediate return. It's possible to have a call to lockdep_unregister_ceph_context, and then a call to lockdep_register_ceph_context while a mutex is being held by another task. In that case, it's possible the lock does not exist in the map when we go to unregister it. That's not a bug though, just a natural consequence of that series of actions. Tracker: http://tracker.ceph.com/issues/20988 Signed-off-by: Jeff Layton (cherry picked from commit 75f41a95782a7ee83a243d91963e8d591402f8a6) commit b66ba62822bc7935d60a4424cedbfe7a845631c6 Author: Jeff Layton Date: Thu Sep 14 09:28:34 2017 -0400 lockdep: don't take lockdep_mutex twice for new lock registrations We can do it under the same mutex, which should be more efficient. Signed-off-by: Jeff Layton (cherry picked from commit 01863bb6fb62ea89aa3e21e43bf4dc4f3da9cfcb) commit 2e0800bd68234a32edb1f23c33402252fa6d200a Merge: ea771cbd92 8941606f5e Author: Yuri Weinstein Date: Tue Jan 23 15:07:07 2018 -0800 Merge pull request #18749 from shinobu-x/wip-21785-luminous luminous: OSDMap cache assert on shutdown Reviewed-by: Josh Durgin Reviewed-by: Greg Farnum commit ea771cbd926656db371ca6fa08ed172c76a88195 Merge: b03323c80c b36bd48204 Author: Yuri Weinstein Date: Tue Jan 23 15:06:19 2018 -0800 Merge pull request #18753 from shinobu-x/wip-21697-luminous luminous: OSDService::recovery_need_sleep read+updated without locking Reviewed-by: Josh Durgin commit b03323c80c4759bece12748268f16b8abe1197a6 Merge: a0b489712d 5c9d212f94 Author: Yuri Weinstein Date: Tue Jan 23 14:52:13 2018 -0800 Merge pull request #18715 from shinobu-x/wip-luminous-21964 luminous: ceph_test_cls_log failures related to cls_cxx_subop_version() Reviewed-by: Casey Bodley commit a0b489712dbcc95c2f8cec8f8988acb532947385 Merge: c8d526b8a8 b6f50a463a Author: Yuri Weinstein Date: Tue Jan 23 14:51:37 2018 -0800 Merge pull request #18732 from shinobu-x/wip-luminous-21875 luminous: ceph-mgr spuriously reloading OSD metadata on map changes Reviewed-by: John Spray commit c8d526b8a8b0eb5fa545c25210e98625232cb8a8 Merge: 029594a080 f0914619a5 Author: Yuri Weinstein Date: Tue Jan 23 14:50:34 2018 -0800 Merge pull request #18735 from shinobu-x/wip-22029-luminous luminous: restarting active ceph-mgr cause glitches in bps and iops metrics Reviewed-by: Kefu Chai Reviewed-by: Sage Weil commit a09a42a4a850ee6f5e64fc0dc79995dbbfbdf13d Author: David Zafman Date: Mon Oct 2 13:51:17 2017 -0700 test: Test case that reproduces tracker 18162 recover_replicas: object added to missing set for backfill, but is not in recovering, error! Signed-off-by: David Zafman (cherry picked from commit b9de5eec267627c8bc5ff0759ddea6c4a8aa7bce) commit 23f7c556d9ad903ad928e6193b50f5a61106429a Author: David Zafman Date: Tue Oct 3 18:32:20 2017 -0700 osd: Better handle failure to get enough EC shards to backfill Fixes: http://tracker.ceph.com/issues/18162 Signed-off-by: David Zafman (cherry picked from commit 6a02bfef3d44a13589c1a90bec29ff0ac64f97aa) commit 081fe72398633b66b93bc961cfd7d8599d5d0cba Author: David Zafman Date: Thu Sep 21 20:35:27 2017 -0700 osd: For recovery get all possible shards to read on errors Signed-off-by: David Zafman (cherry picked from commit 390d12f71a1bd6e07f3516b1c73e467e9960725d) commit 5bed56a620f8c1c333f900f72d818cdec0f6381a Author: David Zafman Date: Wed Sep 27 14:29:25 2017 -0700 test: Use feature to get last array element Signed-off-by: David Zafman (cherry picked from commit 43e3206de2a20bf17ad9f2b16b7e486804097bf2) commit 62e9c05af9c7b1de44ebe4e055e84a419e0542b8 Author: David Zafman Date: Thu Sep 21 17:48:28 2017 -0700 osd: Allow recovery to send additional reads For now it doesn't include non-acting OSDs Added test for this case Signed-off-by: David Zafman (cherry picked from commit 1235810c2ad08ccb7ef5946686eb2b85798f5bca) commit bfd0029910e33840e6ef5d87bb48b222c57bad03 Author: David Zafman Date: Thu Sep 21 14:49:17 2017 -0700 test: Allow modified options to existing setup functions Signed-off-by: David Zafman (cherry picked from commit f92aa6c82449152901a4663c523e9ba43363eca3) commit 39bb3cf8e688a4fd4bc66295c39c66554cb8e089 Author: David Zafman Date: Tue Sep 12 15:09:14 2017 -0700 osd: CLEANUP: Change getattr_maybe_cache() to reflect how it is being used Signed-off-by: David Zafman (cherry picked from commit 143111a112ec64724226f83ed285f66ee52204b2) commit a7a2de03ffcbb829c6ce6f95c5e20fb14ad1a3af Author: David Zafman Date: Tue Sep 12 15:03:33 2017 -0700 test: Add a removal test for erasure code read Test feature: http://tracker.ceph.com/issues/14513 Signed-off-by: David Zafman (cherry picked from commit 50e08b0a5d7dc30a164313c1480b6cc829b6ec65) commit 7522ddd97845b8c2a15ad81a30b46a69e6f1444c Author: Sage Weil Date: Mon Oct 23 16:07:48 2017 -0500 mon/PGMap: 'unclean' does not imply damaged Everything (that I can think of) that would lead to a PG being unclean is already reported via another health message. And there are cases where a PG is unclean (e.g., because it is backfilling) where we are not degraded. Fix by ignoring this flag in the health checks. Signed-off-by: Sage Weil (cherry picked from commit 89e4cb90af3c193fe436390b5e3288701e6fc4af) commit 67fbd3830f2681c49b65b0ac93eccd58c219114c Author: Casey Bodley Date: Tue Jan 23 10:49:31 2018 -0500 boost: include more specific asio headers when building with gcc5 and boost 1.66, this #include fails to compile: In file included from build/boost/include/boost/asio/impl/write.hpp:25:0, from build/boost/include/boost/asio/write.hpp:927, from build/boost/include/boost/asio/buffered_write_stream.hpp:29, from build/boost/include/boost/asio/buffered_stream.hpp:22, from build/boost/include/boost/asio.hpp:41, from src/common/Graylog.h:7, from src/common/LogClient.cc:20: build/boost/include/boost/asio/detail/consuming_buffers.hpp: In member function ‘boost::asio::detail::consuming_buffers::prepared_buffers_type boost::asio::detail::consuming_buffers::prepare(std::size_t)’: build/boost/include/boost/asio/detail/consuming_buffers.hpp:105:50: error: parse error in template argument list while (next != end && max_size > 0 && result.count < result.max_buffers) ^ src/CMakeFiles/common-objs.dir/build.make:135: recipe for target 'src/CMakeFiles/common-objs.dir/common/LogClient.cc.o' failed see also: https://svn.boost.org/trac10/ticket/13368 this commit targets the luminous branch specifically because the issues doesn't reproduce on master (as we require gcc7+) Signed-off-by: Casey Bodley commit 660abda21163bf4297b325ea2ac42a28661d5f62 Author: Casey Bodley Date: Tue Oct 24 17:55:34 2017 -0400 rgw: more documentation for bilog trim config Signed-off-by: Casey Bodley (cherry picked from commit 4d4442168c15775f8346ebc6295a6de912250b8f) commit 2ed35c9f922409ad5e5e1e199687d25977b35cf6 Author: Casey Bodley Date: Tue Oct 24 16:54:59 2017 -0400 rgw: fix notify timeout for BucketTrimWatcher from seconds to msec Signed-off-by: Casey Bodley (cherry picked from commit ada2d108e627d519794e1a6c08af23cecbc5be79) commit 3f06276fc80696f693cc754b49351d5f16b76818 Author: Casey Bodley Date: Tue Oct 24 16:51:39 2017 -0400 rgw: curly brace style for bilog trim classes Signed-off-by: Casey Bodley (cherry picked from commit a0c438eefcf0c99094efa0d98902cc4fca5957f0) commit 6fc4c7a12345b59cf3e14546006bbe7941eb98a3 Author: Casey Bodley Date: Tue Oct 24 14:45:12 2017 -0400 rgw: BucketTrimWatcher checks handle in stop Signed-off-by: Casey Bodley (cherry picked from commit 6e4d9735036a68575696e53b6732e4680a0226d6) commit 24f7fd2cf9efedf86b214c4d5f45113eed0a2f80 Author: Casey Bodley Date: Tue Oct 24 12:23:41 2017 -0400 rgw: hold cr reference in MetadataListCR callback Signed-off-by: Casey Bodley (cherry picked from commit 3a1474031e4266100c5ccaf082756bdf0607a2be) commit 32ebc0e546903a3129defcee76541ed613bba195 Author: Casey Bodley Date: Tue Oct 24 12:23:08 2017 -0400 rgw: add comment for bucket in BucketCounter Signed-off-by: Casey Bodley (cherry picked from commit c6d946b3e1fbb10da43273e62c8584ffe3afbcdf) commit d1620fef15d5dff8ebdfe26af0b37f0f24a6046a Author: Casey Bodley Date: Tue Oct 24 11:08:24 2017 -0400 rgw: move shard marker helper into BucketIndexShardsManager Signed-off-by: Casey Bodley (cherry picked from commit 4be7039964a2d17069b3604a4c994743145a013e) commit 668c2833ace4af4cb805e4d5ae38b22b6e03473a Author: Casey Bodley Date: Tue Oct 10 15:05:01 2017 -0400 qa/rgw: add kwargs for debug output Signed-off-by: Casey Bodley (cherry picked from commit fa8a0713de6b5b8ea9c9400f3ed467d87940612a) commit f1bcf84000c43772cc9e89040356fef6c3b464cc Author: Casey Bodley Date: Wed Sep 20 10:01:17 2017 -0400 test/rgw: add test_bucket_index_log_trim() Signed-off-by: Casey Bodley (cherry picked from commit d9dc58cf2c6f64562843cc1d00df82d99379e640) commit 96baf2d00fe111b131f4fd9fa96e56442ee8b359 Author: Casey Bodley Date: Fri Sep 15 15:54:44 2017 -0400 rgw: add TrimComplete to watch/notify api Signed-off-by: Casey Bodley (cherry picked from commit 916f5995c92acd0c5ba66fde6031fed7cb91cb95) commit 7c430a08afaf6700cf4e64b7f7333e6ceafab2be Author: Casey Bodley Date: Fri Sep 15 14:48:43 2017 -0400 rgw: RGWDataSyncSingleEntryCR calls BucketChangeObserver Signed-off-by: Casey Bodley (cherry picked from commit 1c50d727b5df574e28d90cd99abe15db1742e4b1) Conflicts: sync tracing not backported src/rgw/rgw_data_sync.cc src/rgw/rgw_data_sync.h commit 07a122a3fda26c7b65ecf0879f24a8380568b47f Author: Igor Fedotov Date: Wed Jan 3 14:16:19 2018 +0300 common/throttle: start using 64-bit values Signed-off-by: Igor Fedotov (cherry picked from commit fa37ed1a48fd804ac199509bd78c470480ecbb22) Conflicts: src/common/Throttle.cc : Resolved in _reset_max and put func src/common/Throttle.h : Retained only count var changes commit e3fbaa92cedb101f79c7e0712017662f6d2ec1cf Author: Matt Benjamin Date: Fri Jan 19 13:05:27 2018 -0500 rgw_file: alternate fix deadlock on lru eviction This change is an alternate fix for two problems found and fixed by Yao Zongyou . The deadlock can be avoided just by not taking it in the recycle case, which invariantly holds the lock. The invalidation of the insert iterator by the recyle-path unlink we'd like to handle as a condition in order to preserve the cached insertion point optimization we get in the common case. (The original behavior was, indeed, incorrect.) Based on feedback from Yao, removed the RGWFileHandle dtor version of the unlink check, which I think happened twice. Signed-off-by: Matt Benjamin (cherry picked from commit 3cf0880f86b8f7911139c4e3d672cf47420c8f49) commit 35d2baf47b2ad29c961da2f861b4d71b65835b00 Author: Tao Chen Date: Fri Dec 1 16:42:44 2017 +0800 librgw: fix shutdown err with resources uncleaned Fixed: http://tracker.ceph.com/issues/22296 Signed-off-by: Tao Chen (cherry picked from commit 0fa2be059e649684a4fe15a456d564acde8e7831) commit 238ddccbbc4ae83e1cd49d2522bf1d43d62ae4f9 Author: David Zafman Date: Thu Nov 9 16:11:14 2017 -0800 ceph-objectstore-tool: Fix set-size to clear data_digest if changing disk size Signed-off-by: David Zafman (cherry picked from commit 7ca3ce93099de9412012b7a88121f9fced34f4b8) commit a0c0e637fe276aa1b6223522ca00a810e83d5084 Author: Sage Weil Date: Wed Jan 17 10:38:29 2018 -0600 osd: only exit if *latest* map(s) say we are destroyed It's possible our current map is older, we were destroyed then, but in newer maps our osd was recreated. This happens when the oldest map after a recreated osd happens to land on an epoch where the osd was marked destroyed. Fix by only exiting if one of the newest maps says we are (still) destroyed. Fixes: http://tracker.ceph.com/issues/22673 Signed-off-by: Sage Weil (cherry picked from commit 4e4d6466aab259eb0436989401b80b6c3cf87a7c) commit 21aef40b727ddfd7fba4e4c341ddd1cf2e21c404 Author: Casey Bodley Date: Fri Sep 15 14:09:14 2017 -0400 rgw: start BucketTrimManager in RGWRados Signed-off-by: Casey Bodley (cherry picked from commit 74aedaef0757f56a77671d6de59628538cfa0bbb) commit 0d86081fcd74c85d2834aa7e7279a1fdf26e0c04 Author: Casey Bodley Date: Thu Sep 7 16:27:53 2017 -0400 rgw: add BucketTrimInstanceCR fetches bucket sync status from each peer, calculates the min markers for each shard, and trims the bilog shards. calls the TrimObserver on success Fixes: http://tracker.ceph.com/issues/18229 Signed-off-by: Casey Bodley (cherry picked from commit 35cf54e1059bd0a46e99fecc191c117f656c7b02) commit da126d8fe2637f09688e343edb7ddaf4d0f60b37 Author: Casey Bodley Date: Thu Sep 7 16:24:13 2017 -0400 rgw: add HTTPManager to BucketTrimManager Signed-off-by: Casey Bodley (cherry picked from commit 04400bc60b1cf9efbc49761cba8311e0a95dfc9e) commit 6b41f30976c292dce92a5a19f0949778c55aed56 Author: Casey Bodley Date: Thu Sep 7 16:17:46 2017 -0400 rgw: RGWGetBucketInstanceInfoCR takes rgw_bucket or metadata key Signed-off-by: Casey Bodley (cherry picked from commit 689badbe7fb468d4aec14e2f86bfc4cce9c103a2) commit aba2e0b64c0bc9e8dde20296a258b0d9850b5f72 Author: Casey Bodley Date: Thu Sep 14 16:50:47 2017 -0400 rgw: add RGWRadosBILogTrimCR Signed-off-by: Casey Bodley (cherry picked from commit 0dea44dad3a572cd966d22dd345c1e226c073f6e) commit 1efe55f46ea366233a8aaeb73ea58add944dfea0 Author: Casey Bodley Date: Thu Sep 14 16:51:13 2017 -0400 rgw: add RGWBucketInfo overload for BucketShard::init for use by callers that have already read the bucket instance info Signed-off-by: Casey Bodley (cherry picked from commit f0caa1ca8a1d502128fd45237c1e7c13118a9711) commit 8dc99bbd6931dc47a6197c07e6a6dde383edc0cb Author: Casey Bodley Date: Thu Sep 7 15:26:35 2017 -0400 rgw: add /admin/log rest api for bucket sync status Signed-off-by: Casey Bodley (cherry picked from commit fab595ab9bfe7f477ab03efee66d1a335d91f7cb) commit 001f55774cdef304f1d4ec5a7b4a424ad697d0ae Author: Casey Bodley Date: Thu Sep 7 16:17:00 2017 -0400 rgw: add json decoders for bucket sync status Signed-off-by: Casey Bodley (cherry picked from commit 28c6ac62b877f0eafc94facb77255bfe81f48c38) commit 2c71e5cdda0eb2bfe871653a40b95627546f7612 Author: Casey Bodley Date: Tue Sep 12 12:48:56 2017 -0400 rgw: add rgw_bucket_sync_status() to bypass manager RGWBucketSyncStatusManager::init() is doing a lot of extra work that's not needed to serve the rest api (spawning an http manager thread, fetching the bucket instance info, etc) uses RGWShardCollectCR to limit the number of concurrent reads to 16 Signed-off-by: Casey Bodley (cherry picked from commit 403d139c673180ce00eab420a2fdebdacf366dac) commit b26e997d0a57f9845ed1bc192e0b92c4307da07f Author: Casey Bodley Date: Wed Sep 6 12:59:03 2017 -0400 rgw: add 'radosgw-admin bilog autotrim' Signed-off-by: Casey Bodley (cherry picked from commit 4f736f1ec74f74e8dc1ae4ffb5b71274351c66f0) commit f65ad4a3fa9637c5eb69422d95386dd7ed2a3888 Author: Casey Bodley Date: Thu Sep 7 12:48:47 2017 -0400 rgw: add configure_bucket_trim() Signed-off-by: Casey Bodley (cherry picked from commit d29f96ae3e1b7d65b2f513340e8c2c42cf6de9f1) commit b72f8aefc277b607d6207eac1586de85e02a8f4d Author: Casey Bodley Date: Fri Sep 1 11:06:30 2017 -0400 rgw: BucketTrimManager implements BucketTrimObserver Signed-off-by: Casey Bodley (cherry picked from commit 7be4eab8a339e9e083352a44ad09272da717c73e) commit e7b3a636a76daa60dc12e1aee02b6a7ccfb9a779 Author: Casey Bodley Date: Fri Sep 1 12:40:58 2017 -0400 rgw: collect cold buckets for trim Signed-off-by: Casey Bodley (cherry picked from commit 06a22a134f9af92753fa206eb64025472ec94f40) commit d13f6a10fa3d5a92ed1a676cece453537af1a374 Author: Casey Bodley Date: Fri Sep 1 12:37:56 2017 -0400 rgw: add BucketTrimStatus Signed-off-by: Casey Bodley (cherry picked from commit 2c07d7dd0e6c358bcdba409747ebf13f846a77e1) commit fbc356a2df53368c2fb72d34f6d59faae2eda7a0 Author: Casey Bodley Date: Thu Sep 7 12:12:43 2017 -0400 rgw: add MetadataListCR to loop over bucket instances Signed-off-by: Casey Bodley (cherry picked from commit c9d50860b735809e343a0831ba3d346b0b37973c) commit f746e86bcade6edf1374578b3b64d2f5b9594b88 Author: Casey Bodley Date: Fri Sep 1 12:31:44 2017 -0400 rgw: add BucketTrimCR to spawn trim for active buckets Signed-off-by: Casey Bodley (cherry picked from commit 82c059530871af054c48de0e6a091b1f74f2eb12) commit 8434b7bd3e85352b24f8dff08f805556115c58e8 Author: Casey Bodley Date: Fri Sep 1 12:22:47 2017 -0400 rgw: add BucketTrimPollCR for interval and lease logic Signed-off-by: Casey Bodley (cherry picked from commit 129fc99d5208279029ff1722d21f0ad24c37db62) commit 1d8dbaebca536cbf0341d4d0284dcbad191e64ad Author: Casey Bodley Date: Fri Sep 1 11:29:55 2017 -0400 rgw: add TrimCounters api to BucketTrimWatcher Signed-off-by: Casey Bodley (cherry picked from commit 5bcf109eac30780cfa9ae5d524d2bde638651f40) commit e34da7e6b35ed29560189e8ccf7d2c8f46861df5 Author: Casey Bodley Date: Fri Sep 1 11:26:01 2017 -0400 rgw: add BucketTrimWatcher to serve watch/notify apis Signed-off-by: Casey Bodley (cherry picked from commit f96d9a8e22195bfb3347b5add7a4385895d36f9c) commit fb7e2c0e6377a1f3bd3f32a0fe81a76e7ac53ea2 Author: Casey Bodley Date: Fri Sep 1 10:57:41 2017 -0400 rgw: BucketTrimManager implements BucketChangeObserver Signed-off-by: Casey Bodley (cherry picked from commit b4249cc432b5b74e5c8f545bdc7daddbc9e60d23) commit 92c9fd9521aac1fed4f2bf6d991a62280351c9a6 Author: Casey Bodley Date: Thu Aug 24 10:01:36 2017 -0400 common: introduce BoundedKeyCounter and unit test Signed-off-by: Casey Bodley (cherry picked from commit e9a5ec9f64dd6cd163a855335b846181c3ac83d2) Conflicts: src/test/common/CMakeLists.txt (add_ceph_unittest) commit 3aba3f5975c7abaac104732155f567e2ec2685b7 Author: Casey Bodley Date: Fri Sep 1 10:51:57 2017 -0400 rgw: add skeleton for BucketTrimManager Signed-off-by: Casey Bodley (cherry picked from commit 4309adb36be8dff737ab2196f59743c316b12bca) Conflicts: src/rgw/CMakeLists.txt (sync tracing not backported) commit 82c1b7d75a2b4486e8c41440deafa6326641e73e Author: Casey Bodley Date: Wed Aug 30 16:19:36 2017 -0400 rgw: introduce RGWRadosNotifyCR for aio_notify Signed-off-by: Casey Bodley (cherry picked from commit 964d966969bf757570bffb650ebb4ef515a6592e) commit 7308044ac932ada012b22ce9b0ab65ce70d9bec9 Author: Casey Bodley Date: Tue Aug 29 15:51:56 2017 -0400 rgw: MetadataManager interface takes const string refs Signed-off-by: Casey Bodley (cherry picked from commit b3a5c5a0569f95a5a755ea0dc03ec2c1ea4bdc4d) commit 7d6984b554158f8d5c3bae0223bfdf740ab9926d Author: Jason Dillaman Date: Mon Jan 22 10:34:57 2018 -0500 qa/suites/upgrade: disable broken kraken upgrade cls_rbd test Fixes: https://tracker.ceph.com/issues/22740 Signed-off-by: Jason Dillaman commit e0b85124193882fb1eb04fec0ab2d907523a2dde Author: Matt Benjamin Date: Fri Jan 19 13:05:27 2018 -0500 rgw_file: alternate fix deadlock on lru eviction This change is an alternate fix for two problems found and fixed by Yao Zongyou . The deadlock can be avoided just by not taking it in the recycle case, which invariantly holds the lock. The invalidation of the insert iterator by the recyle-path unlink we'd like to handle as a condition in order to preserve the cached insertion point optimization we get in the common case. (The original behavior was, indeed, incorrect.) Based on feedback from Yao, removed the RGWFileHandle dtor version of the unlink check, which I think happened twice. Signed-off-by: Matt Benjamin (cherry picked from commit 3cf0880f86b8f7911139c4e3d672cf47420c8f49) commit e8822fe18f68eefd8043e36dd803bda1c2aa4f0c Author: David Zafman Date: Tue Jan 16 10:44:29 2018 -0800 tests: recovery-unfound-found test needs to account for correct misplaced calculations The test expected HEALTH_OK when in a state with misplaced objects therefore HEALTH_WARN Signed-off-by: David Zafman (cherry picked from commit 9f103f013c08bb215f58247f75d33f4f95bb3719) commit 40ecfe282e3f6584cb7f61ffbe99ea7cde7dfa18 Author: David Zafman Date: Wed Jan 10 13:30:41 2018 -0800 osd: Don't start recovery for missing until active pg state set I was seeing recovery hang when it is started before _activate_committed() The state machine passes into "Active" but this transitions to activating pg state and after commmitted into "active" pg state. Signed-off-by: David Zafman (cherry picked from commit 64047e1bac2e775a06423a03cfab69b88462538c) commit 02660d92390b4e4ba3ce46ccc1eba7bf0644460b Author: Sage Weil Date: Thu Nov 16 14:26:27 2017 -0600 osd/PG: restart recovery if NotRecovering and unfound found If we are in recovery_unfound state waiting for unfound objects, and we find them, we need to restart the recovery reservation process so that we can recover. Do this by queueing DoRecover() event instead of calling queue_recovery() (which won't do anything since we're not in recoverying|backfilling pg states). Make the parent Active state ignore DoRecovery so that if we are already in some phase of recovery/backfill the event gets ignored. It is already handled by the other important substates that care, like Clean (for repair's benefit). I'm not sure why states like Activating are paying attention tot his vevent... Fixes: http://tracker.ceph.com/issues/22145 Signed-off-by: Sage Weil (cherry picked from commit 4cfe31c63b519f2dce22f061c9951c302f6efb1e) Conflicts: src/osd/PG.h (trivial, not all events exist in luminous) commit 673df31fe2fa6244b4327e646e47605aa482aa43 Author: Sage Weil Date: Thu Nov 16 15:20:01 2017 -0600 qa/suites/rados: test for recovery_unfound bug See http://tracker.ceph.com/issues/22145 Signed-off-by: Sage Weil (cherry picked from commit 25b7965f8803ae68759973f4afe515d3da9c0f3f) commit f4f25becde3d83ea10a02e2fb19ec8bfe1d792c9 Author: Sage Weil Date: Thu Nov 16 15:11:45 2017 -0600 osd/PG: document state hierarchy Signed-off-by: Sage Weil (cherry picked from commit e2a75c91dd21f362d0b7fb6acf6a360beefd4168) commit 23a08fce7db373cfed8e69b379f82654110e436a Author: Sage Weil Date: Tue Dec 12 09:22:37 2017 -0600 osd/PG: include primary in PG operator<< for ec pools Otherwise it is confusing! Signed-off-by: Sage Weil (cherry picked from commit 0d98f262c0894b11b77deb3a4bf494f20be486c3) commit 029594a080b5f02c47da4f08ab4b99c77ddb75a5 Merge: 1a69f3ed7c 747d05c298 Author: Sage Weil Date: Mon Jan 22 09:02:32 2018 -0600 Merge pull request #19257 from ifed01/wip-ifed-rocksdb-fix luminous: rocksdb: fixes early metadata spill over to slow device in commit ac064e1fc846a5a0db0062e6acfca3e0c623137c Author: Sage Weil Date: Tue Oct 24 15:20:49 2017 -0500 ceph_test_objectstore: do not change model for 0-length zero We almost fixed this in the pr merged at 2dbbb351e2e0e3703880023bf51f55790f763e04 but missed this piece. Signed-off-by: Sage Weil (cherry picked from commit 390779d1aa9b46648e4f5390dc431c255c70385d) commit 6a691bb79db67ebe58e2d5179deaa1d7f17e391d Author: Sage Weil Date: Fri Oct 6 15:29:32 2017 -0500 os/bluestore: 0-length zero should not change object size Fixes: http://tracker.ceph.com/issues/21712 Signed-off-by: Sage Weil (cherry picked from commit b588eaf2b0fdf06c94104d5a542bd571499f2b85) commit 445f9b52face0d44fb386bbb9a9704ad803322fa Author: Sage Weil Date: Fri Oct 6 15:29:15 2017 -0500 os/filestore: make 0-length zero avoid touching object length Fixes: http://tracker.ceph.com/issues/21712 Signed-off-by: Sage Weil (cherry picked from commit 979b7c9cd2e4919b8c7d5ed2418019310959f4bf) commit a146e49c51d3ed0d82a293bd5b809e4e97961732 Author: Sage Weil Date: Fri Oct 6 15:28:49 2017 -0500 os/ObjectStore: 0-length zero does not change length of object Signed-off-by: Sage Weil (cherry picked from commit 9ad1f4f10ff7bfe32d0a37361640fe5c65e56699) commit 2260f03b360e8226bf5875c5840dabc07080d174 Author: Sage Weil Date: Wed Jan 17 11:23:39 2018 -0600 mon/OSDMonitor: squelch 'id' arg errors Several commands use 'id' but for other types (e.g., vector). Signed-off-by: Sage Weil (cherry picked from commit 0580f0125bc014e5d97b8317a1bd53a3495f03bf) Conflicts: src/mon/OSDMonitor.cc: pass g_ceph_context instead of cct to cmd_getval() to be consistent with other callsites of this function. we can do the s/g_ceph_context/cct/ in another commit, if it turns out to be a critical fix. commit e07090345249bba6d3720bff62ab13641b1a9c38 Author: Sage Weil Date: Wed Jan 17 11:22:55 2018 -0600 use stringsream::str(), not rdbuf() Fixes: http://tracker.ceph.com/issues/22715 Signed-off-by: Sage Weil (cherry picked from commit 918361e92518056bf89c3d19a03e9398c7a74b97) commit 1a69f3ed7c4b13940848a32bbf474f8b6b77f3bf Merge: 7a4b42185b 58d6d043b2 Author: Sage Weil Date: Fri Jan 19 15:10:19 2018 -0600 Merge pull request #19071 from smithfarm/wip-22181-luminous luminous: rgw segfaults after running radosgw-admin data sync init Reviewed-by: Casey Bodley Reviewed-by: Abhishek Lekshmanan commit 7a4b42185b8948fc3d078ace811cb3fc9178a396 Merge: 6d56c2fdef 23b3c9de3d Author: Sage Weil Date: Fri Jan 19 15:09:45 2018 -0600 Merge pull request #19077 from smithfarm/wip-22171-luminous luminous: rgw: log keystone errors at a higher level Reviewed-by: Casey Bodley commit 6d56c2fdef336f8ce1034a67cd18758690055827 Merge: 63a05442c3 5b9f1281a3 Author: Casey Bodley Date: Fri Jan 19 16:08:03 2018 -0500 Merge pull request #19810 from adamemerson/wip-luminous-21901 luminous: rgw: bucket policy evaluation logical error Reviewed-by: Casey Bodley commit 63a05442c33d191786ff781dba8fd47b5297b0c2 Merge: e24edea8d9 9db61c8b14 Author: Casey Bodley Date: Fri Jan 19 16:06:08 2018 -0500 Merge pull request #19847 from adamemerson/wip-22541-luminous luminous: rgw: put bucket policy panics RGW process Reviewed-by: Casey Bodley commit e24edea8d94639f6fb2da5c75dea397b2e06e258 Merge: ca5ae981f6 c49417d8b0 Author: Yuri Weinstein Date: Fri Jan 19 13:03:58 2018 -0800 Merge pull request #19785 from linuxbox2/luminous-22410 luminous: rgw: fix chained cache invalidation to prevent cache size growth Reviewed-by: Adam Emerson commit ca5ae981f65da0a2f8213cd3ea52cb37db432e96 Merge: f2c5146f40 34864fe7ba Author: Yuri Weinstein Date: Fri Jan 19 13:02:34 2018 -0800 Merge pull request #19788 from adamemerson/wip-luminous-cache luminous: rgw: Random 500 errors in Swift PutObject (needs cache fixes) Reviewed-by: Nathan Cutler Reviewed-by: Abhishek Lekshmanan commit f2c5146f409ab4f51b17ae311ad8090ea3ccf5b7 Merge: e002a82db8 b9bf9b1255 Author: Casey Bodley Date: Fri Jan 19 16:01:40 2018 -0500 Merge pull request #19053 from shinobu-x/wip-22187-luminous luminous: rgw: add cors header rule check in cors option request Reviewed-by: Casey Bodley commit e002a82db83f20812f6aa1c70165f1275c4efff7 Merge: ccd4b53800 5bf8d71001 Author: Yuri Weinstein Date: Fri Jan 19 13:01:10 2018 -0800 Merge pull request #19819 from adamemerson/wip-backport-22601 luminous: rgw: S3 API Policy Conditions IpAddress and NotIpAddress do not work Reviewed-by: Abhishek Lekshmanan Reviewed-by: Nathan Cutler commit ccd4b538003acee14dfb9f72f5c39a38ab6ec12b Merge: 21ff007c65 274fef45c2 Author: Casey Bodley Date: Fri Jan 19 16:00:56 2018 -0500 Merge pull request #19050 from shinobu-x/wip-22184-luminous luminous: Dynamic bucket indexing, resharding and tenants seems to be broken Reviewed-by: Casey Bodley commit 21ff007c656327fe4bfcdb723bb97fdb64e26985 Merge: 599d6a1681 90b56a2903 Author: Casey Bodley Date: Fri Jan 19 16:00:38 2018 -0500 Merge pull request #19085 from shinobu-x/wip-22215-luminous luminous: rgw: bucket index object not deleted after radosgw-admin bucket rm --purge-objects --bypass-gc Reviewed-by: Casey Bodley commit 599d6a16814965d721259bac94ed92fdb1964ce5 Merge: 7a686ddb09 a88e48e7d9 Author: Casey Bodley Date: Fri Jan 19 15:54:24 2018 -0500 Merge pull request #18870 from shinobu-x/wip-21949-luminous luminous: rgw: null instance mtime incorrect when enable versioning Reviewed-by: Casey Bodley commit 7a686ddb093b4198ba44501ab0d5993d868c907f Merge: c143de468a 860625046d Author: Casey Bodley Date: Fri Jan 19 15:54:05 2018 -0500 Merge pull request #18764 from smithfarm/wip-22017-luminous luminous: rgw: Segmentation fault when starting radosgw after reverting .rgw.root Reviewed-by: Casey Bodley commit c143de468a9a279e95e37d65354733bdb3895a20 Merge: 92a1908c88 cb38378e5d Author: Casey Bodley Date: Fri Jan 19 15:53:53 2018 -0500 Merge pull request #18765 from smithfarm/wip-22024-luminous luminous: RGWCrashError: RGW will crash if a putting lc config request does not include an ID tag in the request xml Reviewed-by: Casey Bodley commit 92a1908c88bb862c13363122851acc5739ea6bc1 Merge: 11a3dae295 ed87de0622 Author: Casey Bodley Date: Fri Jan 19 15:53:40 2018 -0500 Merge pull request #18766 from smithfarm/wip-22021-luminous luminous: rgw: modify s3 type subuser access permission fail Reviewed-by: Casey Bodley commit 11a3dae29509ceb00bcc93da94b9873cc457229f Merge: 7ae70b8b6a dbd70f0b82 Author: Casey Bodley Date: Fri Jan 19 15:53:28 2018 -0500 Merge pull request #18867 from shinobu-x/wip-22027-luminous luminous: multisite: destination zone does not compress synced objects Reviewed-by: Casey Bodley commit 7ae70b8b6a3efc89895e8fdf74e929a1d41eaf1c Merge: 262bd71520 2dc009e68e Author: Yuri Weinstein Date: Fri Jan 19 12:48:23 2018 -0800 Merge pull request #18868 from shinobu-x/wip-22026-luminous luminous: Policy parser may or may not dereference uninitialized boost::optional sometimes Reviewed-by: Adam Emerson commit 262bd71520169871cbf5cf050bf4dd6eeee4f7da Merge: 14ca29d5cf eb0c60f7d7 Author: Casey Bodley Date: Fri Jan 19 15:46:58 2018 -0500 Merge pull request #19538 from shinobu-x/wip-22434-luminous luminous: rgw: user stats increased after bucket reshard Reviewed-by: Casey Bodley commit 14ca29d5cfc6c088c39bb47b26d2a9533f552f48 Merge: b26aeac2c4 4085e87b84 Author: Casey Bodley Date: Fri Jan 19 15:46:21 2018 -0500 Merge pull request #19489 from dplyakin/luminous-get-website-error-code luminous: rgw: fix GET website response error code Reviewed-by: Casey Bodley commit b26aeac2c462e47d91f9847a25e79ae5e6b9b0e2 Merge: 370abb546d 05b60db8ef Author: Yuri Weinstein Date: Fri Jan 19 12:44:55 2018 -0800 Merge pull request #19090 from pdvian/wip-22179-luminous luminous: rgw: Fix swift object expiry not deleting objects Reviewed-by: Casey Bodley Reviewed-by: Amit Kumar commit 370abb546d871d8cb72f61bc7d74a547b90e4112 Merge: 85558065d4 b013f7fdb5 Author: Yuri Weinstein Date: Fri Jan 19 12:43:07 2018 -0800 Merge pull request #19434 from cbodley/wip-luminous-pr-16807 luminous: rgw: revert PR #16807 Reviewed-by: Abhishek Lekshmanan commit 85558065d46dde2d1d26b2b159b09e77f73de432 Merge: eaedb0f06c 27a1a7f6fa Author: Yuri Weinstein Date: Fri Jan 19 12:41:22 2018 -0800 Merge pull request #19502 from shinobu-x/wip-22397-luminous luminous: rgw: radosgw-admin reshard command argument error. Reviewed-by: Orit Wasserman commit eaedb0f06cbf1287c636cb0a3394179edd049506 Merge: ad11a8e9b2 72de713bf6 Author: Yuri Weinstein Date: Fri Jan 19 12:40:45 2018 -0800 Merge pull request #19506 from shinobu-x/wip-22388-luminous luminous: rgw: 501 is returned When init multipart is using V4 signature and chunk encoding Reviewed-by: Radoslaw Zarzynski Date: Tue Nov 28 17:51:52 2017 -0800 test: Verify stat calculations during backfill Signed-off-by: David Zafman (cherry picked from commit 88ce0c1a91178c25c75e9b627c2a2b64b0969648) commit 98cf928941303ed96b9b8e51e16c3e99b12812e3 Author: David Zafman Date: Fri Nov 17 17:01:56 2017 -0800 test: Verify stat calculations during recovery Signed-off-by: David Zafman (cherry picked from commit f5af1af6d3727f915dfcf7ea8a243b4a52b43859) commit f7bc7475b072724104003f12f439fa7502754598 Author: David Zafman Date: Thu Jan 11 14:22:54 2018 -0800 ceph-helpers.sh: Add flush_pg_stats() to wait_for_clean() to make it reliable osd-scrub-repair.sh: Fixes for omap keys landing on different OSDs due to flush Signed-off-by: David Zafman (cherry picked from commit aeba36a660cd4ab59669ee15bf92cb9640df0371) commit 82b1a41e10641dc9e8ccc90f7dd2f00e366eec92 Author: David Zafman Date: Mon Dec 4 13:02:04 2017 -0800 qa: Ignore degraded PGs when injecting random eio errors Signed-off-by: David Zafman (cherry picked from commit c77941f593755f79e4a800545d7ff437bbe0562d) commit 7065a781a42c0ab94155f92b0719e429105a8072 Author: David Zafman Date: Sat Nov 18 10:16:53 2017 -0800 osd: Improve the way insufficient targets is handled to be compatible with EC Signed-off-by: David Zafman (cherry picked from commit 91d1487ecfb497905a20556baf699ce8dde1cc30) Conflicts: src/osd/PG.cc (trivial) commit 6a652944e376e9f319996a87d0c7a253eed019c1 Author: David Zafman Date: Tue Oct 31 18:15:53 2017 -0700 osd: Improve pg degraded state setting based on _update_calc_stats() degraded count Signed-off-by: David Zafman (cherry picked from commit 74f9e70903723bdad190bc3f71a2ca2109bfe4f7) Conflicts: src/osd/PG.cc (trivial, ignore change to code not present) src/osd/PG.h (trivial, white space diff) commit 1afeb8bc5e3291d517beed781b8698bba3280b2c Author: David Zafman Date: Thu Oct 26 15:36:54 2017 -0700 osd: Handling when recovery sources have missing Signed-off-by: David Zafman (cherry picked from commit b769b98686e98b16a74318346ddf0cee88ac7476) commit b683ba9fe9d01b1010c2fbbb48a4d654a5831f5b Author: David Zafman Date: Wed Oct 25 21:44:14 2017 -0700 osd: Base pg degraded state on num_degraded_objects Signed-off-by: David Zafman (cherry picked from commit 5ffde0002f605401fb99535ff86afa0eb0404c67) commit 15bd330eac78e00d20b0dab6a5ac3dbc1fc9897d Author: David Zafman Date: Wed Oct 25 09:37:00 2017 -0700 osd: Rewrite _update_calc_stats() to make it cleaner and more accurate Signed-off-by: David Zafman (cherry picked from commit e1075f107372c97580b984c74666027fc2e71daf) Conflicts: src/osd/PG.cc (trivial) commit 9142833bec8adf3d4e0355c796d5c5f5c5fa115c Author: David Zafman Date: Tue Nov 14 13:00:06 2017 -0800 osd: cleanup: Remove unused const vars Signed-off-by: David Zafman (cherry picked from commit 9a66a6f517ea0c9c7963b2f3127f5efb78aea346) commit 50cdacb9b9bfc658b8825b269cafdb48210b5003 Author: David Zafman Date: Thu Oct 26 10:36:32 2017 -0700 osd: cleanup: Fix log message Signed-off-by: David Zafman (cherry picked from commit 01ac714aa3522dde130b48d5901e72533aa423b1) commit ad11a8e9b2d4e53389fe91748e75baacd47dbbf5 Merge: abb91dc4f1 ad9db7eea8 Author: John Spray Date: Thu Jan 18 10:02:00 2018 +0000 Merge pull request #19929 from zmc/wip-luminous-prom-pg_deep luminous: mgr/prometheus: add missing 'deep' state to PG_STATES in ceph-mgr pro… Reviewed-by: John Spray commit abb91dc4f194696ea2b91a37d159ece5ccbbc9ce Merge: 2b7410003e 124b4e3465 Author: Jason Dillaman Date: Wed Jan 17 20:13:36 2018 -0500 Merge pull request #19943 from dillaman/wip-22676-luminous luminous: cls/rbd: remove incompatible group features from partial implementation Reviewed-by: Mykola Golub commit 124b4e3465ddf8751011dbd920851e251e1d2c57 Author: Jason Dillaman Date: Wed Jan 17 17:21:47 2018 -0500 qa/suites: filter out cls_rbd tests against unsupported methods Signed-off-by: Jason Dillaman commit 2b7410003e332e8dd87ee5a367e712439d0744be Merge: cc6a3561cc 9f9db135a0 Author: Abhishek L Date: Wed Jan 17 23:12:37 2018 +0100 Merge pull request #18865 from shinobu-x/wip-21631-luminous luminous: doc: remove region from "INSTALL CEPH OBJECT GATEWAY" Reviewed-By: Abhishek Lekshmanan commit cadd8426b358647505e950017d9a146e6f7f6696 Author: Jason Dillaman Date: Sat Jan 13 16:15:48 2018 -0500 cls/rbd: remove incompatible group features from partial implementation Fixes: https://tracker.ceph.com/issues/22676 Signed-off-by: Jason Dillaman commit 5353569eea13e1ef6ddd752c8ebfdc367c6e7245 Author: Yan, Zheng Date: Wed Jan 10 13:04:55 2018 +0800 mds: handle 'inode gets queued for recovery multiple times' Fixes: http://tracker.ceph.com/issues/22647 Signed-off-by: "Yan, Zheng" (cherry picked from commit 06bbf939ac7af9969765d50fad275c24b204a188) commit 4063b807daa9e02be7fc9829f2338d13364baeee Author: Yan, Zheng Date: Wed Jan 10 11:52:35 2018 +0800 mds: use elist to track recover queue items Signed-off-by: "Yan, Zheng" (cherry picked from commit 74f2a87887e81dc1d5895976982efb51b0a5e97e) Conflicts: src/mds/CInode.cc commit 91be5317da05dea8f8cf9695b2059a72c4c151c4 Author: Xuehan Xu Date: Sat Jan 6 10:40:33 2018 +0800 common: compute SimpleLRU's size with contents.size() instead of lru.size() As libstdc++ earlier than version 5 implement the list::size() as a O(n) operation, this should be needed to avoid regression of various ceph component's performance. Signed-off-by: Xuehan Xu (cherry picked from commit 7e0a27a5c8b7d12d378de4d700ed7a95af7860c3) commit dc96e3c1f876ecb7fccb3238f8641d79b0dcc51d Author: Patrick Donnelly Date: Tue Jan 9 16:48:18 2018 -0800 qa: increase osd count for ec testing Missing in d0732fc96fbc6849dd51b391d85f765c74cfb593. Fixes: http://tracker.ceph.com/issues/22646 Signed-off-by: Patrick Donnelly (cherry picked from commit f8c275b8b6c59c2f269990ec81e503a54c0dc7c1) commit 1142dacc6467af2643c317fa74fdeed5064c0565 Author: Patrick Donnelly Date: Tue Jan 9 16:47:25 2018 -0800 qa: add missing openstack configs First introduced in: ec6fb28eaf8e2db327e4afc115879a40c7664e07 Signed-off-by: Patrick Donnelly (cherry picked from commit e8a0f1c554cad7529e49dcacfd8506c7e33f949a) commit 8e3edae0c8b472a5fc9db119173de21a3374dffd Author: Sage Weil Date: Tue Jan 9 09:40:06 2018 -0600 debian: add -dbg packages for python-{rados,rgw,rbd,cephfs} Signed-off-by: Sage Weil (cherry picked from commit b5bd43323fd96aa975e0500261b65a46067b68fa) commit 98eaf4435c92ae158a45df2dfef5c70506548fed Author: Sage Weil Date: Tue Jan 9 08:37:15 2018 -0600 debian/rules: strip ceph-base libraries This includes rados classes and ec plugins. Fixes: http://tracker.ceph.com/issues/22640 Signed-off-by: Sage Weil (cherry picked from commit c3a7895178751051f8a0f2d0afae577cfe7badbd) commit cc76ab1eefab661b794d2852836441fe6e7a761b Author: YunfeiGuan Date: Wed Jan 3 11:43:07 2018 +0800 ceph-fuse: ::rmdir() uses a deleted memory structure of dentry leads a core we should add the dentry ref immediately after "get_or_create" in case of the ref be put to zero. Fixes: http://tracker.ceph.com/issues/22536 Signed-off-by: YunfeiGuan (cherry picked from commit 65fcccc04a6239a9c7cbf0192de14f963ebf1079) commit 44643cae8da8068c3b9ab086b9dfc367f653e043 Author: Igor Fedotov Date: Wed Jan 3 16:48:51 2018 +0300 objectstore/store_test: fix lack of flush prior to collection_empty() call Fixes: http://tracker.ceph.com/issues/22409 Signed-off-by: Igor Fedotov (cherry picked from commit f669fcdfd5e5facd1b702ed113cac6f1d56bef5b) commit d2a68571a834e0c8ee8495fd82cde5d85b872098 Author: Sage Weil Date: Wed Jan 3 08:37:12 2018 -0600 mgr/DaemonServer: fix error string Signed-off-by: Sage Weil (cherry picked from commit 78b7f10d298070515418f5a71931d6235498c87f) commit adc47b95958b96322a649e6e2f5d6d4e2d58701e Author: Sage Weil Date: Wed Jan 3 08:37:02 2018 -0600 qa/tasks/ceph_manager: tolerate failure to force backfill/recoery The pool may have been deleted out from underneath us. Signed-off-by: Sage Weil (cherry picked from commit 8651e15c93ce419529b82c4c19feef1ab2b647c8) commit 80ea9ed3b30bcea3e1b561a160525280238af4bf Author: Patrick Donnelly Date: Mon Dec 18 21:29:11 2017 -0800 qa: check pool full flags Cluster-wide flag removed in b4ca5ae462c6f12ca48b787529938862646282cd. Fixes: http://tracker.ceph.com/issues/22475 Signed-off-by: Patrick Donnelly (cherry picked from commit 6e046dfc90e0a119ceb13935dc6d15efb2845184) commit 5fd01ca888b9f8b0077d07f18e62e8a4d108a38e Author: dongdong tao Date: Wed Jan 3 21:35:16 2018 +0800 mds: fix dump last_sent issue: http://tracker.ceph.com/issues/22562 Signed-off-by: dongdong tao (cherry picked from commit 3d3df18bdb21aff25d1f1111718eccbb6640b5a6) commit e56106cfc1056e2479919ab8cc721cbf75791ca5 Author: Vasu Kulkarni Date: Mon Jan 15 13:57:51 2018 -0800 create 4 lv's by default for ceph-volume tests Signed-off-by: Vasu Kulkarni commit ad9db7eea86f3518faa54820ac62c6e34dc7cc7c Author: Peter Woodman Date: Sat Nov 11 16:32:37 2017 -0800 mgr/prometheus: add missing 'deep' state to PG_STATES in ceph-mgr prometheus plugin without this the endpoint throws 500s when any PG is doing a deep scrub. Signed-off-by: Peter Woodman Fixes: http://tracker.ceph.com/issues/22116 (cherry picked from commit 8c93c0eeaff406af0878916ef5b9ce958896d586) commit a1ca7f7d93acd5f7fa58b25371b9bcfb7855ef25 Author: Jan Fajerski Date: Mon Nov 13 14:42:56 2017 +0100 pybind/mgr/prometheus: don't crash when encountering an unknown PG state Signed-off-by: Jan Fajerski (cherry picked from commit 67a1b66b06bf74274c2cf1d4b5c900f2c56a990d) commit cc6a3561cc8ec65e05451073d0dfdd6efb4b9700 Merge: 905b73442b 1961e32ce9 Author: John Spray Date: Mon Jan 15 14:42:55 2018 +0000 Merge pull request #19925 from jcsp/wip-dashboard-trim-luminous2 luminous: mgr: remove unused static files from dashboard module Reviewed-by: Sebastien Han commit e9f5612bd789ca17a01c6e51794e2f65e40ecaed Author: huangjun Date: Mon Oct 9 22:05:21 2017 +0800 qa/standalone/osd/osd-mark-down: create pool to get updated osdmap faster Mon send osdmap to random osds after we mark osd down, the down osd may use more than $sleep time to get updated osdmap if there is no osd ping between osds. So create pool after setup cluster. Signed-off-by: huangjun (cherry picked from commit ee618a38a9ed06b3ea4d02e46cdeae6afb376b82) commit d5e2e43de82cb1bf02b32bfea5a8e4c9bb495494 Author: Xuehan Xu Date: Sat Jan 6 10:40:33 2018 +0800 common: compute SimpleLRU's size with contents.size() instead of lru.size() As libstdc++ earlier than version 5 implement the list::size() as a O(n) operation, this should be needed to avoid regression of various ceph component's performance. Signed-off-by: Xuehan Xu (cherry picked from commit 7e0a27a5c8b7d12d378de4d700ed7a95af7860c3) commit 8b13643b324fe1be39cb45777c11652da2a80c61 Author: Josh Durgin Date: Wed Jan 10 21:39:28 2018 -0500 config: lower default omap entries recovered at once For large omap DBs, reading 64k leads to heartbeat timeouts. There are numerous callchains leading to this recovery step, many of which do not have heartbeat handles, so for an easily backported version just change the default number of entries read. DBs approaching 100GB may require an even lower setting, but this should be good enough for most clusters, without sacrificing recovery speed. Fixes: http://tracker.ceph.com/issues/21897 Signed-off-by: Josh Durgin (cherry picked from commit 72c2076f2c14778982fb944ffade3f071a727d1a) commit 1961e32ce997917e819b40b7f8ec21fd14d7d174 Author: John Spray Date: Wed Aug 2 15:11:07 2017 +0100 mgr/dashboard: remove unneeded bits of AdminLTE AdminLTE is delivered in a handy all-in format, but we don't need all this stuff at runtime. Signed-off-by: John Spray (cherry picked from commit 38df5150ee026c6dc9ec763f757713c7e4d99cdb) commit 492c170a6874f0fe12bef21ae702f5f6314b2107 Author: John Spray Date: Wed Aug 2 15:15:04 2017 +0100 mgr/dashboard: remove non-minified bootstrap Signed-off-by: John Spray (cherry picked from commit 5ce51ebbf047e373eac624e49e2d2bcfb46f09ff) commit f081fde52e523f10f2e57133c3a86e6fa9596714 Author: John Spray Date: Wed Aug 2 15:54:17 2017 +0100 mgr/dashboard: remove un-minified datatables source Signed-off-by: John Spray (cherry picked from commit 888f0f934e59fe28fdd97bca044702ce445bfe2e) commit 47194cdcca4e065616f74415de62e7f562eb7eb0 Author: John Spray Date: Mon Sep 18 06:10:31 2017 -0400 mgr/dashboard: don't include remote js/css This will help anyone running away from the internet. Signed-off-by: John Spray (cherry picked from commit 199d7ff26959961f200b86062f6290c9c17cba9a) commit 8e4cc4f9991bc2ee5fcf01d0b1f6ff9334c6c853 Author: John Spray Date: Mon Sep 18 06:13:45 2017 -0400 mgr/dashboard: re-arrange static files Signed-off-by: John Spray (cherry picked from commit 85a87da7aa46266abd8d4c42218bcbccce386316) commit 905b73442b8b16a898fcc8168a8ef4da792424bc Merge: 0e57572c31 723b0f2050 Author: Kefu Chai Date: Thu Jan 11 14:31:26 2018 +0800 Merge pull request #19880 from jcsp/wip-doc-dashboard-standby-luminous doc: update mgr/dashboard doc about standbys Reviewed-by: Kefu Chai commit f59380e1598981468cdafe9f575bace41c9a61b3 Author: Igor Fedotov Date: Sat Dec 30 02:05:13 2017 +0300 os/bluestore: add asserts for fsck's used_blocks bitmap access as boost doesn't always handle out-of-range access properly. Signed-off-by: Igor Fedotov (cherry picked from commit 121c40286a60a63cda32c7111aaade2043ff18df) commit f919061a53ef8a7e4e01e841428f4bbb02e640d7 Author: Igor Fedotov Date: Fri Dec 29 20:59:16 2017 +0300 os/bluestore: refactor FreeListManager to get clearer view on the number of alloc units it tracks. This also fixes out-of-range access for fsck's used_blocks bitmap that might happen when checking stores created prior to v12.2.2 Fixes http://tracker.ceph.com/issues/22535 Signed-off-by: Igor Fedotov (cherry picked from commit f64c236f278732b0fa211cd9e93c4f9d5a77a356) Conflicts: src/os/bluestore/BlueStore.cc: Removed argument 'what' from apply. commit 723b0f2050dcb06f2772c7b1f3e5f94e68a4b2e0 Author: John Spray Date: Tue Jan 9 22:23:33 2018 +0000 doc: update mgr/dashboard doc about standbys Signed-off-by: John Spray (cherry picked from commit fa14812cbb584b61cb2ce687ae4c14225eecd9fe) commit 2adc133b86a50a6c79d59f81b245b6bfd5cf1e14 Author: Matt Benjamin Date: Tue Oct 31 18:31:13 2017 -0400 RGW: S3 POST policy should not require Content-Type In the current implementation, putting a key in the acting RGWPolicyEnv makes it required in RGWPolicyEnv::match_policy_vars. I'm not sure this is the intent, but in any case, add it to the env only if sent. Fixes: http://tracker.ceph.com/issues/20201 Signed-off-by: Matt Benjamin (cherry picked from commit af29276c32f7009a78bd8b90d8f7d19026859c74) commit 6cc4a33933109bd9cd333034a538b2cc3621f56a Author: John Spray Date: Tue Dec 19 11:41:08 2017 +0000 packaging: explicit jinja2 dependency for dashboard We were getting this via python-flask, when mgr was installed on nodes with ceph-mon, but for correctness in general we should depend on it from ceph-mgr (it is imported by the dashboard module). Fixes: http://tracker.ceph.com/issues/22457 Signed-off-by: John Spray (cherry picked from commit 87399bea8321a7a9ab554085be3a3d5e0563f27f) commit 0e57572c311df7597c0e7c235dfe195ae04b7cfd Merge: 0706e9b842 2042881c67 Author: John Spray Date: Tue Jan 9 11:13:04 2018 +0000 Merge pull request #19844 from jcsp/wip-22615 luminous: mgr/dashboard: Fix PG status coloring Reviewed-by: Wido den Hollander Reviewed-by: John Spray commit 0706e9b842cb6027f70adea8d8cc996763fe642b Merge: 3de49aa945 0f0df7db74 Author: Jos Collin Date: Tue Jan 9 09:16:58 2018 +0000 Merge pull request #19858 from joscollin/wip-luminous-doc-misc-fixes luminous: doc: misc fixes for CephFS best practices Reviewed-by: Abhishek Lekshmanan commit 0f0df7db74069fef8b526c849002f82f9d8087dd Author: Jos Collin Date: Fri Jan 5 11:51:00 2018 +0530 doc: misc fixes misc fixes for best-practices. Signed-off-by: Jos Collin (cherry picked from commit ecacd1078db5020e6d65436f9d56cc9ed0016666) commit 2e7c40232ff12393a1afe10b8ff1669f13047b14 Author: Jos Collin Date: Fri Jan 5 11:24:23 2018 +0530 doc: fix heading Fixed 'Which kernel version?' heading not having brown color background. Signed-off-by: Jos Collin (cherry picked from commit a068fd842f431a54f9db2d437a71cdc11907a8e1) commit 17aa16dc6b993373539cecc2a4e110c0455dcbb6 Author: Jason Dillaman Date: Wed Dec 20 16:55:06 2017 -0500 librbd: filter out potential race with image rename Fixes: http://tracker.ceph.com/issues/18435 Signed-off-by: Jason Dillaman (cherry picked from commit 001d2fc35c170f17e706aeae541fb86dac960319) commit 9d617b4ecceda094040f4fe4db490588e6684a18 Author: Sage Weil Date: Thu Dec 7 16:40:04 2017 -0600 mon/LogMonitor: add mon_cluster_log_to_stderr Optionally send cluster log messages to stderr (prefixed by the channel). Signed-off-by: Sage Weil (cherry picked from commit 49b327e8657695a359ea63e427b5de2e4525f480) commit acc76d713f11f52c39f6ba5868ca1e502c1985b2 Author: Sage Weil Date: Thu Dec 7 16:39:30 2017 -0600 log: add log_stderr_prefix option Allows you to set a prefix for debug log messages send to stderr (e.g., "debug "). Signed-off-by: Sage Weil (cherry picked from commit c142ae3f1530067f35b8a1e1bac9ddeea4d81c09) commit 473f12d4fd4c0ee747f7a1822f513ad1f87cdfb0 Author: Casey Bodley Date: Sun Nov 26 15:46:26 2017 -0500 cmake: add WITH_BOOST_CONTEXT option adds a more specific option for this boost::context dependency, which was previously only used by the radosgw beast frontend. see http://tracker.ceph.com/issues/20048 for more background Signed-off-by: Casey Bodley (cherry picked from commit b589b73c70bc4c644b7a040d9aee5083881254b9) Conflicts: src/include/config-h.in.cmake (missing HAVE_GETENTROPY from https://github.com/ceph/ceph/pull/17972) commit b1ad4b1e9e27d2b6e0cfe2b05e7693aa4c298eb5 Author: Casey Bodley Date: Mon Dec 18 22:23:53 2017 -0500 cmake: remove Beast submodule and include path the beast library is included in boost 1.66 Signed-off-by: Casey Bodley (cherry picked from commit d85f42dd697365ad6c6be80af2e31a6415f55d5e) Conflicts: src/rgw/CMakeLists.txt (missing SYSTEM flag from https://github.com/ceph/ceph/pull/18711) commit eb6f089631225ab4779184a3d20c7a1cbde44853 Author: Casey Bodley Date: Tue Dec 5 14:43:06 2017 -0500 rgw: update beast frontend for boost 1.66 Fixes: http://tracker.ceph.com/issues/22600 Signed-off-by: Casey Bodley (cherry picked from commit a66a6449296d39e57724fa733ac3069fecfdcdaa) commit b1bfa824b096189c8e5ee8e42f9bf400c44059d7 Author: Casey Bodley Date: Mon Dec 18 13:03:35 2017 -0500 cmake: update minimum boost version to 1.66 Signed-off-by: Casey Bodley (cherry picked from commit 0e47a339dface3807ab0061f9ce62fe1d2654d29) commit 1d2e15ac08f43ad2125ba0892f3beb1c632eda20 Author: Casey Bodley Date: Mon Nov 27 14:16:33 2017 -0500 submodule: update Beast to ceph/ceph-master branch pulls beast submodule up from v116 tag to v124, with an additional bug fix for async_read_some() Signed-off-by: Casey Bodley (cherry picked from commit efe90765957371a78b36305727dc3eb34252ad91) commit acb49cfc8f2cc79b0bf38ab0a95c07398a9c5061 Author: Casey Bodley Date: Thu Nov 9 23:20:40 2017 -0500 rgw: RGWEnv::set() takes std::string the beast frontend will either pass a string_ref or a string_view, depending on the boost version. we can't overload RGWEnv::set() for both, because a call to env.set("literal") would be ambiguous both string_ref and string_view have a to_string() member function, so RGWEnv::set() now takes a std::string by value and moves it into the map. this involes a single string copy, whether we pass a temporary std::string (in beast) or a const char* (in civetweb) Fixes: http://tracker.ceph.com/issues/22101 Signed-off-by: Casey Bodley (cherry picked from commit 6fbd3f358a17debea8c04f976946d5f245576f31) commit 0f08d3d9a53f1cb1f1ea13181a3c96a14f7b65dd Author: Casey Bodley Date: Wed Oct 18 20:22:11 2017 -0400 rgw: fix for pause in beast frontend pause_for_new_config() was only stopping the listener on pause, but existing keepalive connections would keep trying to read. this prevented the frontend thread calls to io_service::run() from returning to complete the pause Fixes: http://tracker.ceph.com/issues/21831 Signed-off-by: Casey Bodley (cherry picked from commit c64ac823adfb6d717570d3d3e82b89ea73dc3a77) commit 3d79dadf4aba3f15352fbf9750ff69afe0adaf9e Author: Casey Bodley Date: Tue Sep 26 14:35:56 2017 -0400 qa/rgw: add beast frontend to some rgw suites added a qa/rgw_frontend directory for civetweb.yaml and the new beast.yaml. the rgw suites for multifs and singleton now symlink rgw_frontend/civetweb.yaml. the multisite, tempest and verify suites symlink rgw_frontend to test both. this doubles the number of jobs in those suites Signed-off-by: Casey Bodley (cherry picked from commit 607b72f553b3d4c1ad6feba9ee7199a4d91272f1) commit 53ff61f95db75a892da0c085e9eaf9e3425e94d2 Author: Casey Bodley Date: Wed Oct 11 09:01:35 2017 -0400 rgw: ask beast parser about keepalive/close Signed-off-by: Casey Bodley (cherry picked from commit 5d7f154a77eb2cf82b6e4e49d20687c8ed839113) commit c3bb0bae2a9878ec8c3302168f82a22aed5a983d Author: Casey Bodley Date: Mon Sep 25 11:13:08 2017 -0400 rgw: remove boost::coroutine and context deps the beast frontend no longer uses stackful coroutines, so these dependencies aren't necessary Signed-off-by: Casey Bodley (cherry picked from commit b904b575042d6598a9cf74d23beecd4884e097d2) commit 30b4d2bb9385dfc7837f7941df5a7a3c507c8b72 Author: Casey Bodley Date: Mon Jul 17 10:12:07 2017 -0400 rgw: beast frontend discards unread body before next header Signed-off-by: Casey Bodley (cherry picked from commit c6bc7e1b0cbaf553f000aaa2893fa1486fc02a75) commit c897636ff13750acd446cd7a37b4b5c83e4d4dde Author: Casey Bodley Date: Wed Jul 12 11:39:26 2017 -0400 rgw: beast frontend uses callbacks instead of coroutines Signed-off-by: Casey Bodley (cherry picked from commit 7767d8d88aecac8e88ed4c87a187c7a2ed67cf48) commit 0a024c978b72402e4f8792c60c017d3ccbc60050 Author: Casey Bodley Date: Tue Jul 11 16:12:02 2017 -0400 rgw: set header/body size limits on beast parser Signed-off-by: Casey Bodley (cherry picked from commit 1f72684eb4c6fb9fd021240ae55812894afc19e9) commit 1bc2728da84992c2a737202f04ad358470140a34 Author: Casey Bodley Date: Thu Jul 6 16:31:23 2017 -0400 rgw: update beast frontend/submodule to v116 Signed-off-by: Casey Bodley (cherry picked from commit 746c218c620d7681f6c9d769631ee1ac0d2b5987) commit 2dd6512cdd9ed2548ceae42d5785c140b1ac2a35 Author: Jiaying Ren Date: Thu Aug 10 15:21:32 2017 +0800 rgw: fix opslog uri as per Amazon s3 According to s3[1], current Request-URI opslog entry are missing: + request method + query string + http version number [1] http://docs.aws.amazon.com/AmazonS3/latest/dev/LogFormat.html Fixes: http://tracker.ceph.com/issues/20971 Reported-by: Zhang Shaowen Signed-off-by: Jiaying Ren (cherry picked from commit 96bb12a158ab899ec219d88e140499a4e27b4ba5) commit e7bd12a38716d17a7d6b752d6730d5913916cded Author: Jiaying Ren Date: Mon Aug 7 17:30:27 2017 +0800 rgw: fix opslog can't record referrer when using curl as client Fixes: http://tracker.ceph.com/issues/20935 Reported-by: Zhang Shaowen Signed-off-by: Jiaying Ren (cherry picked from commit 23323b7f725dc1e99e4a81512b57d342dab9a3fe) commit 9db61c8b14a915c8f99dfe896610a2431f277f68 Author: Bingyin Zhang Date: Tue Dec 26 17:06:44 2017 +0800 rgw: put bucket policy panics RGW process Fixes: http://tracker.ceph.com/issues/22541 Signed-off-by: Bingyin Zhang (cherry picked from commit f05a044cad18c16ebc2c9a177f38b4bdc76cfd66) commit 3de49aa94560625909794153b4ab965aeeba4b37 Merge: 907a72465e ca980a63c3 Author: Jos Collin Date: Mon Jan 8 17:22:30 2018 +0000 Merge pull request #19505 from shinobu-x/wip-22392-luminous luminous: mds: tell session ls returns vanila EINVAL when MDS is not active Reviewed-by: Jos Collin commit 907a72465e0f3546724d9686b3cafe5d7208cf9f Merge: 70755a02e6 567bd5acad Author: John Spray Date: Mon Jan 8 16:18:12 2018 +0000 Merge pull request #19831 from pdvian/wip-22576-luminous luminous: qa: configure zabbix properly before selftest Reviewed-by: John Spray Reviewed-by: Wido den Hollander commit 2042881c6702a9d660b57385a908e9da7df4cd0f Author: Wido den Hollander Date: Mon Dec 11 11:33:16 2017 +0100 mgr/dashboard: Fix PG status coloring This was broken in the 12.2.2 release and PGs would not get a color anymore. This commit also makes sure pools/PGs are marked as red when one or more PGs are inactive, inconsistent, incomplete, down, etc. Signed-off-by: Wido den Hollander (cherry picked from commit 450aa2133d05916822083969260ee07622591e33) commit 0a347506af4b45decf604bea4144ac1eafc2d34e Author: Kefu Chai Date: Thu Jan 4 16:08:25 2018 +0800 rpm: adjust ceph-{osdomap,kvstore,monstore}-tool feature move this is the rpm's counterpart of debian/control changes related to the ceph-{osdomap,kvstore,monstore}-tool feature move. see #19328 and #19356. the commit introducing this move is 6dba25e. and $ git describe 6dba25e v12.2.2-8-g6dba25e39d so the first release that have this change is 12.2.2-8. in other words, ceph-{base,osd,mon} >= 12.2.2.8 cannot co-exist with ceph-test < 12.2.2-8 in the same system. so we let ceph-test Requires ceph-common with the same version. and since ceph-{osd,mon} Requires ceph-base with the same version, and ceph-base Requires ceph-common with the same version, so by tiering ceph-test with ceph-common with the same version, we enforce this restriction. Fixes: http://tracker.ceph.com/issues/22558 Signed-off-by: Kefu Chai (cherry picked from commit b07aa210aa0ede54ffc3dbe49e334bd51a8f6342) commit 70755a02e66eb82fefc7d3a0114fe5a98a7f1dc2 Merge: c859483fee 4a79dd9065 Author: Yehuda Sadeh Date: Mon Jan 8 12:15:39 2018 +0200 Merge pull request #19787 from linuxbox2/luminous-21984 rgw: fix rewrite a versioning object create a new object bug commit b5892a13b9e10f1d5cc5e3c4e869bac06c3926ca Author: Ilya Margolin Date: Thu Dec 14 17:07:01 2017 +0100 pybind/mgr/prometheus: fix metric type undef -> untyped Fixes "Prometheus exporter can't get metrics after update to 12.2.2" Fixes: http://tracker.ceph.com/issues/22313 Signed-off-by: Ilya Margolin (cherry picked from commit 58669bedf530e1ccaf4d6c4bf018eeec43b78647) commit 567bd5acad2d1d5598a63835b9ce874e697cf2a8 Author: John Spray Date: Thu Dec 21 08:27:45 2017 -0500 qa: configure zabbix properly before selftest Even though the selftest routine doesn't care about the settings, we should set them to avoid emitting nasty log/health messages when enabling the module. Fixes: http://tracker.ceph.com/issues/22514 Signed-off-by: John Spray (cherry picked from commit c64c9ff00d2df2177135bcd2735bd7aeac625643) commit 98e3e2a7c840f985baa4ae82a9ce943e85b99657 Author: Patrick Donnelly Date: Tue Dec 19 11:37:14 2017 -0800 mds: check metadata pool not cluster is full CEPH_OSDMAP_FULL flag was obsoleted by b4ca5ae462c6f12ca48b787529938862646282cd. So, check if the metadata pool is full instead which is a decent proxy (as metadata operations can still proceed). However, the data pool may still be full which would result in some operations still not completing (like inode backtrace updates). Fixes: http://tracker.ceph.com/issues/22483 Signed-off-by: Patrick Donnelly (cherry picked from commit d678415bb03afd1a67edaa0eac031a6a9cf3fbf9) commit dd6b6626ccc97666a9de24fcc770f9b7ee9b3a31 Author: Patrick Donnelly Date: Wed Dec 13 18:07:00 2017 -0800 vstart_runner: ignore failed dentry invalidation Signed-off-by: Patrick Donnelly (cherry picked from commit cd1e5f53f0a732e4398efadbbf5b6ce914ddd647) commit c5c0435efcdce7e2e87cf95a0e13642b40b9a42f Author: Patrick Donnelly Date: Wed Dec 13 17:50:32 2017 -0800 vstart_runner: set ec_profile attribute Signed-off-by: Patrick Donnelly (cherry picked from commit 0f507d88559e7060e86a4a3ea9fcffb02c013bec) commit e97399d2eefb13ff8083ce2d9c9686e69c30e488 Author: Patrick Donnelly Date: Fri Dec 15 14:23:01 2017 -0800 mds: reduce debugging level for balancer messages Fixes: http://tracker.ceph.com/issues/21853 Signed-off-by: Patrick Donnelly (cherry picked from commit 435babe4a44c73d094eeddb5e621ebd29a62aff8) commit 5bf8d71001bf929813fd5e3bb101c60630ab6c5d Author: John Gibson Date: Tue Sep 19 09:17:22 2017 -0400 rgw: Policies now properly evaluate the X-Forwarded-For header. Signed-off-by: John Gibson (cherry picked from commit 5f7d9c4ff6c78f65d074dbdf8a181cb9ae09851e) commit ac3e81b8ad887cc84e553d7dd24aa002ea536269 Author: John Gibson Date: Tue Sep 19 10:55:12 2017 -0400 rgw: Test of proper parsing of the X-Forwarded-For header for policies. Signed-off-by: John Gibson (cherry picked from commit e02facbf5c7b06b4d1d628ddb83ee74935493def) commit 65b515b9d0c119dde57f57d1d7c75d81e175113b Author: John Gibson Date: Sun Dec 24 15:49:50 2017 -0500 rgw: Bucket IP address policy evaluation now uses rgw_remote_addr_param. Previously bucket policy ip address restrictions were only being evaluated against the REMOTE_ADDR environment variable and ignoring the header specified by the rgw_remote_addr_param configuration option. This rendered ip-based bucket policies worthless when running behind a reverse proxy. Signed-off-by: John Gibson (cherry picked from commit c4c24ca986f17c68b75f76fc48ad489002fcf87e) commit a75908b8098ef39ed0193616e9d4fb723d73883c Author: John Gibson Date: Thu Sep 14 10:12:59 2017 -0400 rgw: rgw_iam test harness can now stringify() the fake identity. Signed-off-by: John Gibson (cherry picked from commit 557bdb617e6eb78f9f0e6d6ba35b79b318a6c275) commit 1d8f684a3cf11c5fc422462790b5a87d5ab5b587 Author: John Gibson Date: Sun Dec 24 15:48:00 2017 -0500 rgw: Fixed several bugs in policies related to IPv6 addresses. The IPv6 conversion was not properly converting the address to host byte order. The text conversion of IPv6 addresses was using raw byte values instead of the converted number. The portions of the addresses were grouped by bytes instead of 16-bit words. The prefix length was erroneously being rendered in hex. http://tracker.ceph.com/issues/20991 Signed-off-by: John Gibson (cherry picked from commit a25ca37401d1e8dc4349201b9f64aa6990bea0d5) commit 9f7aeb4bd52bbdaef4ffd0127b38c3084e95a706 Author: John Gibson Date: Sun Dec 24 15:44:54 2017 -0500 rgw: Fixed several bugs in policies related to IP Addresses. Comparisons of two individual IP addresses caused an assertion error. The text conversion of IPv4 addresses was using raw byte values instead of the converted number. NotIpAddress condition now works with multiple values. http://tracker.ceph.com/issues/20991 Signed-off-by: John Gibson (cherry picked from commit ca21596c95aa3871d14ac9112840d3b1101a8635) commit 5fbb50c1880b0dafc64892e0c23d188c2eccbec7 Author: John Gibson Date: Tue Dec 26 16:40:31 2017 -0500 rgw: Added tests for S3 Policy IP Address feature. Signed-off-by: John Gibson (cherry picked from commit 3d260f26e6826182b4de520878b548a28a96e81b) commit e4a5cf95e645d118a781b65c4ca6e5c37dd7eb6c Author: John Gibson Date: Thu Aug 10 13:39:35 2017 -0400 rgw: Added support for testing the wildcard principal in policies. Signed-off-by: John Gibson (cherry picked from commit e83b647d44153a7775647693fca1848e592ee107) commit 7cca4a6c1034ce32c41b73bace538965c5e77a8c Author: yuliyang Date: Mon Nov 27 14:32:44 2017 +0800 rgw: implement ipv4 aws:SourceIp condition for bucket policy Signed-off-by: yuliyang (cherry picked from commit 2fb445b6f7c1e997e83b1c7da2a1fecdde164d35) commit e49bf0889e6144b6683d8cb4554e38efe29b7a8a Author: Casey Bodley Date: Fri Nov 3 10:42:37 2017 -0400 rgw: simplify use of map::emplace in iam the piecewise_construct overloads of map::emplace() are only needed when there's ambiguity around which arguments go to which constructor Signed-off-by: Casey Bodley (cherry picked from commit 9787fe6b8dc545419b0f3375c06a57a414ae8aa6) commit 8eb36333bc8e709cfb9e4958821548f813e105da Author: Abhishek Lekshmanan Date: Thu Jul 27 17:36:37 2017 +0200 rgw: policy: support for s3 conditionals in ListBucket This adds support for s3:prefix,delimeter & maxkeys identifiers when specified as conditionals in policy. Signed-off-by: Abhishek Lekshmanan (cherry picked from commit a7184ca8c507b66ef01687bd85528342ea3bf934) commit 3f8ab0b8940c332b95c517d02e4aa658f394cd6d Author: Jiaying Ren Date: Mon Aug 7 15:55:19 2017 +0800 rgw: fix opslog can not record remote_addr Fixes: http://tracker.ceph.com/issues/20931 Reported-by: Zhang Shaowen Signed-off-by: Jiaying Ren (cherry picked from commit 650d30d64e084df12f4cfcb3d1edaefb5d475bb4) commit 5b9f1281a3bfe3e75bd4ecd2d3983543396d119d Author: Adam C. Emerson Date: Fri Oct 27 22:48:18 2017 -0400 rgw: Fix evaluation of bucket management permissions I spent some more time looking through the documentation of how work is evaluated, and the examples on Have convinced me that the behavior that was requested is more correct than what we were doing before. Fixes: http://tracker.ceph.com/issues/21901 Signed-off-by: Adam C. Emerson (cherry picked from commit 343a25aa2134b6fdddeca6c9dfbaefde2dc9c66a) commit febd39cf67eb026cf577d72440f5d5b61c98c3ad Author: Adam C. Emerson Date: Fri Oct 27 20:17:52 2017 -0400 rgw: Refactor checking of some ops Since some operations check the user against the bucket owner in the absence of a policy, rather than open-coding that everywhere, act like a proper computer scientist and abstract it. Fixes: http://tracker.ceph.com/issues/21896 Signed-off-by: Adam C. Emerson (cherry picked from commit 8818a0cb5e699135976e057061fb8e9d99850cd3) commit c859483fee526656b6c6cd4d9d0c975cff57b565 Merge: bccb4839ee ce1fe2480c Author: Jos Collin Date: Fri Jan 5 17:07:00 2018 +0000 Merge pull request #19449 from smithfarm/wip-22398-luminous luminous: doc: man page for mount.fuse.ceph Reviewed-by: Jos Collin commit 34864fe7ba839e0d593437b6e62f6812f1c1a375 Author: Adam C. Emerson Date: Wed Dec 20 17:06:32 2017 -0500 rgw: Plumb refresh logic into object cache Now when we force a refetch of bucket info it will actually go to the OSD rather than simply using the objects in the object cache. Fixes: http://tracker.ceph.com/issues/22517 Signed-off-by: Adam C. Emerson (cherry picked from commit d997f657750faf920170843e62deacab70008d8b) commit 5ceb7cb998b505d2fe0cd98b534a918424e4d809 Author: Adam C. Emerson Date: Tue Dec 19 16:47:09 2017 -0500 rgw: Add expiration in the object cache We had it in the chained caches, but it doesn't do much good if they just fetch objects out of the object cache. Fixes: http://tracker.ceph.com/issues/22517 Signed-off-by: Adam C. Emerson (cherry picked from commit 82a7e6ca31b416a7f0e41b5fda4c403d1d6be947) commit 767fec1e22247ec1c6010c308d51ba77dd31cb39 Author: Adam C. Emerson Date: Tue Dec 19 12:53:05 2017 -0500 rgw: retry CORS put/delete operations on ECANCELLED Fixes: http://tracker.ceph.com/issues/22517 Signed-off-by: Adam C. Emerson (cherry picked from commit bff7e61ca5a66b301ec49c1cf9054d1b74535832) commit bfd4091f7faf43ebc4eb81c8215e8b3ec2d4a8c4 Author: Adam C. Emerson Date: Fri Nov 17 17:15:26 2017 -0500 rgw: Expire entries in bucket info cache To bound the degree to which an RGW instance can go out to lunch if the watch/notify breaks down, force refresh of any cache entry over a certain age. Fifteen minutes by default, and expiration can be turned off entirely. This is separate from the LRU. The LRU removes entries based on the last time of access. This expiration patch forces refresh based on the last time they were updated. Signed-off-by: Adam C. Emerson (cherry picked from commit 4489cb58a15647a31ac0546d70400af5668404cb) Fixes: http://tracker.ceph.com/issues/22517 commit ce1fe2480c0cbef7337f870400bf441ef13e9544 Author: Jos Collin Date: Fri Jan 5 19:37:31 2018 +0530 doc: fix typo Signed-off-by: Jos Collin (cherry picked from commit a925bb4520797376b4b169bbcfe613cd1fa36429) commit fa86f78983e990b10a24d6c25526014e47370a71 Author: Jos Collin Date: Fri Jan 5 12:41:37 2018 +0530 doc: add mount.fuse.ceph to index Added mount.fuse.ceph to index.rst Fixes: http://tracker.ceph.com/issues/22595 Signed-off-by: Jos Collin (cherry picked from commit db23f4ce30cd521b5b8909edc8c26dc87aeecd98) commit e21b3af6a378cfffe58df3d56a9f9884adba8a36 Author: Jos Collin Date: Mon Nov 27 16:16:35 2017 +0530 doc: doc for mount.fuse.ceph Created doc for mount.fuse.ceph. Fixes: http://tracker.ceph.com/issues/21539 Signed-off-by: Jos Collin (cherry picked from commit 6c39818eaf39358cab8dd3dce579c932fba0d05d) commit 88c987cdb5f02ddc445389b40f9473e8fc14b8af Author: Jason Dillaman Date: Tue Oct 3 13:25:33 2017 -0400 qa/workunits/rbd: fixed variable name for resync image id Fixes: http://tracker.ceph.com/issues/21663 Signed-off-by: Jason Dillaman (cherry picked from commit 22816ed3f48b0a08a6d4e0cf0b0e14dabbb9ce69) commit a15eb7df1e3b6d273b4366e7afefd3c2ade4775e Author: Jason Dillaman Date: Wed Sep 27 09:40:08 2017 -0400 librbd: hold cache_lock while clearing cache nonexistence flags When transitioning from a snapshot that had an associated parent to a snapshot where the parent was flattened and removed, the cache was being referenced without holding the required lock. Fixes: http://tracker.ceph.com/issues/21558 Signed-off-by: Jason Dillaman (cherry picked from commit 16ef97830cde30efb96f7aee69834b3a5c2d5248) commit 84fff7aca5d7387566304206eb4cfbb934c2d536 Author: Casey Bodley Date: Mon Dec 18 11:42:21 2017 -0500 rgw: dont log EBUSY errors in 'sync error list' these temporary errors get retried automatically, so no admin intervention is required. logging them only serves to waste space in omap and obscure the more serious sync errors Fixes: http://tracker.ceph.com/issues/22473 Signed-off-by: Casey Bodley (cherry picked from commit ca4510bc76ad7beed1128539aa9e424d29dd8585) Conflicts: src/rgw/rgw_data_sync.cc ("multisite log tracing" feature - see https://github.com/ceph/ceph/pull/16492 - is not being backported to luminous) commit e51e1fa45e6a1740b94b56977bc927d302474f5c Author: Adam C. Emerson Date: Fri Nov 17 16:16:38 2017 -0500 rgw: Handle stale bucket info in RGWDeleteBucketPolicy Signed-off-by: Adam C. Emerson (cherry picked from commit e397b7e6d0c49d625fb2b2363311e6486f2045fe) Fixes: http://tracker.ceph.com/issues/22517 commit 08bf9c07e012a365d40901088feede4c4c200f84 Author: Adam C. Emerson Date: Fri Nov 17 16:15:04 2017 -0500 rgw: Handle stale bucket info in RGWPutBucketPolicy Signed-off-by: Adam C. Emerson (cherry picked from commit 1738b4f6b726b462abb436f78026c1577b55f05e) Fixes: http://tracker.ceph.com/issues/22517 commit 3f2fa364fb7f8140d20ad7de13a69cd2cc31ec77 Author: Adam C. Emerson Date: Fri Nov 17 16:05:06 2017 -0500 rgw: Handle stale bucket info in RGWDeleteBucketWebsite Signed-off-by: Adam C. Emerson (cherry picked from commit f4d274248e43cb38ff2b27782c010b2c35b12b2b) Fixes: http://tracker.ceph.com/issues/22517 commit b266755161b6769af0a08bf647c771a00a7aa50d Author: Adam C. Emerson Date: Fri Nov 17 16:03:13 2017 -0500 rgw: Handle stale bucket info in RGWSetBucketWebsite Signed-off-by: Adam C. Emerson (cherry picked from commit b2b7385f194def1025a8947bab876c9856b06400) Fixes: http://tracker.ceph.com/issues/22517 commit 672eea5d88099a590ddf847ad080214944c780cb Author: Adam C. Emerson Date: Fri Nov 17 15:59:44 2017 -0500 rgw: Handle stale bucket info in RGWSetBucketVersioning Signed-off-by: Adam C. Emerson (cherry picked from commit a0a1e7c2ef992b8758bcfb20d893730c1b202475) Fixes: http://tracker.ceph.com/issues/22517 commit fc271e4e3c44e520d8a530ebfc758a438c110737 Author: Adam C. Emerson Date: Fri Nov 17 15:53:05 2017 -0500 rgw: Handle stale bucket info in RGWPutMetadataBucket Signed-off-by: Adam C. Emerson (cherry picked from commit ebb86301b20098e15824f469001f6153b27965f5) Fixes: http://tracker.ceph.com/issues/22517 commit dea29460ed513f965de2dcac9e29180b06d56b58 Author: Adam C. Emerson Date: Fri Nov 17 15:51:42 2017 -0500 rgw: Add retry_raced_bucket_write If the OSD informs us that our bucket info is out of date when we need to write, we should have a way to update it. This template function allows us to wrap relevant sections of code so they'll be retried against new bucket info on -ECANCELED. Signed-off-by: Adam C. Emerson (cherry picked from commit 1a3fcc70c0747791aa423cd0aa7d2596eaf3d73c) Fixes: http://tracker.ceph.com/issues/22517 commit 4a79dd90657c6635f39de555fcc33daa584ecbd7 Author: Enming Zhang Date: Tue Oct 31 15:21:21 2017 +0800 rgw: fix rewrite a versioning object create a new object bug Fixes: http://tracker.ceph.com/issues/21984 Signed-off-by: Enming Zhang (cherry picked from commit 700a0292362128cb29586a64ef8215a07d96736b) Signed-off-by: Matt Benjamin commit 2e3a89f019a776ad5731e49cd16b51b3f8e2e807 Author: Adam C. Emerson Date: Thu Nov 16 14:42:58 2017 -0500 rgw: Add try_refresh_bucket_info function Sometimes operations fail with -ECANCELED. This means we got raced. If this happens we should update our bucket info from cache and try again. Some user reports suggest that our cache may be getting and staying out of sync. This is a bug and should be fixed, but it would also be nice if we were robust enough to notice the problem and refresh. So in that case, we invalidate the cache and fetch direct from the OSD, putting a warning in the log. Signed-off-by: Adam C. Emerson (cherry picked from commit 9114e5e50995f0c7d2be5c24aa4712d89cd89f48) Fixes: http://tracker.ceph.com/issues/22517 commit c49417d8b0f614a895bdc6f19b00868d694f8ef8 Author: Mark Kogan Date: Tue Dec 12 10:34:05 2017 -0500 rgw: fix chained cache invalidation to prevent cache size growth above the rgw_cache_lru_size limit Fixes: http://tracker.ceph.com/issues/22410 Signed-off-by: Mark Kogan (cherry picked from commit a6a1b664d313a54ad9d2f64b859296b1352b1ce4) commit ad0ea9fa530d641fdadcae29059660446b428588 Author: Matt Benjamin Date: Tue Oct 31 18:31:13 2017 -0400 RGW: S3 POST policy should not require Content-Type In the current implementation, putting a key in the acting RGWPolicyEnv makes it required in RGWPolicyEnv::match_policy_vars. I'm not sure this is the intent, but in any case, add it to the env only if sent. Fixes: http://tracker.ceph.com/issues/20201 Signed-off-by: Matt Benjamin (cherry picked from commit af29276c32f7009a78bd8b90d8f7d19026859c74) commit 08edb1631191ad67266a1a8a66b1d147ed0abf9f Author: Yan, Zheng Date: Wed Dec 20 09:56:50 2017 +0800 mds: respect mds_client_writeable_range_max_inc_objs config get_layout_size_increment() return layout.get_period(). A period contain layout.stripe_count objects. The config is for limiting number of objects need to probe when recovering a file. Fixes: http://tracker.ceph.com/issues/22492 Signed-off-by: "Yan, Zheng" (cherry picked from commit ee11b6cd9ce97820c4f53ed98380bbd63fab891f) Conflicts: src/mds/Locker.cc commit 8d6c2c2dea78ed485d3d63fd39085ca1e82c7d71 Author: Yan, Zheng Date: Mon Dec 18 16:48:51 2017 +0800 mds: track dirty dentries in separate list this should improve performance of large directory Fixes: http://tracker.ceph.com/issues/19578 Signed-off-by: "Yan, Zheng" (cherry picked from commit 49559663825ff742bde386355be864e03e646ffa) commit bccb4839ee8fde4636001976df1890b7a0f1f0a0 Merge: 840cc7b174 c1c539f1ef Author: Sage Weil Date: Wed Jan 3 11:07:39 2018 -0600 Merge pull request #19243 from liewegas/wip-fix-statfs-luminous mon/Monitor: fix statfs handling before luminous switchover happens Reviewed-by: Greg Farnum Reviewed-by: Patrick Donnelly commit 840cc7b1747fe12e42e7e182afd8298d1f8684b0 Merge: d94942cdfe 2928e2cf3a Author: vasukulkarni Date: Tue Jan 2 14:23:35 2018 -0800 Merge pull request #19746 from ceph/wip-use-installer0-luminous2 qa/tests - Added options to use both cases: mon.a and installer.0 commit 2928e2cf3ab02357b0eea1ae9afe332bc240af06 Author: Yuri Weinstein Date: Tue Jan 2 10:12:54 2018 -0800 qa/tests - Added options to use both cases: mon.a and installer.0 Signed-off-by: Yuri Weinstein (cherry picked from commit 10fc85089c3bb64ced8c3a0ea17987e9ec5f46a2) Signed-off-by: Yuri Weinstein commit 322966a68940637a8575f9e06f07b0475ed271ca Author: Yuri Weinstein Date: Wed Dec 20 13:05:22 2017 -0800 qa/tests: run ceph-ansible task on installer.0 role/node Signed-off-by: Yuri Weinstein (cherry picked from commit 3b2a26d919c173c887fd193f186ea56c33fcd9ae) commit 6791e67faf352b31c9e6d4e191b82ba8fac7adda Author: Yao Zongyou Date: Fri Dec 22 16:44:31 2017 +0800 erasure-code: use jerasure_free_schedule to properly free a schedule Signed-off-by: Yao Zongyou (cherry picked from commit 8113fa522d223a2e6708c345a5f216cbe4a5e758) commit e950f5dbf07ed8d7fb37047be797f3cfe60a1fe1 Author: Yao Zongyou Date: Sat Dec 23 12:48:47 2017 +0800 common/dns_resolve: fix memory leak Signed-off-by: Yao Zongyou (cherry picked from commit d7c3b5940fae5a474a27b7eee72582223e1b0504) commit d94942cdfe8fc9fccf9fd79ece8bc828d608e713 Merge: c4905daa94 eb9b1c0413 Author: Sage Weil Date: Tue Jan 2 09:17:16 2018 -0600 Merge pull request #19741 from joscollin/luminous luminous: doc: update Blacklisting and OSD epoch barrier commit eb9b1c0413aadd7f81d55a0917e840de62af67ff Author: Jos Collin Date: Tue Jan 2 16:17:39 2018 +0530 doc: fix grammar mistake Fixed grammar mistake in a sentence. Signed-off-by: Jos Collin commit 0e43f0f01cd800fee4cd800f1545405b449fa55b Author: Jos Collin Date: Tue Jan 2 16:11:30 2018 +0530 doc: update Blacklisting and OSD epoch barrier Updated missing OSD epoch barrier section, which is referenced by http://docs.ceph.com/docs/luminous/cephfs/full/#hammer-and-later. Fixes: http://tracker.ceph.com/issues/22552 Signed-off-by: Jos Collin commit c4905daa94d8b2fbad32a4e39c3e3076380363c5 Merge: 7aa78f01c9 5ec37f1f49 Author: Sage Weil Date: Sat Dec 30 14:12:19 2017 -0600 Merge pull request #19721 from tchaikov/wip-luminous-boost-un-submodule luminous: boost un-submodule Reviewed-by: Sage Weil commit 5ec37f1f49acf8fbfcb7667be43b139a68d3aa96 Author: Kefu Chai Date: Fri Nov 24 13:56:02 2017 +0800 make-dist: exclude unused bits in boost the docs, examples and tests are not used. so drop them. we could go further by removing unused components in boost. but that'd be an issue if somebody added a component in CMakeLists but forgets to update this script. also, we need to remove boost/$component and lib/$component to achieve this goal. this also introduces extra complicity. so leave it for another change. Signed-off-by: Kefu Chai (cherry picked from commit 65f91227a6b052ad60b01276d7d72dea07cbb1a4) commit 5eb42af8ce5b96b41b7d8c80cd2cc95b7eff541d Author: Kefu Chai Date: Fri Nov 24 10:27:19 2017 +0800 make-dist: repackage boost in the correct path before this change, boost is put into ./src. after this change, it is put into $outfile/src. i.e. something like ceph-12.1.2-4592-gf5f2ced624/src . Signed-off-by: Kefu Chai (cherry picked from commit 2218efa005ab871f26402b5dd33b2e599897c11f) commit 9e49711c2293761bd8b501ef1bc948bbf1e08066 Author: Brad Hubbard Date: Thu Sep 28 15:28:53 2017 +1000 make-dist,cmake: Try multiple URLs to download boost before failing Remove SPOF during boost download for make-dist and later cmake > 3.7 Signed-off-by: Brad Hubbard (cherry picked from commit 598556f98b65a03ad40ad076623b9cc8b507810e) commit 7929ff7f7edeff6df27adfdf6da5b4a89e36c4e5 Author: Sage Weil Date: Tue Sep 26 17:15:17 2017 -0400 make-dist,cmake: move boost tarball location to download.ceph.com Sourceforge is down. Also, we can hammer our servers instead of theirs. Signed-off-by: Sage Weil (cherry picked from commit 19987549e0ce526e353e24adddc7df8b1f165aab) commit b798225cf69da9d156e46ebe4fec463f79430d70 Author: Kefu Chai Date: Fri Sep 1 01:00:04 2017 +0800 make-dist: download and repackage boost Signed-off-by: Kefu Chai (cherry picked from commit f04436b189376cf7b53d710ab2f2bf01d47482ad) commit 3c6f83bb7c4b544118626fa92f7e9e5a407a1973 Author: Kefu Chai Date: Tue May 30 23:58:44 2017 +0800 boost: remove boost submodule Signed-off-by: Kefu Chai (cherry picked from commit 6b23056cf8d0d1bbd269bc96eef33dc27c25fd9e) commit 7aa78f01c95a3eac32e27c2c90ffe6d867cb89dc Merge: f519145585 84d60ae507 Author: Nathan Cutler Date: Fri Dec 29 18:27:43 2017 +0100 Merge pull request #19680 from tchaikov/wip-luminous-22220 luminous: install-deps.sh: revert gcc to the one shipped by distro Reviewed-by: Sage Weil Reviewed-by: Nathan Cutler commit f519145585040612801416b3f23097494d46667b Merge: 88b4534a0f 96b924fc75 Author: Kefu Chai Date: Tue Dec 26 01:27:04 2017 +0800 Merge pull request #19048 from shinobu-x/wip-22196-luminous luminous: mgr[zabbix] float division by zero (osd['kb'] = 0) Reviewed-by: John Spray commit 84d60ae507db8c290f9b8fab302bdaaacce247e1 Author: Kefu Chai Date: Fri Dec 22 22:42:16 2017 +0800 install-deps.sh: update g++ symlink also we need to update g++ symlink also, if it points to the wrong version http://tracker.ceph.com/issues/22220 Signed-off-by: Kefu Chai Conflicts: the libboost issue does not affect master. as master builds boost from source. so, it's not cherry-picked from master. (cherry picked from commit 248a157635b46d3cf23e37ae263c62b0dc4e0e59) commit 54abba8cf87414f0d5444b1de78fd35fa379a461 Author: Kefu Chai Date: Thu Dec 14 21:01:43 2017 +0800 install-deps.sh: readlink /usr/bin/gcc not /usr/bin/x86_64-linux-gnu-gcc See: http://tracker.ceph.com/issues/22220 Signed-off-by: Kefu Chai (cherry picked from commit 5be6ff11e536cb492dd50dedf8a04fb9acc1222e) commit 2d377d351120d06382ae303e10a330de143f0a85 Author: Kefu Chai Date: Wed Dec 13 13:36:54 2017 +0800 install-deps.sh: point gcc to the one shipped by distro to define a struct in a method is legal in C++11, but it causes internal compiler error due to https://gcc.gnu.org/bugzilla/show_bug.cgi?id=82155 if we are using GCC-7. so we need to either workaround in our source code by moving the struct definition out of the member method or revert to a GCC without this bug. but if we go with the first route, the jewel build still fails, because GCC-7 starts to use the new CXX11 ABI, which is not compatible with the libboost we use in jewel. the libboost was still built with the old ABI for backward compatibility. so let's just fix the install-deps.sh to point gcc to the origin one. See: http://tracker.ceph.com/issues/22220 Signed-off-by: Kefu Chai Conflicts: the libboost issue does not affect master. as master builds boost from source. so, it's not cherry-picked from master. (cherry picked from commit ccc4dea90e483ea8bf6bee0721ef929e7f48ff5a) commit 1643d5a4f8f726ce27ac9f5cc8160c34c07c09bb Author: Yan, Zheng Date: Sat Dec 9 14:09:24 2017 +0800 mds: properly eval locks after importing inode We should call Locker:eval() for all imported inodes who have non-zero 'wanted caps'. MDS does not properly handle following case. - client open a inode for read, it send a cap message to MDS.a (the cap message updates 'wanted caps') - MDS.a receive the cap message, the inode is non-auth and is ambiguous auth. MDS.a can not request 'wanted caps' from auth mds. - MDS.a finishes importing the inode from. But no caps are imported and mds_caps_wanted map is empty. The bug can cause read hang. Fixes: http://tracker.ceph.com/issues/22357 Signed-off-by: "Yan, Zheng" (cherry picked from commit debb556076c5685f0d8bb8029a95684b9e552eb8) commit e3bb21529b8e5a8f483cf8dab4d0012279e0373b Author: Abhishek Lekshmanan Date: Tue Nov 28 14:09:31 2017 +0100 test/cls: add a basic test for rgw usage log A basic test for cls_rgw for usage that sets 512 usage log entries, reads them and deletes them Signed-off-by: Abhishek Lekshmanan (cherry picked from commit ca5be9fdf88ae2f35122ee0bafb5575731b872f6) commit 7810567fa37308bfeadf63e65f08adc36bfb886e Author: Abhishek Lekshmanan Date: Tue Nov 21 16:38:27 2017 +0100 cls/rgw: trim all usage entries in cls_rgw Currently trim usage will only trim upto 128 omap entries, since we need to run this in a loop until we're done, actually make the cls return -ENODATA so that we know when to stop the loop (inspired by a similar call in cls_log) this involves the following changes * return -ENODATA when iterate entries goes through and the value of iter (which is set as the last value of key when succeeded) * use IoCtx for calling the loop from within cls rather than in rgw * drop the goto call in rgw_rados since we can return once we're done processing Fixes: http://tracker.ceph.com/issues/22234 Signed-off-by: Abhishek Lekshmanan (cherry picked from commit b548a3f3443452210d92cad574bcb73ba6d2ce42) commit 7ffe8c41d3c095ebaf635aceaf2ee39d8eb58001 Author: Patrick Donnelly Date: Tue Dec 12 19:58:01 2017 -0800 memstore: write fsid to fsid not fs_fsid Fixes: http://tracker.ceph.com/issues/20736 Signed-off-by: Patrick Donnelly (cherry picked from commit e8c6882cc9c28b07ae1d5dc1169920c3ee821ed5) commit ce54fd9b995a5e28fbba462e5843c17ee58fea29 Author: Sage Weil Date: Mon Nov 20 09:30:50 2017 -0600 os/bluestore: prevent mount if osd_max_object_size >= 4G Signed-off-by: Sage Weil (cherry picked from commit f284bbb0593eafc25013c01c8aa7b8bdfc39d906) commit 47f74bcd136c3f9d0034ed1414442fdb03824bf8 Author: Patrick Donnelly Date: Mon Dec 18 18:41:31 2017 -0800 qa: don't configure ec data pool with memstore Fixes: http://tracker.ceph.com/issues/22436 Signed-off-by: Patrick Donnelly (cherry picked from commit b2284f23b813399613f8540efd2d426b3c6f9839) commit 6eea75200fed82a2c5513795a79834527090957c Author: root Date: Fri Dec 1 15:26:00 2017 +0800 cephfs: potential adjust failure in lru_expire Fix: the first adjust is no needed,it will never take real effect. the second 'adjust' may never get the chance to be executed suppose we can reach the second 'adjust', it will crash because the bottom list is empty now. Fixes: http://tracker.ceph.com/issues/22458 Signed-off-by: dongdong tao (cherry picked from commit 590c39eab02e64de7393c35ae7a9efb6ce626770) commit d0d66c73b02065dcdaf70905a69fc37c0f7a6e6e Author: root Date: Sun Dec 10 15:35:21 2017 +0800 cephfs-journal-tool: tool would miss to report some invalid range Fixes: http://tracker.ceph.com/issues/22459 Signed-off-by: dongdong tao (cherry picked from commit 7708bff40cdf059a362902156a6d0660ba21fb14) commit 3d0f4fa752751a54ee2a88faa0abbb96a55fea11 Author: Jason Dillaman Date: Fri Dec 15 14:51:46 2017 -0500 rbd-mirror: cluster watcher should ensure it has latest OSD map Fixes: http://tracker.ceph.com/issues/22461 Signed-off-by: Jason Dillaman (cherry picked from commit 9eb247961ae8bfe63c4025032ad532b7c3e3a1e7) commit 40954a9c27cb3813c8f15c2f89d7223069b9031c Author: Dan van der Ster Date: Mon Dec 18 13:53:20 2017 +0100 mgr/balancer: fix KeyError in balancer rm Fix the typo in the plan name which leads to a KeyError in balancer rm. Signed-off-by: Dan van der Ster Fixes: http://tracker.ceph.com/issues/22470 (cherry picked from commit 4f581e738ee7c9d519fc4113f6bb0d0d1ce827ff) commit 5a40ef22449481bca0223f48be8823a3c9c2b198 Author: Yan, Zheng Date: Mon Dec 11 09:06:07 2017 +0800 osdc/Journaler: introduce STATE_STOPPING state Signed-off-by: "Yan, Zheng" commit 05397e405f552bb720e12b9c0557b4c9234ffaa0 Author: Yan, Zheng Date: Mon Dec 11 08:43:57 2017 +0800 osdc/Journaler: add 'stopping' check to various finish callbacks These callbacks are executed by finisher. When they are being executed, Journaler can be in stopping state. Fixes: http://tracker.ceph.com/issues/22360 Signed-off-by: "Yan, Zheng" (cherry picked from commit 7ffa87e6a2ab8fb2c64588411c6c2ebff2f91f93) commit 4943899005267250f42909e2a91a8a54e9479da4 Author: Yan, Zheng Date: Wed Nov 29 18:44:30 2017 +0800 mds: handle client session messages when mds is stopping handle session messages except CEPH_SESSION_REQUEST_OPEN. The problem I found is that mds ignores CEPH_SESSION_REQUEST_RENEWCAPS, which causes client sessions to become stale. Locker::revoke_stale_caps() increases client caps' sequence number. This causes clients to warn about caps' sequence number mismatch when handle caps import/export message. mds should handle CEPH_SESSION_FLUSHMSG_ACK message too. Because one step of exporting subtree is flushing session messages. Fixes: http://tracker.ceph.com/issues/22460 Signed-off-by: "Yan, Zheng" (cherry picked from commit 559fdcaec28c8459c5d801efc3842e604324f91f) commit 88b4534a0fab6f7020874ff8903b2f2eb9d06eb2 Merge: 5bef6fa349 d223a8f1b9 Author: Josh Durgin Date: Mon Dec 18 13:25:15 2017 -0800 Merge pull request #18771 from ceph/wip-yuriw-22048-luminous tests - Initial checkin for luminous point-to-point upgrade Reviewed-by: Josh Durgin commit bb3d091b463dce78f147e9f1333bd667cf2836ed Author: John Spray Date: Wed Dec 13 13:08:46 2017 +0000 mgr/balancer: don't use 'foo' tags on commands This looks weird in logs when code elsewhere logs that it didn't handle a command with tag 'foo' Fixes: http://tracker.ceph.com/issues/22361 Signed-off-by: John Spray (cherry picked from commit 432d07a0fe3e9992c21539898ce5be6c6b12a745) commit 2b9bbf605e776640b07ec52d15606f6c04fe39f8 Author: Nathan Cutler Date: Wed Dec 13 18:15:27 2017 +0100 build/ops: rpm: set permissions 0755 on rbd resource agent Fixes: http://tracker.ceph.com/issues/22362 Signed-off-by: Nathan Cutler (cherry picked from commit 37e4bc3141ee36a4152834bd10e570bad7389807) commit c5cb0cab9f307f5bd1b58ce94b0a9e1fb510db9e Author: Dan van der Ster Date: Wed Dec 13 17:02:44 2017 +0100 mgr/balancer: cast config vals to int or float upmap_max_iterations and other config vals need to be numeric. Cast them appropriately. Signed-off-by: Dan van der Ster Fixes: http://tracker.ceph.com/issues/22429 (cherry picked from commit d7713e6f823dd62a33a27996e1da8e29f3f7b7c5) commit 9a9ed272a76a60d9dafebbcc8ff1356ff4d7a9a6 Author: Haomai Wang Date: Wed Nov 15 18:33:17 2017 +0800 msg/async/AsyncConnection: unregister connection when racing happened Signed-off-by: Haomai Wang (cherry picked from commit 5216309c25522e9e4a3c3a03ceb927079de91e9b) commit 5bef6fa349207cfc88356a8a24c9ab2557179132 Merge: d268faf20b e134317b81 Author: Kefu Chai Date: Fri Dec 15 20:52:14 2017 +0800 Merge pull request #19522 from ceph/wip-ceph-disk-deprecation-revert Revert "ceph-disk add deprecation warnings in favor of ceph-volume" Reviewed-By: Nathan Cutler Reviewed-by: Kefu Chai commit eb0c60f7d73a89dd7e77f4edcda83b81fa20e784 Author: Orit Wasserman Date: Thu Nov 30 12:09:10 2017 +0200 rgw: reshard should not update stats when linking new bucket instance Fixes: http://tracker.ceph.com/issues/22124 Signed-off-by: Orit Wasserman (cherry picked from commit fc95210f05367d08c679ec77d23bee6467d01980) Conflicts: src/rgw/rgw_bucket.h: i chosed to pick code from commit fc95210 because conflicts were caused by exsiting parameter and newly added parameter which is used in a method. commit d223a8f1b91c73ee558a2d28f8af812e9583353b Author: Yuri Weinstein Date: Mon Nov 6 12:22:27 2017 -0800 tests - Initial checkin for luminous point-to-point upgrade Fixes http://tracker.ceph.com/issues/22048 Signed-off-by: Yuri Weinstein commit d268faf20bc30748d6efe949dc81c12812955f39 Merge: d242b1c767 fb3b176a80 Author: Andrew Schoen Date: Thu Dec 14 22:03:23 2017 +0100 Merge pull request #19532 from ceph/luminous-rm22297 luminous ceph-volume handle inline comments in the ceph.conf file Reviewed-by: Andrew Schoen commit d242b1c7677df993eeee64fbd2a9b128ff4ddcd4 Merge: dd24747b44 1018f803bd Author: Andrew Schoen Date: Thu Dec 14 21:50:22 2017 +0100 Merge pull request #19530 from ceph/luminous-rm22326 luminous ceph-volume: warn on missing ceph.conf file Reviewed-by: Andrew Schoen commit fb3b176a80b0ada18445bd725d7670c9492509fc Author: Alfredo Deza Date: Mon Dec 4 08:22:23 2017 -0500 ceph-volume configuration allow inlined comments for # and ; Signed-off-by: Alfredo Deza (cherry picked from commit aec5f5042a684952c2b07922695e7675e13645ec) commit 8fad755dd5ab1de78c636cb93dce5864cc9f07a2 Author: Alfredo Deza Date: Mon Dec 4 08:21:29 2017 -0500 ceph-volume tests verify that INI comments can be inlined Signed-off-by: Alfredo Deza (cherry picked from commit fe6c4c0d6055c46baadd89723c0a2b78b5ffb7f8) commit dd24747b44bd70ef82ff81efe835d7eceefb4d22 Merge: e3f0c349b0 8748c8fb84 Author: Alfredo Deza Date: Thu Dec 14 13:48:53 2017 -0500 Merge pull request #19525 from ceph/backport-19363 luminous: ceph-volume: removed the explicit use of sudo Reviewed-by: Alfredo Deza commit 1018f803bde2f748ea5b15928d587f80f9bc938d Author: Alfredo Deza Date: Tue Dec 5 13:59:22 2017 -0500 ceph-volume test logging ignored ceph.conf file Signed-off-by: Alfredo Deza (cherry picked from commit 69482d1d8059c3b264532f7623c9e6592c57b7c3) commit 72dfb268fab89bd2fa93e7d9af0894498df6c45f Author: Alfredo Deza Date: Tue Dec 5 13:58:53 2017 -0500 ceph-volume main warn on inability to load ceph.conf, don't raise Signed-off-by: Alfredo Deza (cherry picked from commit 44c768db0da0163340b52643122b66ecebbefe23) commit e3f0c349b0ea34378bcbc183c1ea366aa484c5c4 Merge: 4909437230 2b06a7f413 Author: Andrew Schoen Date: Thu Dec 14 18:30:52 2017 +0100 Merge pull request #19528 from ceph/luminous-rm22305 luminous ceph-volume should be able to handle multiple LVM (VG/LV) tags Reviewed-by: Andrew Schoen commit 4909437230d8ba9f38205c4ee6ed94a4b0b549e8 Merge: d87dbedc8f 21924133a9 Author: Andrew Schoen Date: Thu Dec 14 18:26:30 2017 +0100 Merge pull request #19527 from ceph/luminous-rm22299 luminous ceph-volume Format correctly when vg/lv cannot be used Reviewed-by: Andrew Schoen commit d87dbedc8f83fa4c1d4ec5698e53b27e852de788 Merge: c7383d20c4 775369d5c8 Author: Andrew Schoen Date: Thu Dec 14 18:22:22 2017 +0100 Merge pull request #19526 from ceph/luminous-rm22280 luminous ceph-volume: handle leading whitespace/tabs in ceph.conf Reviewed-by: Andrew Schoen commit 2b06a7f413796e2b42faca1172ee97f60adf2f1b Author: Alfredo Deza Date: Mon Dec 4 09:56:00 2017 -0500 ceph-volume tests.api ensure that we can handle non-ceph tags in lvs Signed-off-by: Alfredo Deza (cherry picked from commit c98731ecede09d8d15dca99c57e331212747060e) commit 47d46678449c13a1aa45f0ae08cecac6397950a1 Author: Alfredo Deza Date: Mon Dec 4 09:54:35 2017 -0500 ceph-volume api.lvm only consider 'ceph.' tags in logical volumes Signed-off-by: Alfredo Deza (cherry picked from commit a7e7df14602a10190a2830262a67202e7ad38f49) commit 21924133a9c8f954d86ca9da531867d09dd6298b Author: Alfredo Deza Date: Fri Dec 1 13:53:39 2017 -0500 ceph-volume tests verify proper formatting of RuntimeError on vg/lv error Signed-off-by: Alfredo Deza (cherry picked from commit d7320313a15ad8bb948e501382269c8f33a8cb70) commit 9119839d6ae002fa602901340ef28271833ce08d Author: Alfredo Deza Date: Fri Dec 1 13:53:06 2017 -0500 ceph-volume lvm.prepare correctly format argument for vg/lv error Signed-off-by: Alfredo Deza (cherry picked from commit 7c0b00615d46a9451cb3a7731594cd7864eabc07) commit 775369d5c8832cbbb9815aeb0de8c36f33aabfd8 Author: Alfredo Deza Date: Thu Nov 30 08:58:37 2017 -0500 ceph-volume test leading whitespace is handled in configurations Signed-off-by: Alfredo Deza (cherry picked from commit ec7f8a27b8c51e1e1d02a2a13d9219bbb1a58c26) commit 89c9183aad044f2811dafa7e072a1db2f3412d31 Author: Alfredo Deza Date: Thu Nov 30 08:58:04 2017 -0500 ceph-volume trim tabbed/whitespaced configuration files when loading them Signed-off-by: Alfredo Deza (cherry picked from commit 9706e8cc9eb53d0f01fc78978625c1d18bf4667e) commit 8748c8fb84c23ab747aaed8acae5e5f8ce88f659 Author: Andrew Schoen Date: Wed Dec 6 10:40:55 2017 -0600 ceph-volume: removed the explicit use of sudo This will allow users to run ceph-volume on systems that do not have sudo installed. Fixes: http://tracker.ceph.com/issues/22282 Signed-off-by: Andrew Schoen (cherry picked from commit aee71a3f5169043f750fd768e7fea9c74309e12d) commit e134317b8162efd3a6fa25c1c803e43228535f38 Author: Alfredo Deza Date: Thu Dec 14 08:28:58 2017 -0500 Revert "ceph-disk add deprecation warnings in favor of ceph-volume" This reverts commit b8bf0d047868054135592188c7ebe186181310c5. Deprecation warnings for ceph-disk will no longer be present in any Luminous release beyond 12.2.2 - but are still present in master and any newer release. Signed-off-by: Alfredo Deza commit 7dabbe91310cb0f97b9d6d09f697f74be2a8ac80 Author: Radoslaw Zarzynski Date: Thu Aug 17 15:05:41 2017 -0400 rgw: incorporate the Transfer-Encoding fix for CivetWeb. This commit updates the version of CivetWeb used for RadosGW to get the fix for handling clients that send Content-Length together with Transfer-Encoding. The current handling differs from S3 and thus affects AWSv4. Fixes: http://tracker.ceph.com/issues/21015 Signed-off-by: Radoslaw Zarzynski (cherry picked from commit c9e1daeffb10f36a191960328977f09ef2c05c19) commit 72de713bf697c4e444adf80fe76df87873119f1a Author: Jeegn Chen Date: Thu Nov 16 09:12:24 2017 +0800 rgw: AWS v4 authorization work when INIT_MULTIPART is chunked Add RGW_OP_INIT_MULTIPART as a the single chunk special case like RGW_OP_COMPLETE_MULTIPART. Fixes: http://tracker.ceph.com/issues/22129 Signed-off-by: Jeegn Chen (cherry picked from commit c8827e5017ce81377cb174ad74cc2f44c3368457) commit ca980a63c3125a356cff79eba9df06fd2b37d731 Author: Jos Collin Date: Wed Nov 22 17:20:58 2017 +0530 pybind: return error message when ceph_mds_command() returns error Returned the error message when ceph_mds_command() returns error. Signed-off-by: Jos Collin (cherry picked from commit 941b58c968f6b0e359f279c8bc9e7decf51e75d1) commit 42bb91a258512c34ea402f34f130518069db3835 Author: Jos Collin Date: Wed Nov 22 17:12:43 2017 +0530 mds: Fix error message when mds not active Fix error message for mds not active state. Also fixed the 'unrecognized command' logic by avoiding the new stringstream local object. Signed-off-by: Jos Collin (cherry picked from commit 10f93cf5828afbc8ff25ef81b2437ef5c0362396) commit feb4d7ba37260599952c076c23033288e671f30e Author: PCzhangPC Date: Fri Oct 20 17:07:41 2017 +0800 test:add a test case in test_librbd Signed-off-by: PCzhangPC (cherry picked from commit be5ad8764f65f43b6844eaa99c591a31c4d3af82) commit 13ef77c450a7f108233a6fa5f15d72393db33feb Author: PCzhangPC Date: Mon Oct 16 22:46:40 2017 +0800 rbd:can not copy all image-metas if we have more than 64 key/value pairs Signed-off-by: PCzhangPC (cherry picked from commit 71178643b2361557f31094e2490d4124b27cd7ff) commit 1507015169f062cbaa9b3b1e652bb34ba5d586b5 Author: PCzhangPC Date: Sat Oct 28 14:29:45 2017 +0800 test_librbd:add a test case of 70 key/val pairs in TestClone2 Signed-off-by: PCzhangPC (cherry picked from commit 85713bdcc3870728dce02f87ee3cfd36257ae533) commit c4715235c2b3d28c3aaa4584f58d3a49a89daf79 Author: PCzhangPC Date: Sat Oct 21 11:31:02 2017 +0800 test_librbd:add a test case of 70 key/val pairs in TestClone Signed-off-by: PCzhangPC (cherry picked from commit 3e08577a296bc4ce2c1592849108926b45e9b541) commit fecfd013a8d2e251057840ee1fd112e20af45a5e Author: PCzhangPC Date: Fri Oct 20 14:21:09 2017 +0800 librbd: cannot clone all image-metas if we have more than 64 key/value pairs Signed-off-by: PCzhangPC (cherry picked from commit ccc56384032be7f6bd48e28b6825b3ce589c7cf7) commit 27a1a7f6fa6e5368fe1b3b56acfd81dbc49f85c6 Author: Yao Zongyou Date: Mon Oct 9 17:19:45 2017 +0800 rgw: fix command argument error for radosgw-admin. Fixes: http://tracker.ceph.com/issues/21723 Signed-off-by: Yao Zongyou (cherry picked from commit bba3a89591683e6129a514af3144f1c0e8e1cc05) commit 5e5d5c8196cf99137009be049cca159aca9396ea Author: Sage Weil Date: Wed Nov 29 10:29:52 2017 -0600 qa/suites/rados/thrash: extend mgr beacon grace when many msgr failures injected Fixes: http://tracker.ceph.com/issues/21147 Signed-off-by: Sage Weil (cherry picked from commit 431d1482ffef80f9e52826a0425cae899c5dfbda) commit c22f06b3c5ee51f7bc6f4d1a0340ed7f63c106a9 Author: Sage Weil Date: Fri Sep 15 10:17:07 2017 -0400 osd/PrimaryLogPG: move cache_mode==none check to top Signed-off-by: Sage Weil (cherry picked from commit 5a26ac4d0f643f90d1f513ee33081faa9b0a7e7d) commit 21880df6bf27465637c97e504b5a6dcf2ea31d9f Author: Sage Weil Date: Mon Aug 7 18:42:57 2017 -0400 osd/PrimaryLogPG: send requests to primary on cache miss If a client has {BALANCE,LOCALIZE}_READS and sends a request to a replica, but the object isn't in the cache, send them back to the primary. Otherwise we might do something rash (like trigger a promotion from a replica). Fixes: http://tracker.ceph.com/issues/20919 Signed-off-by: Sage Weil (cherry picked from commit 741a8720996b74434a036b143209111ce5203cbc) commit 4085e87b84a788a4285b5f18f1cd64556c08038d Author: Dmitry Plyakin Date: Wed Nov 29 16:03:02 2017 +0300 rgw: fix GET website response error code Change NoSuchKey error code to NoSuchWebsiteConfiguration, when bucket doesn't have website configuration. Fixes: http://tracker.ceph.com/issues/22272 Signed-off-by: Dmitry Plyakin (cherry picked from commit 56344f0e147e1781bb359bfde6878511b077487f) commit de05d2c81913e608ca9089b40df971f339ed0ba9 Author: Jason Dillaman Date: Fri Oct 6 15:03:34 2017 -0400 librbd: refresh image after applying new/removing old metadata Fixes: http://tracker.ceph.com/issues/21711 Signed-off-by: Jason Dillaman (cherry picked from commit ad01a883c5d9139d3ad3176a39a538bb8b247388) commit 235032ec642a7da077b04b26147cfd86e3ce3d86 Author: Jason Dillaman Date: Thu Sep 28 14:00:29 2017 -0400 rbd-mirror: sync image metadata when transfering remote image Fixes: http://tracker.ceph.com/issues/21535 Signed-off-by: Jason Dillaman (cherry picked from commit 9aa5dfb8ec46e8babd0f167dcc0e234a7c60a50c) commit 35bb3d4d17db84eac03496d22969b69ee6111b2b Author: Jeff Layton Date: Tue Oct 24 08:49:27 2017 -0400 mds: fold mds_revoke_cap_timeout into mds_session_timeout Right now, we have two different timeout settings -- one for when the client is just not responding at all (mds_session_timeout), and one for when the client is otherwise responding but isn't returning caps in a timely fashion (mds_cap_revoke_timeout). The default settings on them are equivalent (60s), but only the mds_session_timeout is communicated via the mdsmap. The mds_cap_revoke_timeout is known only to the MDS. Neither timeout results in anything other than warnings in the current codebase. There is also a third setting (mds_session_autoclose) that is also communicated via the MDSmap. Exceeding that value (default of 300s) could eventually result in the client being blacklisted from the cluster. The code to implement that doesn't exist yet, however. The current codebase doesn't do any real sanity checking of these timeouts, so the potential for admins to get them wrong is rather high. It's hard to concoct a use-case where we'd want to warn about these events at different intervals. Simplify this by just removing the mds_cap_revoke_timeout setting, and replace its use in the code with the mds_session_timeout. With that, the client can at least determine when warnings might start showing up in the MDS' logs. Signed-off-by: Jeff Layton (cherry picked from commit 3321cc7b375a5e0ea1da4ab197ab447639ca4db3) commit 9981a1db90984b18431017d348f0a3e7ffb76f61 Author: Jeff Layton Date: Wed Oct 18 07:27:49 2017 -0400 client: add new delegation testcases Test basic acquire/break functionality from both other clients and the same client, for different conflicting opens, as well as changes to the namespace. Then test delegation timeout behavior. Open file, take delegation in main thread. Spawn another thread to open the file again, breaking delegation. Have main thread ignore it, and wait for the spawned thread to be joined. Once it is, ensure that subsequent access of the cmount returns -ENOTCONN. Signed-off-by: Jeff Layton (cherry picked from commit d9c6a9129eb65031eb1fc6f769cfe59bf3bb1cff) commit 0aa3c5c577ce4011b727123b45e89648bba32596 Author: Jan Fajerski Date: Fri Dec 8 16:13:19 2017 +0100 mon: reenable timer to send digest when paxos is temporarily inactive Fixes: http://tracker.ceph.com/issues/22142 Signed-off-by: Jan Fajerski (cherry picked from commit 0457043d2f0040629ec425a356b1bb1f96cf0332) commit 9490a1551dcb09b9d05e5e49632dfb3277f13981 Author: Jeff Layton Date: Tue Nov 14 07:26:56 2017 -0500 client: add delegation support for cephfs Add the ability for ceph userland clients to request a delegation for a cephfs open file. With this, userland will get a callback if there is a request for access that conflicts with the delegation. The client is then expected to return the delegation at which point the conflicting access will succeed. Handing out a delegation means that we're trusting the application to give back the caps when they are needed elsewhere. If it fails to uphold its end of the bargain, we need to take steps to forcibly revoke them, so that other clients can make progress. When that occurs, shut down the mount such that further calls return -ENOTCONN. That should prevent the application from holding on to caps indefinitely, and ensure that it will no longer do damage. It should also prevent the client from ending up blacklisted. Since this is an implicit agreement between ceph and the application, we require that the application call ceph_set_deleg_timeout() to set the delegation return timeout before we hand out any delegations. This helps ensure that the various application and ceph cluster timeouts are all in agreement. An open call on ceph can easily return without any caps being granted to the client. This is not generally a problem on ceph since it will wait on caps before doing any operations on that open Fh, but it's less than ideal when there is a delegation outstanding. Both NFS and SMB servers can grant deny locks, and at least in the case of NFS, those can be cached such that the server is unaware of them if the client holds a delegation. Most applications that use delegations will also want to ensure that delegations are broken before allowing an open to proceed. A non-zero delegation return timeout also cues the client to wait on a minimal set of caps after an open return to ensure that this is the case. Signed-off-by: Jeff Layton (cherry picked from commit fad99776195e299b37071892e459ad8abf2fddfc) Conflicts: src/client/Inode.cc (luminous does not have a5d97ad257d6fae8f66200513805e1778cb0c8ba so we preserve the following lines at the end of the Inode dtor: delete fcntl_locks; delete flock_locks; ) commit 3a613f6966ef3f32eb5f26ec5ebc5772a4dc2bf0 Author: Jeff Layton Date: Thu Oct 12 08:29:28 2017 -0400 common: remove data_dir_option from common_preinit and global_pre_init No one ever passes anything in there. Signed-off-by: Jeff Layton (cherry picked from commit 85f2315bc59106b8fba6099c5e79209ea9b96efa) commit 5e8c4f83f4f161a595e75340cecbdd41b1803f39 Author: Mykola Golub Date: Tue Dec 5 15:48:58 2017 +0200 pybind/rbd: raise KeyError when metadata does not exist Signed-off-by: Mykola Golub (cherry picked from commit 7a1b13f8849ce7bd325316bafad5bab660c77825) commit 2d7bea6148756fd8f3f7e124148d296d4290691f Author: Mykola Golub Date: Tue Dec 5 15:48:23 2017 +0200 pybind/rbd: fix metadata functions error handling Fixes: http://tracker.ceph.com/issues/22306 Signed-off-by: Mykola Golub (cherry picked from commit 2e6872b28eccd10b8bafdadaf3e6049123792022) commit c7383d20c4258984de2fb524f2751207aa150d78 Merge: b3fd46810c 5bd076ef41 Author: Sage Weil Date: Tue Dec 12 09:04:20 2017 -0600 Merge pull request #19042 from liewegas/wip-22128 mon/OSDMonitor: fix ruleset-* to crush-* fixup Reviewed-by: Greg Farnum Reviewed-by: xie xingguo commit 08a2358f8520efe03eac435d1f8b2b30e936a021 Author: Jason Dillaman Date: Fri Sep 29 12:22:57 2017 -0400 librbd: avoid dynamically refreshing non-atomic configuration settings Fixes: http://tracker.ceph.com/issues/21529 Signed-off-by: Jason Dillaman (cherry picked from commit ede691323d94dc04a30f81aca5576a3d6d1930af) commit caf17a876e75cd448947b7b729a8e39689a304df Author: Dongsheng Yang Date: Fri Aug 11 17:44:19 2017 +0800 librbd: notify watcher when updating image metadata Signed-off-by: Dongsheng Yang (cherry picked from commit b10d26dfa84627b2622d405d272b1133bb773245) Conflicts: src/librbd/image/RefreshRequest.h (luminous does not have 233482cf708aa7b4723c529d387c4605bdf0722f so we leave the following variables uninitialized: uint8_t m_order; uint64_t m_size; uint64_t m_features; uint64_t m_incompatible_features; uint64_t m_flags; ) commit 34450ed6c0a324ed86fd47770662554e7cdfeae2 Author: Nathan Cutler Date: Mon Dec 11 17:09:25 2017 +0100 doc: globally change CRUSH ruleset to CRUSH rule Since kraken, Ceph enforces a 1:1 correspondence between CRUSH ruleset and CRUSH rule, so effectively ruleset and rule are the same thing, although the term "ruleset" still survives - notably in the CRUSH rule itself, where it effectively denotes the number of the rule. This commit updates the documentation to more faithfully reflect the current state of the code. Fixes: http://tracker.ceph.com/issues/20559 Signed-off-by: Nathan Cutler (cherry picked from commit aea9fa01ae630f4ffd2744f577723f5305c718b4) Conflicts: (trivial resolution) doc/man/8/ceph.rst doc/rados/configuration/pool-pg-config-ref.rst doc/rados/operations/pools.rst commit b013f7fdb5b57725f01fc5305d94d32d7599f732 Author: fang.yuxiang Date: Fri Aug 4 11:19:00 2017 +0800 rgw: revert PR #8765 As talk with cbodley in PR(#16716), we shouldn't not mix such dangerous configuration reload with periodically SIGHUP for logs zip. Signed-off-by: fang yuxiang fang.yuxiang@eisoo.com (cherry picked from commit 2f30bbb492db3df0522888905d497174e36d0596) commit b3fd46810c01be00ebf26b9893d8bab107ee93f2 Merge: 092ea5174c f6fb266699 Author: Kefu Chai Date: Fri Dec 8 10:54:12 2017 +0800 Merge pull request #19355 from tchaikov/wip-18589-luminous luminous: build/ops: move ceph-*-tool binaries out of ceph-test subpackage Reviewed-by: Ken Dreyer commit 1d6644d091ab6c3aa64879e0ae68aa697877f32d Author: Vasu Kulkarni Date: Wed Dec 6 12:13:40 2017 -0800 qa/tests: Add debug info when creating ceph volumes Signed-off-by: Vasu Kulkarni (cherry picked from commit 9f1a1e0543da6be15b75c3168603cc84a62bd157) commit 0616bcba07b24ff6aced3048753f8e6535aedf06 Author: Vasu Kulkarni Date: Fri Dec 1 14:16:45 2017 -0800 qa/tests: add tests for ceph-volume Signed-off-by: Vasu Kulkarni (cherry picked from commit 95fb43b54d843e4a7bedaad72b4a71c3489f0dea) commit bccaa6e627f32725698b544499cab44f3ed72807 Author: Vasu Kulkarni Date: Fri Dec 1 14:11:55 2017 -0800 qa/tests: update tests to use new ceph-volume syntax Signed-off-by: Vasu Kulkarni (cherry picked from commit 3ecaebd0d884033c55a501d218ab426385d92357) commit bd558d7e729005aece735fedadeb3325efae235d Author: Vasu Kulkarni Date: Thu Dec 7 17:56:55 2017 -0800 qa/tests: default to wip branch for final upgrade. Signed-off-by: Vasu Kulkarni commit 3f28b195d3f391f1a9e36073209f2c895dee89f9 Author: Vasu Kulkarni Date: Thu Dec 7 17:47:01 2017 -0800 qa/tests: when no branch is specified, use wip branch under test for upgrade Signed-off-by: Vasu Kulkarni (cherry picked from commit 307e52109411e237e34b824da78fa5b175b16a9d) commit f6fb266699f7a39005cb76a2c878bdff5b3a7ff0 Author: Kefu Chai Date: Wed Dec 6 10:19:09 2017 +0800 debian/control: adjust ceph-{osdomap,kvstore,monstore}-tool feature move this is a follow-up of #19328. we need to get this change into 12.2.3. so better off do the switch somewhere after 12.2.2 which has been tagged, and before 12.2.3, which is not tagged yet. please note, this is not targetting master, because i want to make sure the change number (the in << 12.2.2-) is correct. it does not hurt if it's not, as long as it is ">> 12.2.2", so the replace machinery in 12.2.3 works, and it covers the releases where the ceph-{osdomap,kvstore,monstore}-tool are not move yet. but why don't make it more right? Signed-off-by: Kefu Chai (cherry picked from commit 57bb57f76d733d6e6dea4fc0682058e4e6abe7df) commit ee06a16ee8c8434f75adf05c04f94607a5ec090f Author: Yan, Zheng Date: Wed Nov 22 12:33:09 2017 +0800 mds: remove useless check in Migrator::export_dir There is an assert(dest != mds->get_nodeid()) at very beginning of the function. There also is a check for if 'dest' mds is active. Signed-off-by: "Yan, Zheng" (cherry picked from commit 7afe92f644c97ae1282837bc75cde0b4a8054d0f) commit 443fa3546593f667b01cca7466e3692223c4f226 Author: Yan, Zheng Date: Wed Nov 22 11:49:43 2017 +0800 mds: ignore export pin for unlinked directory Otherwise, stray directory inode may have pinned subtree dirfrag. The subtree dirfrag prevents stray inode from getting purged. Fixes: http://tracker.ceph.com/issues/22219 Signed-off-by: "Yan, Zheng" (cherry picked from commit b7cee60467582d7398bbe85e051a722e30c0897e) commit d57d0945dbb6ec4b46e1749fe7989b48794c3544 Author: Sage Weil Date: Mon Dec 4 21:25:16 2017 -0600 debian/control: adjust ceph-{osdomap,kvstore,monstore}-tool feature move The backport didn't make 12.2.2, but it will be in 12.2.3. Fixes: http://tracker.ceph.com/issues/22319 Signed-off-by: Sage Weil (cherry picked from commit e0c814266fe653311673e07bd7c7dbb51d866f82) commit 00503d1880e829024cd1d5e10cca0d388718dedb Author: liuchang0812 Date: Fri Aug 18 21:52:52 2017 +0800 doc: add ceph-kvstore-tool's man Signed-off-by: liuchang0812 (cherry picked from commit 0667db4eb61338887743d17c93a61c0e89868b08) commit 92093824d5deddad4dfe4219fee3c95fe4c38f55 Author: Kefu Chai Date: Mon Oct 23 20:15:16 2017 +0800 debian: fix package relationships after d3ac8d18 d3ac8d18 moves ceph-client-debug from ceph-test to ceph-base without updating the package relationships between the two involved packages. which results in: dpkg: error processing archive /var/cache/apt/archives/ceph-test_12.2.1-241-g43e027b-1trusty_amd64.deb (--unpack): trying to overwrite '/usr/bin/ceph-client-debug', which is also in package ceph-base 10.2.10-14-gcbaddae-1trusty dpkg-deb: error: subprocess paste was killed by signal (Broken pipe) Signed-off-by: Kefu Chai (cherry picked from commit ed988fc660c5da3cb97e48e82ac5b7ad8f3bcd41) commit a566ab942b35af11f2cd0fd269f48a4e9fdc3a66 Author: Kefu Chai Date: Mon Oct 23 15:26:35 2017 +0800 debian: fix package relationships after 40caf6a6 we have issues when running upgrade tests: dpkg: error processing archive /var/cache/apt/archives/ceph-osd_13.0.0-2201-g6cc0b41-1trusty_amd64.deb (--unpack): trying to overwrite '/usr/bin/ceph-osdomap-tool', which is also in package ceph-test 10.2.10-14-gcbaddae-1trusty in 40caf6a6, we moves some tools from ceph-test out into ceph-osd, ceph-mon and ceph-base respectively. but didn't update the relationships between these packages accordingly. this causes the upgrade failure. see https://www.debian.org/doc/debian-policy/#document-ch-relationships for more details on "Breaks" and "Conflicts". the reason why the package version to be replaced/conflicted is 12.2.2 is that: i assume that this change will be backported to luminous, and the next release of it will be 12.2.2 . Signed-off-by: Kefu Chai (cherry picked from commit 50dad805d9f635f2d8085237e94ee8fd0948dd3c) commit 09fd3c730804e094624cbbbd3e6c4481d1a3a86e Author: Nathan Cutler Date: Fri Oct 13 10:34:52 2017 +0200 build/ops: deb: move ceph-*-tool binaries out of ceph-test subpackage ceph-osdomap-tool into ceph-osd subpackage ceph-monstore-tool into ceph-mon subpackage ceph-kvstore-tool into the ceph-base subpackage Fixes: http://tracker.ceph.com/issues/21762 Signed-off-by: Nathan Cutler (cherry picked from commit 40caf6a6d85fbde930b74df57a62381f6225c3fd) commit 6dba25e39d07aeb7fe073fc77f3af054c3481ce0 Author: Nathan Cutler Date: Fri Oct 13 10:34:52 2017 +0200 build/ops: rpm: move ceph-*-tool binaries out of ceph-test subpackage ceph-osdomap-tool into ceph-osd subpackage ceph-monstore-tool into ceph-mon subpackage ceph-kvstore-tool into the ceph-base subpackage Fixes: http://tracker.ceph.com/issues/21762 Signed-off-by: Nathan Cutler (cherry picked from commit d7b493a7108a68302bc0f48337bf3c253a720266) commit 092ea5174c069f060bcccc1fa044d5e67da3938c Merge: 117e91ef29 753279aac6 Author: Alfredo Deza Date: Tue Dec 5 07:01:35 2017 -0500 Merge pull request #19299 from tchaikov/wip-pr-19196-luminous luminous: ceph-disk: fix signed integer is greater than maximum when call major Reviewed-by: Alfredo Deza commit c514d3d427176eb957c3cd57b99b7ed590468ed7 Author: Yan, Zheng Date: Tue Nov 28 17:06:47 2017 +0800 mds: handle client reconnect gather race Fixes: http://tracker.ceph.com/issues/22263 Signed-off-by: "Yan, Zheng" (cherry picked from commit e4ecf26a8ef622f8290129a682afb1c64b3e7e00) commit c3c3bb0bc90c980214ff3247823dfbf06046ff95 Author: Adam Wolfe Gordon Date: Fri Sep 29 15:32:38 2017 +0000 doc: Update rbd-mirror docs to reflect data pool selection changes Signed-off-by: Adam Wolfe Gordon (cherry picked from commit 57745b94394ac65436a17249733a22c463d51c0c) commit b634fdaaf0b0f998a62c52ec59a40a4709a41331 Author: Adam Wolfe Gordon Date: Tue Sep 26 20:30:06 2017 +0000 rbd-mirror: Improve data pool selection when creating images Previously we used the source image's data pool name unconditionally. There were two problems with that: 1. If a pool with the same name didn't exist locally, creation of the local image would fail. 2. If the local pool had a default data pool configured it would be ignored. Change local image creation so it uses the default pool if configured, and uses the remote pool name only if a pool with that name exists locally. If neither of those is true, leave the data pool unset. Signed-off-by: Adam Wolfe Gordon (cherry picked from commit 2e239c05518e67b3db80500dbdd3fc5dde30e443) commit 753279aac6beeacda7fb7c7d15129dff1a27d214 Author: Song Shun Date: Tue Nov 28 11:28:43 2017 +0800 ceph-disk: fix signed integer is greater than maximum when call major fix signed integer is greater than maximum when call os.major using python 2.7.5 in Centos 7 Signed-off-by: Song Shun (cherry picked from commit f77934b19939796d7ab52daf4dac44846a2ad162) commit 117e91ef2986f65b059203e23d6efe689a182fb5 Merge: cf0baeeeeb ce90e0b764 Author: Alfredo Deza Date: Fri Dec 1 09:57:44 2017 -0500 Merge remote-tracking branch 'gh/luminous' into luminous commit ce90e0b7648587c4dc8bf354abbfaa15b63b50b1 Merge: 9c23617d6d 6d02b2a1a2 Author: Kefu Chai Date: Fri Dec 1 13:48:29 2017 +0800 Merge pull request #19001 from tchaikov/wip-pr-18848-luminous luminous: mgr/dashboard: fix audit log loading Reviewed-by: John Spray commit 9c23617d6d3e3f6d6d21de0156e17f9b7cb5f0e9 Merge: 83684b91a3 77258ed181 Author: Kefu Chai Date: Fri Dec 1 13:47:17 2017 +0800 Merge pull request #19270 from tchaikov/wip-pr-19225-luminous luminous: qa/suites/rados/singleton: more whitelist Reviewed-by: Sage Weil commit 77258ed181dbe04e56da249a71edd3220baf954a Author: Kefu Chai Date: Wed Nov 29 13:51:49 2017 +0800 qa/suites/rados/singleton: more whitelist * SLOW_OPS is normal in a cluster with flattering OSDs * so is OBJECT_MISPLACED. Signed-off-by: Kefu Chai (cherry picked from commit 0e987665fe480530a0d1664392604713828bfa5c) commit 3bfb493fa0c61f33ac930320f7b69ce4a96c2eb6 Author: Kefu Chai Date: Tue Nov 28 15:21:55 2017 +0800 cmake,common/RWLock: check for libpthread extensions pthread_rwlockattr_setkind_np() is a GNU extension of libpthread. and Tianshan Qu pointed out, we cannot use ifdef(PTHREAD_RWLOCK_PREFER_WRITER_NONRECURSIVE_NP) to detect the availability of this function, because it's an enum not a macro. so, like other *_np() extensions, we check this one also using cmake at the configure phase. Reported-by: Tianshan Qu Signed-off-by: Kefu Chai commit 5cec46600de8b60385b43cba1e7dc4d1f9a380ac Author: Dan Mick Date: Wed Nov 29 15:29:17 2017 -0800 ceph_disk: allow "no fsid" on activate The intent was to allow "no fsid" configurations when only one conf file named 'ceph.conf' was present, but the code has a bug in that ceph-osd --show-config-value will return a default all-0 uuid. Treat 'all-0' as we were treating 'None'. Signed-off-by: Dan Mick (cherry picked from commit 4d010fb83d3cad50953cdf010d1bb20c62588b3c) commit 7e8dc9763039bf784030480f60b2554f6ff7ffd5 Author: Kefu Chai Date: Thu Nov 30 19:32:05 2017 +0800 ceph-disk: silence deprecate warnings while testing Signed-off-by: Kefu Chai (cherry picked from commit ecd4f69c12f57b0d2eacd18bd4bef30f1ce24dd2) commit d0b0e03ef37a20760e702805fdc95862ec3daeba Author: Kefu Chai Date: Thu Nov 30 19:31:09 2017 +0800 ceph-disk/tests: mock get_fsid should offer a valid fsid for the ceph-disk under testing. Signed-off-by: Kefu Chai (cherry picked from commit cc49d1d2b99dcc49c60db8ea1ee90caf58650aa4) commit 747d05c298b13c1d4350b90beb8cec8828262332 Author: Igor Fedotov Date: Thu Nov 30 16:05:20 2017 +0300 luminous: rocksdb: fixes early metadata spill over to slow device in bluefs. Fixes http://tracker.ceph.com/issues/22264 Signed-off-by: Igor Fedotov commit c1c539f1efc8f2fe448bf3c4cbbed8f6401ab24d Author: Sage Weil Date: Wed Nov 29 15:20:59 2017 -0600 mon/Monitor: fix statfs handling before luminous switchover happens After the mons are luminous but before we switch over to using the MgrStatMonitor's new info, the version on mgrstat will generally be << than that of pgmon, and the client will send that version with the request. This means that the statfs message will perpetually appear to be in the future and fail the is_readable() check. Fix this with any ugly hack that resets the version to 1 if we haven't completed the luminous upgrade yet. Signed-off-by: Sage Weil commit 82c9e3dbe1b6ff04feae76a089d1a821c3ed819f Author: Sage Weil Date: Mon Nov 27 16:45:25 2017 -0600 mon/MgrMonitor: limit mgrmap history Keep 500 by default (like we do osdmaps and mdsmaps). Fixes: http://tracker.ceph.com/issues/22257 Signed-off-by: Sage Weil (cherry picked from commit 00fa8b10990f945f3e875c9850eedeb65af3fd2e) [removed const from get_trim_to() for luminous] commit bdb9d385a5e1675ac2832300e3cfa1ba6b746ce2 Author: Kefu Chai Date: Mon Nov 20 15:42:09 2017 +0800 cmake: only create sysctl file on linux and check 64bit platform by using the sizeof(void*) Signed-off-by: Kefu Chai (cherry picked from commit 89a48189ea7b9bd58db1bc95ccd7b3d9fb9b1bb3) commit acccd2332cd0bc0f8b8f00276ce25a9582c083d7 Author: David Disseldorp Date: Tue Nov 14 16:32:39 2017 +0100 sysctl.d: set kernel.pid_max=4194304 on 64-bit systems For CONFIG_BASE_FULL Linux kernels, the maximum number of proc/thread IDs is set to 32768 by default. This default limit can be quite easily hit during recovery on nodes with high OSD counts. To avoid hitting the pid_max default limit, attempt to configure it to 4194304, which corresponds to the maximum limit possible on 64-bit CONFIG_BASE_FULL kernels. Fixes: http://tracker.ceph.com/issues/21929 Signed-off-by: David Disseldorp (cherry picked from commit 14a0c2a727dd71560c3cf18171378ccde6e14699) commit 718528304063c7ea03d7777c3eaf6e26a0900f59 Author: David Disseldorp Date: Tue Nov 14 15:55:39 2017 +0100 sysctl: relocate 90-ceph-osd.conf under etc/sysctl/ Signed-off-by: David Disseldorp (cherry picked from commit 4d0b70068894af2d9c2c4b6fea1451a47864bfd8) commit 8f67e4553ac67ff062f3d9d1187a3bd67f02f020 Author: tangwenjun Date: Mon Nov 20 15:28:29 2017 +0800 os/bluestore: fix the allocate in bluefs when bluefs succeed to reserve but failed to alloc in db space, it will cause a assert, just because of the space fragmentation. in this situation, it could not use slow device space, and it would happen in stupid or avl allocator. Signed-off-by: tangwenjun (cherry picked from commit d4f868ae7b9df723a9f785cbe6caee1f718e5e33) commit 17ff1f1fd2bfa744c141741af9dc81761218da63 Author: Jason Dillaman Date: Mon Nov 20 11:03:17 2017 -0500 rbd: disk usage on empty pool no longer returns an error message Fixes: http://tracker.ceph.com/issues/22200 Signed-off-by: Jason Dillaman (cherry picked from commit f4528122d66d1bc7dd07a64d1af7b65fe53f7b0b) commit 4dda1b6ead6b1f04f996403881f61e9b7d94dba0 Author: Patrick Donnelly Date: Sun Nov 19 15:08:18 2017 -0800 client: anchor Inode while trimming caps This prevents the Inode from being deleted until after cap trimming is finished. In particular, this prevents remove_all_caps from being called which screws up the traversal of caps in trim_caps. Fixes: http://tracker.ceph.com/issues/22157 Signed-off-by: Patrick Donnelly (cherry picked from commit 1439337e949c9fcb7d15eb38c22d19eb57d3d0f2) commit 05b60db8ef1999bb7a75f7cbbc721f64e67cec48 Author: Pavan Rallabhandi Date: Wed Nov 8 21:35:54 2017 +0530 rgw: Fix swift object expiry not deleting objects In cls_timeindex_list() though `to_index` has expired for a timespan, the marker is set for a subsequent index during the time boundary check. This marker is further returned to RGWObjectExpirer::process_single_shard(), where this out_marker is trimmed from the respective shard, resulting in a lost removal hint and a leaked object. Fixes: http://tracker.ceph.com/issues/22084 Signed-off-by: Pavan Rallabhandi (cherry picked from commit 70adfaae5073d2680a9722526a6a19795dd18780) commit fe5c8b34a7dbaa1585de54668abeae005dc56388 Author: lvshanchun Date: Wed Nov 1 15:52:48 2017 -0400 radosgw-admin zonegroup get and zone get return defaults when there is no realm Fixs: http://tracker.ceph.com/issues/21615 Signed-off-by: lvshanchun (cherry picked from commit 2c1653ca379d0bf78dfcfe931d895a16b9bfd21f) commit 14434beda69510108a37ae14d8d5318c93d78da4 Author: Yao Zongyou Date: Thu Nov 9 20:40:15 2017 +0800 rgw: check going_down() when lifecycle processing Fixes: http://tracker.ceph.com/issues/22099 Signed-off-by: Yao Zongyou (cherry picked from commit b761989033ac0722a1504238648a806c8004b8e3) commit 90b56a2903db892bcd736d9f9eeca3a8847b353b Author: Aleksei Gutikov Date: Tue Nov 14 15:45:39 2017 +0300 rgw: fix radosgw-admin bucket rm with --purge-objects and --bypass-gc Call RGWRados::delete_bucket() from rgw_remove_bucket_bypass_gc() instead of partial copy of code of RGWRados::delete_bucket(). Fix updating user stats after radosgw-admin bucket rm --purge-objects and --bypass-gc Due to rgw_user(const std::string& s) was called incorrect version of rgw_bucket_sync_user_stats(). Fixes: http://tracker.ceph.com/issues/22122 Fixes: http://tracker.ceph.com/issues/19959 Signed-off-by: Aleksei Gutikov (cherry picked from commit db42e385d26ee4d1ef94b900102b705d6a794029) commit ed24c8ce8f74a9b66d3da3dc2ae3ab5c4a4c86e6 Author: Yanhu Cao Date: Fri Oct 27 13:28:03 2017 +0800 mgr: 'osd status' command return add state Signed-off-by: Yanhu Cao (cherry picked from commit f3475c9fdf5b32443df01c1a735c7289066c7508) commit 85667ad225882cdf4af58166b0d302e69b2d17c0 Author: Yanhu Cao Date: Mon Oct 9 13:47:11 2017 +0800 mgr: fix "osd status" command exception if OSD not in pgmap stats Fixes: http://tracker.ceph.com/issues/21707 Signed-off-by: Yanhu Cao (cherry picked from commit ce3953248a04c065643938898029cfbf9c843ea3) commit 23b3c9de3d3a950eb64ff9a4b77da145f7d1d865 Author: Abhishek Lekshmanan Date: Fri Nov 17 14:48:17 2017 +0100 rgw: keystone: bump up logging when error is received When keystone engine is used and there is an error (often due to misconfigured urls etc) these do not show up at the final logs as we just return -1 at the final stage. Adding logs at level 5 for these cases, as they can help identify a possible misconfigured url or admin token. Eg: ``` 5 Failed keystone auth from http://localhost:5000/v3/v3/auth/tokens with 404 ``` Fixes: http://tracker.ceph.com/issues/22151 Signed-off-by: Abhishek Lekshmanan (cherry picked from commit 347c5148a57de9007d6c1293add6bf27004fce41) commit 58d6d043b20bd2ce185c560814315f82fc33c399 Author: Casey Bodley Date: Fri Nov 10 14:27:59 2017 -0500 rgw: set num_shards on 'radosgw-admin data sync init' Fixes: http://tracker.ceph.com/issues/22083 Signed-off-by: Casey Bodley (cherry picked from commit 90e860c25b87670810426e94b04b515af1e4f154) commit 239a2f8eb0ad5ba3869d0168cfa5bed81479a9d7 Author: Casey Bodley Date: Wed Nov 8 10:21:55 2017 -0500 dencoder/rgw: expose rgw sync status types Signed-off-by: Casey Bodley (cherry picked from commit 662b02489c9394a359834cd3d3527bce5b903100) commit 442120898b8abf22fcd66e06eb8607481653c890 Author: Abhishek Lekshmanan Date: Thu Nov 9 15:50:56 2017 +0100 rgw: data sync: set num_shards when building full maps When radosgw-admin data sync init is called on a cluster, the next run of rgw crashes as when it processes ListBucketIndexesCR, num_shards isn't set which is later referenced in ListBucketIndexesCR. Setting the n sync_info.num_shards correctly to handle this case Fixes: http://tracker.ceph.com/issues/22083 Signed-off-by: Abhishek Lekshmanan (cherry picked from commit 4015a57c473d896164b0617be93777d0947a9576) commit 2070fcf7d4260e3a13219ce708a46b2c550ccc14 Author: Jason Dillaman Date: Wed Nov 15 10:35:16 2017 -0500 librbd: invalidating the cache shouldn't hold write lock This can cause deadlock when readahead is in-progress since neither can make forward progress. Fixes: http://tracker.ceph.com/issues/22131 Signed-off-by: Jason Dillaman (cherry picked from commit 6a335481d20c6a765c84d561a01fb52172eccba4) commit 3e91197ccd0df2bb0560e057861a2f0639da02bd Author: Jason Dillaman Date: Wed Nov 15 10:34:32 2017 -0500 rbd-nbd: rescan partition table after image resize event Fixes: http://tracker.ceph.com/issues/22131 Signed-off-by: Jason Dillaman (cherry picked from commit db13e4757451b463e7fb65c43247e033f24d45e5) commit 818c354a43ca83be10daa1f0cd251635b57371b9 Author: Jason Dillaman Date: Wed Nov 15 09:09:15 2017 -0500 librbd: prevent overflow of discard API result code Prevent discard/writesame lengths larger than 2GB. Fixes: http://tracker.ceph.com/issues/21966 Signed-off-by: Jason Dillaman (cherry picked from commit 3effd324db181e625665be33b5c6529dca723cc5) Conflicts: PendingReleaseNotes src/librbd/librbd.cc: i chosed to pick code from 3effd32 in order to prevent discard/writesame against lengths larger than 2GB. `len` used in picked code was already in luminous. so it's safe to pick. commit 5bd076ef41fd8bdd366654507f2e7c287d1f95dc Author: Sage Weil Date: Mon Nov 20 14:35:48 2017 -0600 qa/suites/upgrade/jewel-x: move mon_warn_on_pool_no_app to global Check runs on mgr. Signed-off-by: Sage Weil commit bd3e7795ca9315aa4d550eeef42884f6f0516029 Author: Patrick Donnelly Date: Fri Oct 27 12:36:06 2017 -0700 MDSMonitor: only clog changes to active Otherwise we get constant INFO messages that an MDS is active. Fixes: http://tracker.ceph.com/issues/21959 Signed-off-by: Patrick Donnelly (cherry picked from commit 997a688d0809221500e0e3dd30e12ccec6c19780) commit b9bf9b1255d3b3f8d581fcc54e7c39912517f6d0 Author: yuliyang Date: Thu Oct 26 14:35:56 2017 +0800 rgw: add cors header rule check in cors option request fix http://tracker.ceph.com/issues/22002 Signed-off-by: yuliyang (cherry picked from commit 72e8fc52fb078fa1443e1ba5321718882969aa78) commit 2d9aafe1bc07cfa65e1bd3d5ea28b575a15eaf53 Author: Li Wang Date: Wed Nov 1 09:21:29 2017 +0000 rbd-nbd: fix unused nbd device search bug in container In some container scenarios, the host may choose to map a specific nbd device, for example, /dev/nbd6 into the container, in that case, the nbd device available in the container is not numbered from 0. The current unused nbd device search function will return no result. This patch fixes it. Fixes: http://tracker.ceph.com/issues/22012 Signed-off-by: Li Wang Reviewed-by: Yunchuan Wen (cherry picked from commit be0f9581f9727187ca03232e0b368e7da7a60609) commit 210b826ff06fd145f730e2b5303e6472d82fd8e5 Author: Li Wang Date: Wed Nov 1 06:31:10 2017 +0000 rbd-nbd: not abort in listing mapped nbd devices In some container scenarios, the file '/sys/block/nbd0/pid' in the container records the host pid of the process which opened the device, therefore the file '/proc/pid/cmdline' corresponding to the pid in the container does not exist, in that case, 'rbd-nbd list-mapped' will cause the following assertion failure, this patch fixes it. src/tools/rbd_nbd/rbd-nbd.cc: In function 'int get_mapped_info(int, Config*)' /src/tools/rbd_nbd/rbd-nbd.cc: 834: FAILED assert(ifs.is_open()) ceph version 13.0.0-1632-gf9cfe84 (f9cfe843a1c749d145b6c3b81a0519cc6536eb28) mimic (dev) 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x102) [0x7fc6b1054ba2] 2: (()+0x15c58) [0x560cd4697c58] 3: (main()+0x9) [0x560cd46937f9] 4: (__libc_start_main()+0xf0) [0x7fc6b003f830] 5: (_start()+0x29) [0x560cd46938f9] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. Aborted (core dumped) Fixes: http://tracker.ceph.com/issues/22011 Signed-off-by: Li Wang (cherry picked from commit 07d5b81b57f17696f4f693225fcb5a1f7060e9d9) commit 274fef45c2ca619aaad3ec14e8c6d80d22832a85 Author: Orit Wasserman Date: Wed Nov 8 12:01:38 2017 +0200 rgw: allow tenant without user id for reshard commands Fixes: http://tracker.ceph.com/issues/22046 Signed-off-by: Orit Wasserman (cherry picked from commit 0a2142e83b58fa8e238bcb748d1cb97bdba674c5) commit 58870ed9111fe3f3e05eadde1fecc8eae7b0eda2 Author: Orit Wasserman Date: Tue Nov 7 15:37:34 2017 +0200 rgw: use tenant when updating the reshard log Fixes: http://tracker.ceph.com/issues/22046 Signed-off-by: Orit Wasserman (cherry picked from commit 055c0d34394b8274a7e48d4f2e3913c059099898) commit 96b924fc750008e66e1cea29a67f1e8c7008a3ed Author: Ilja Slepnev Date: Tue Oct 24 20:27:59 2017 +0300 mgr/zabbix: ignore osd with 0 kb capacity Fixes: http://tracker.ceph.com/issues/21904 Signed-off-by: Ilja Slepnev (cherry picked from commit e532949556453668158d0e261e59e0fae48e461f) commit 654678ab2de4b4909e1ad12384e790105709ec1a Author: Jianpeng Ma Date: Fri Nov 10 22:47:41 2017 +0800 common/buffers: add function parameter to limit buffers size. Fixes: http://tracker.ceph.com/issues/21932 Signed-off-by: Jianpeng Ma (cherry picked from commit fba63cebba215b36ce850f47bd8e706edaa8023e) commit 488228e919980d1aba3313bf5f1601e30bfdc03d Author: Jianpeng Ma Date: Thu Nov 9 01:00:52 2017 +0800 osdc/Objecter: record correctly value for l_osdc_op_send_bytes. Fixes: http://tracker.ceph.com/issues/21982 Signed-off-by: Jianpeng Ma (cherry picked from commit c5c7ad56c0d5c758874de0f5ea66504963bbb75e) commit 134818228b5c0fa54e934c842038de0b9ad63620 Author: Sage Weil Date: Mon Nov 20 08:55:59 2017 -0600 mon/OSDMonitor: fix mon_fixup_legacy_erasure_code_profiles Signed-off-by: Sage Weil commit 6f983c14d94f51b458730a8ef1d1723d8aa8c1e2 Author: Jan Fajerski Date: Wed Nov 8 10:30:09 2017 +0100 tools/crushtool: skip device id if no name exists When an OSD with an id < max_id is removed, i.e. the osd ids are not continuous, crushtool decompile prints bogus info. Skip any device ids without a name. Signed-off-by: Jan Fajerski (cherry picked from commit 85737f94571208f21c972c11530ab56bfaededa2) commit d5f2e1660055d296640eebcfcaf13435dff212ef Author: Kefu Chai Date: Tue Nov 14 18:42:02 2017 +0800 osd: subscribe osdmaps if any pending pgs subscribe from monitor continously for new osdmaps so osd is able to get the osdmap instructing it to delete PGs even if nobody is poking it. Fixes: http://tracker.ceph.com/issues/22113 Signed-off-by: Kefu Chai (cherry picked from commit 4037ab44b30ab23b2e464b24a82fee6fff9a2d08) commit c6bc756304ff567f39e163e2140d6bb00e8adc6a Author: Abhishek Lekshmanan Date: Tue Nov 14 17:39:16 2017 +0100 rgw: set sync_from_all as true when no value is seen In order to support jewel-luminous multisite scenarios where when a jewel cluster is the master, the sync-from-all json field wouldn't be set leading to the secondary not syncing data as this value can't be overriden to true, so defaulting the value of sync_from_all to true when we don't encounter it in the json Fixes: http://tracker.ceph.com/issues/22062 Signed-off-by: Abhishek Lekshmanan (cherry picked from commit e0116fa42020ff5c0d78101b9b3b15e62236b4b8) commit c21e1c7831e43ed5d07924589b24ce826bb86467 Author: John Spray Date: Wed Nov 8 11:54:44 2017 -0500 mon: clean up cluster logging on mon events These changes come from observing the output when killing a mon and watching the survivors form a new quorum. Fixes: http://tracker.ceph.com/issues/22082 Signed-off-by: John Spray (cherry picked from commit 294fe62feeb155e1ec06d508d92435de98cce90a) commit 6875bdaf5fcc000c4a4615e3f6a03a530bdd74da Author: John Spray Date: Wed Nov 8 11:53:45 2017 -0500 mon: monmap log on active should be debug This isn't even a change to the map, just something that got echoed every time a new mon was the leader. Signed-off-by: John Spray (cherry picked from commit c9eb2793e6074521d161632e20954295527d3653) commit 1bdd81e095a4bd628c278a3e6abd7faf62ba5399 Author: Sage Weil Date: Sun Nov 12 19:49:02 2017 -0600 mgr/balancer: skip CRUSH_ITEM_NONE Signed-off-by: Sage Weil (cherry picked from commit e2cbb4bc7a2badc3ff89fb4bf4d678d8e9f94652) commit 3efc4c8d542060cd43fa509022063323d1eecaac Author: Sage Weil Date: Wed Nov 8 17:37:29 2017 -0600 qa/suites/rados: stop testing firefly tunables We can't mix the balancer compat-set testing with firefly tunables because it requires that all buckets be straw2. Signed-off-by: Sage Weil (cherry picked from commit 6455954d29e180e24dcadfabce685699477e2736) commit 5f8a59a48e2399bf41c7a0b6ce9662f6b150ba44 Author: Sage Weil Date: Wed Nov 8 17:35:35 2017 -0600 mgr/balancer: fail a bit more gracefully if we can't create a compat weight-set Signed-off-by: Sage Weil (cherry picked from commit 476b2335fd08568d142d17848bfc58dc6e5c86d2) commit 6d02b2a1a276c98dd3f9881d3e113f1db14fee04 Author: John Spray Date: Thu Nov 9 07:33:10 2017 -0500 mgr/dashboard: fix audit log loading Signed-off-by: John Spray (cherry picked from commit 85e1dadb287ee5216985b16b4a144dfe148d6a60) commit de1d25c2dd31ac731225ee2a3c3d2805513881b4 Author: Mitch Birti Date: Thu Nov 16 16:27:20 2017 -0600 udev: Fix typo in udev OSD rules file The rule for lockbox partitions had an invalid UUID for the "change" action. Signed-off-by: Mitch Birti (cherry picked from commit 515417294c2548694c2101639c8bbf7f88e716aa) commit a170e67760ee3e36d6f5c0393710b0d58fd94005 Author: Yan, Zheng Date: Tue Aug 22 09:59:12 2017 +0800 mds: fix CDir::log_mark_dirty() the 'will dirty' check is wrong because we don't always project fnode. Fixes: http://tracker.ceph.com/issues/21584 Signed-off-by: "Yan, Zheng" (cherry picked from commit 461bbc7e89a9ed440478f30145158b4989c137d0) commit 0d18c24fcedd82fa0203ce37bac9099bdd455f35 Author: Pavan Rallabhandi Date: Wed Nov 8 21:35:54 2017 +0530 rgw: Fix swift object expiry not deleting objects In cls_timeindex_list() though `to_index` has expired for a timespan, the marker is set for a subsequent index during the time boundary check. This marker is further returned to RGWObjectExpirer::process_single_shard(), where this out_marker is trimmed from the respective shard, resulting in a lost removal hint and a leaked object. Fixes: http://tracker.ceph.com/issues/22084 Signed-off-by: Pavan Rallabhandi (cherry picked from commit 70adfaae5073d2680a9722526a6a19795dd18780) commit 74a6592d7fc8a4e1d4000025615b72e7297285a1 Author: Yehuda Sadeh Date: Tue Sep 26 13:04:26 2017 -0700 options.cc: document rgw config options Signed-off-by: Yehuda Sadeh (cherry picked from commit 79e58a346e743b95b4435fdc72f2fd7ac01e3f1e) commit 7acb9a8da2c843ac4fb577c3e5eb6c699604da9e Author: Kefu Chai Date: Fri Sep 29 10:51:54 2017 +0800 common/options: use user-defined literals for sizes Signed-off-by: Kefu Chai (cherry picked from commit 6788ea0e9a51217cd56b282c40533a8971a36530) commit ecf102618ec34afee96093812e68382e6b9c4872 Author: Kefu Chai Date: Wed Aug 23 14:35:19 2017 +0800 common/options: use user-defined literals for default intervals for better readablility Signed-off-by: Kefu Chai (cherry picked from commit 3704fe283b606737533d62a082337381b95bdbd9) commit bc94035a30360a8c2353afb5f0355e0675abbdeb Author: Kefu Chai Date: Thu Aug 24 17:08:00 2017 +0800 common/options: use user-defined literals for default sizes for better readablity. Signed-off-by: Kefu Chai (cherry picked from commit d9b38a1d6172a43e7ebb755be5cf044f19d7035d) commit 7a8e915483397d740e3be3815ece23f4cd3c6b7a Author: John Spray Date: Wed Oct 11 11:36:36 2017 +0100 mds: additional damage handling case in EImportStart Fixes: http://tracker.ceph.com/issues/21759 Signed-off-by: John Spray (cherry picked from commit b4c662c2cf47f7b6f952e776d0212156861eaffb) commit 1ec93753a37a24a7e72566848ccc0baab1df84ce Author: Yan, Zheng Date: Wed Aug 2 17:26:56 2017 +0800 mds: track snap inodes through sorted map Current mds track both head inodes and snap inodes through unsorted map. The unsorted map makes finding snap inode that follows a given snapid difficult. Currnt MDCache::pick_inode_snap() use snap set to guess snap inode's last. The method isn't reliable because snap set may change after creating the snap inode. For example: MDS cows inode[2,head] with snap set[5,6], which results inode[2,6] and inode[7,head]. Later mds wants to find snap inode that follows snapid 2. But the snap set become [5], mds can't find snap inode [2,5]. Signed-off-by: "Yan, Zheng" (cherry picked from commit 7b9eae62c8c654ff82684451c222257d2c93be64) Conflicts: src/mds/MDCache.cc: when i cherry-picked 3ca602e, there were conflicts in same file. to fix conflicts, i need to introduce snap_inode_map because it's not in luminous. to intoroduce this, i need to cherry-pick 7b9eae62. After cherry-picking 7b9eae62, there were conflicts. i picked code from head because it's introduced by 3ca602e which is required to fix http://tracker.ceph.com/issues/21928. commit 85791328fc563932673e2d6ad8701179fee0c3a3 Author: Yan, Zheng Date: Thu Oct 26 16:21:41 2017 +0800 mds: fix inode count check in MDCache::check_memory_usage() Fixes: http://tracker.ceph.com/issues/21928 Signed-off-by: "Yan, Zheng" (cherry picked from commit 3ca602e3bdaa211ae2446d4bbf62431e14d41bef) Conflicts: src/mds/MDCache.cc commit e485b89ffd5754b5c6356ccd018fe602d7e55e07 Author: Yan, Zheng Date: Fri Oct 20 17:03:10 2017 +0800 mds: disable early reply for dir layout and quota related requests These requests impacts whole subtree tree, replaying them when mds recovers may break order of requests in multimds cluster. Signed-off-by: "Yan, Zheng" (cherry picked from commit edd76fc88377b398b5f2d274684d10f6ed3314cd) commit cb8eff43b1abd8c268df9e57906d677ff4be8d95 Author: Yan, Zheng Date: Wed Oct 18 20:58:15 2017 +0800 mds: don't rdlock locks in replica object while auth mds is recovering Auth mds may take xlock on the lock and change the object when replaying unsafe requests. To guarantee new requests and replayed unsafe requests (on auth mds) get processed in proper order, we shouldn't rdlock locks in replica object while auth mds of the object is recovering Signed-off-by: "Yan, Zheng" (cherry picked from commit 0afbc0338e1b9f32340eaa74899d8d43ac8608fe) commit f1be92ee34399617cad31a489ec49e993c6eaff6 Author: Yan, Zheng Date: Wed Oct 18 20:35:33 2017 +0800 mds: move ScatterLock::state_flags into SimpleLock Signed-off-by: "Yan, Zheng" (cherry picked from commit da6e50890de8b682eb5756d0e4e08583908b3778) commit ebd7a6670fbb9724bf72f7b0587e5c18ade03a48 Author: Yan, Zheng Date: Fri Aug 11 17:52:19 2017 +0800 mds: make mksnap/setlayout wait for unsafe requests on other mds This guarantees replayed unsafe requests (on other mds) and mksnap/setlayout get processed in proper order. Signed-off-by: "Yan, Zheng" (cherry picked from commit dee3711bd745704fba224dc6f90e88c04cacff58) commit a88e48e7d9cba77db8cb5aeb9512733d62a08382 Author: lu.shasha Date: Tue Oct 10 15:51:04 2017 +0800 rgw: don't change rados object's mtime when update olh null instance and olh share rados object. Null instance using the rados object mtime as its mtime. When olh updated, the rados object mtime will change. Thus the null instance will be incorrect. When list objects in bucket, the mtime is correct, down the null instance the last modified time is incorrect. So when update olh, using the previous mtime, thus the null instance mtime will be correct. Fixes: http://tracker.ceph.com/issues/21743 Signed-off-by: Shasha Lu (cherry picked from commit 3189edc28ea6d3c117c2e2f0664f45c69f67ab50) commit 1410b5a5f5505ba7df0fdf313eefee4306807f3a Author: Zhi Zhang Date: Mon Oct 9 14:33:05 2017 +0800 mds: no assertion on inode being purging in find_ino_peers() Signed-off-by: Zhi Zhang (cherry picked from commit e55b2dbfcacb3b5afddba2d2b24c0386a0a48ebb) commit 2dc009e68ef9e850aebcc55634a1505212786309 Author: Adam C. Emerson Date: Fri Oct 27 15:57:18 2017 -0400 rgw: Fix dereference of empty optional Due to the lack of a return, there was a case where an invalid ARN could cause a dereference of an uninitialized boost::optional. As a bit of defensive programming, restructure a couple functions to make that kind of error impossible by ensuring the optional is only in scope when it is initialized and relying less in early return on error. Fixes: http://tracker.ceph.com/issues/21962 Signed-off-by: Adam C. Emerson (cherry picked from commit 5249139be7a2748eabbf898cf340989875bfa509) commit dbd70f0b82a8eb512614b68d8de0c861da9e9d47 Author: Casey Bodley Date: Mon Oct 23 14:23:32 2017 -0400 rgw: fix extra_data_len handling in PutObj filters the RGWPutObj_Compress filter relies on a starting offset of 0 to ensure that we only compress entire objects for RGWRados::fetch_remote_obj(), we also fetch object metadata. the replies come back with a 'Rgwx-Embedded-Metadata-Len' header, which specifies how many bytes of object metadata are at the front of the request body. when this is present, the offsets passed from RGWRadosPutObj down to the RGWPutObjDataProcessor filters are offsets into the http response body, rather than logical offsets into the object data itself this commit adds a transformation to RGWRadosPutObj so that only that logical offset is visible to the RGWPutObjDataProcessor Fixes: http://tracker.ceph.com/issues/21895 Signed-off-by: Casey Bodley (cherry picked from commit ac7ffe6ef7faef4e251c970ce2efd04616c0459c) commit 9f9db135a058c3134a96a845900bb795d8e708fc Author: Orit Wasserman Date: Sun Oct 1 08:40:27 2017 +0300 doc: replace region with zonegroup in configure bucket sharding section Fixes: http://tracker.ceph.com/issues/21610 Signed-off-by: Orit Wasserman (cherry picked from commit 8a1034bccdd514a3eaa6abfdc62c5e3d1e46d5cf) commit eeb12253d4caec2194970a84a9440d7ccf372d3d Author: Piotr Dałek Date: Tue Oct 17 09:48:15 2017 +0200 tools/ceph-conf: dump parsed config in plain text or as json This is useful for finding differences between ceph.conf on disk and in osd/mon memory. Signed-off-by: Piotr Dałek (cherry picked from commit 951434f3a40fd697e5eae8fbcbf779abceb3ba34) commit 8cb21e6ea6fa0abe50d0c5d2b2070963c3ee3de2 Author: Jason Dillaman Date: Fri Oct 27 16:45:54 2017 -0400 cls/journal: ensure tags are properly expired Previously, if only the local image was using the journal or if a disconnected peer was attached, the tag entries could not be expired even if unreferenced. Fixes: http://tracker.ceph.com/issues/21960 Signed-off-by: Jason Dillaman (cherry picked from commit 19fa1c7f5b2809e9a223b7b196dfc031e97a5dcd) commit c86675dadc8d97c60cb94f32713c9819941607b2 Author: Jason Dillaman Date: Thu Oct 26 10:57:20 2017 -0400 test/librbd: added update_features RPC message to test_notify Fixes: http://tracker.ceph.com/issues/21936 Signed-off-by: Jason Dillaman (cherry picked from commit b003ff320036437b42bd8f31c0096162a3c47fca) commit efd4147fa895ea86b65c807aee7991c016c4d7aa Author: Douglas Fuller Date: Thu Nov 2 16:30:17 2017 -0400 cephfs: Do not check auth gid when not specified For auth caps that omit the gid, do not check for a gid match. Fixes: http://tracker.ceph.com/issues/22009 Signed-off-by: Douglas Fuller (cherry picked from commit 0e2cfdf507ab3ec8459bc6ea9b73b7a1285274d0) commit 369dcb36c70ec6c9ab974eb6b7542068ed9fcbda Author: Jason Dillaman Date: Fri Nov 3 12:03:49 2017 -0400 librbd: added preprocessor macro for detecting compare-and-write support Fixes: http://tracker.ceph.com/issues/22036 Signed-off-by: Jason Dillaman (cherry picked from commit 95d716cc7098cb04063649584e0774f49a580e1d) commit 4db6b35548fdbf5cd2a84413933170547abf9a01 Author: John Spray Date: Wed Nov 1 12:08:03 2017 -0400 mgr: emit cluster log message on serve() exception Fixes: http://tracker.ceph.com/issues/21999 Signed-off-by: John Spray (cherry picked from commit f4763c32fa84f57c93c559d742203fb3fd2985a8) commit d73deb017ce9ad2525a2bda54553c01f386f0b35 Author: Patrick Donnelly Date: Tue Oct 31 11:55:29 2017 -0700 ceph.in: pass RADOS inst to LibCephFS This avoids multiple instances of the admin socket and other redundancies. Fixes: http://tracker.ceph.com/issues/21967 Fixes: http://tracker.ceph.com/issues/21406 Signed-off-by: Patrick Donnelly (cherry picked from commit 76a950b2ec473741ed3cffca5c198e8c56f32a1c) commit ab103b9fe36ad0dd7e470c4c2cf701963cc8bf3c Author: Yan, Zheng Date: Mon Oct 30 20:03:29 2017 +0800 mds: trim 'N' log segments according to how many log segments are there Config 'mds_log_max_expiring' is 20 by default. It means that at most 20 log segments get trimmed in each tick. For busy cluster, this can cause mds behind on trimming log segments. Fixes: fixes: http://tracker.ceph.com/issues/21975 Signed-off-by: "Yan, Zheng" (cherry picked from commit 7de37382ace36b6b724b07dcd58178137c49e6b8) commit ddba907279719631903e3a20543056d81d176a1b Author: Yan, Zheng Date: Tue Oct 31 16:56:51 2017 +0800 mds: fix MDS_FEATURE_INCOMPAT_FILE_LAYOUT_V2 definition Fixes: http://tracker.ceph.com/issues/21985 Signed-off-by: "Yan, Zheng" (cherry picked from commit 6c1543dfc55d6db8493535b9b62a30236cf8c638) commit e0186c9e259898286b720e0e671e708f580cbf25 Author: Yan, Zheng Date: Tue Oct 31 16:37:07 2017 +0800 mds: remove useless incompat feature definination in FSMap.h Signed-off-by: "Yan, Zheng" (cherry picked from commit 990088ef16f901588169cd20ed99d0415ec1f7f4) commit cb38378e5dc4821c232b15a99b30aeeb7b9f9744 Author: Enming Zhang Date: Wed Oct 25 21:10:56 2017 +0800 rgw: loadgen fix rgw crash issue When generating random bucket name and object name during loadgen processing, RGW will crash. The reason is calling "gen_rand_alphanumeric" with "cct = NULL". Fixes: http://tracker.ceph.com/issues/22006 Signed-off-by: Enming Zhang (cherry picked from commit 7d021782e29c74eca36a9bb6ca59930f87093115) commit ed87de0622cdb1ac93915967949d581ed62e31d9 Author: yuliyang Date: Tue Oct 31 14:20:14 2017 +0800 rgw: modify_s3_type_subuser_access_permissions_fail_through_admin_rest_api fix: http://tracker.ceph.com/issues/21983 Signed-off-by: yuliyang (cherry picked from commit bd1385d4766ee496bf87587dd40086ee3d510672) commit 03d1cd996cf189d10a23c839aead4075cbdb888f Author: Enming Zhang Date: Wed Oct 25 18:56:08 2017 +0800 rgw: lc fix rgw crash when lc configuration xml not including ID When a putting lc config request does not include an ID tag in lc configuration xml, RGW should generate a random ID for the lc configuration. At present RGW will crash when generate a random ID for lc configuration. Fixes: http://tracker.ceph.com/issues/21980 Signed-off-by: Enming Zhang (cherry picked from commit 3d2f63de494f8cd065d3af3cfa49e111a294622c) commit 5e44bedf293beffb6823ae866373314a3675c431 Author: Enming Zhang Date: Wed Oct 25 02:36:47 2017 +0800 rgw: lc support Content-MD5 in request headers According to AWS S3, this header must be used as a message integrity check to verify that the request body was not corrupted in transit. Content-MD5 is the base64-encoded 128-bit MD5 digest of the data Signed-off-by: Enming Zhang (cherry picked from commit 9c0abe6d638c92f1f29798afa846f8a80fa64814) commit 860625046d4a0b79e01b0bbcc5027fbbbfbad07a Author: Orit Wasserman Date: Wed Nov 1 17:49:46 2017 +0200 rgw: add missing current_history initialization Fixes: http://tracker.ceph.com/issues/21996 Signed-off-by: Orit Wasserman Signed-off-by: Casey Bodley (cherry picked from commit cc3246cf46ba26604e4b3bef1f8fe0c9990bdeb6) commit 32ca69b0d9fec21f1ecb94707850eb9d3322c435 Author: Orit Wasserman Date: Wed Nov 1 12:15:35 2017 +0200 rgw: init oldest period after setting run_sync_thread Fixes: http://tracker.ceph.com/issues/21996 Signed-off-by: Orit Wasserman (cherry picked from commit 8b301679161d1405fdae379ff93c33d4d637698d) commit f0914619a5cc0d9108c4c6efd0f3464ec4fd0fff Author: Kefu Chai Date: Tue Oct 31 18:42:51 2017 +0800 mon/PGMap: use new-style options Signed-off-by: Kefu Chai (cherry picked from commit 7a23097c34c35c1bf6ec09e86ed3acbb0807068c) Conflicts: src/common/legacy_config_opts.h src/mon/PGMap.cc: in 729a089, PGMap::get_health() is removed from master branch, but in luminous, this function is still around for backward compatibility. but that function is still using the old-style options. in this change, i choose to keep some of the old-style options removed in the cherry-picked cleanup to appease PGMap::get_health(), instead of migrating all of them to the new-style because of less efforts. commit fd956acfd674b32a7518cb4699a500fc4c845172 Author: John Spray Date: Wed Sep 27 17:11:24 2017 -0400 tools: update monstore tool for fsmap, mgrmap Fixes: http://tracker.ceph.com/issues/21577 Signed-off-by: John Spray (cherry picked from commit 0ad7bd0b43b473211de30ed9f2050059e069e4d3) commit 0ebc4cc18f974975a27c77212ea0c38bcbdcb67c Author: John Spray Date: Wed Sep 27 14:59:26 2017 -0400 tools: handle decode errors in monstore tool Print a single line message instead of dumping a backtrace. Signed-off-by: John Spray (cherry picked from commit 61ceafc8bba3747b492f8b0f779793813a682ecb) commit b36bd48204f80b47e3d90189fce366caad474650 Author: Neha Ojha Date: Thu Sep 28 09:50:24 2017 -0700 osd: hold lock while accessing recovery_needs_sleep Signed-off-by: Neha Ojha (cherry picked from commit 4bf8d13273b66cc89a94e55259b6fcb106061bd6) commit 8941606f5e559850dd968ea2966c2364c09b976a Author: Greg Farnum Date: Mon Oct 9 14:16:05 2017 -0700 common: by default, do not assert on leaks in the shared_cache code Update the standard qa suite cluster configs so that we continue asserting in our nightlies, but users don't hit this. Fixes: http://tracker.ceph.com/issues/21737 Signed-off-by: Greg Farnum (cherry picked from commit 165b61a7df2731602e9e28c5107a873444bf0507) commit acccae56c4b67bbb0a788e8ca7b248e9b8826285 Author: Sage Weil Date: Fri Sep 15 16:03:38 2017 -0400 osd: fix waiting_for_peered vs flushing on_flush() requeues waiting_for_peered, but we flush twice on the primary during peering, and we don't want to requeue the first one (when we have the master pg log merged). Fix by moving waiting_for_peered to waiting_for_flush if we aren't already flush on _activate_committed. If we get an op and are peered but not flushed, queue ops there. (We can simplify this check a bit since pgbackend inactive message handling doesn't care about flushed or not flushed.) When flushed, we requeue waiting_for_flush. The waiting_for_flush, waiting_for_peered, and waiting_for_active lists are all mutually exclusive, so this mostly serves to clarify what we are waiting for (not to keep items separate). And it means that on_flushed() will only requeue things that were waiting for it specifically. Fixes: http://tracker.ceph.com/issues/21407 Signed-off-by: Sage Weil (cherry picked from commit 8f7dc8b0d8135b20322c09a5909ad43d4c83b0ea) commit 5234ef2bcaa905861e7c15a9774b82467b2e84e7 Author: Haomai Wang Date: Mon Oct 23 12:38:56 2017 +0800 msg/async/AsyncConnection: state will be NONE if replacing by another one Fixes: http://tracker.ceph.com/issues/21883 Signed-off-by: Haomai Wang (cherry picked from commit f2eb981377f4061debfa67cbd88ca2921a9dfb27) commit c281456346f0c65e3fa4bb8ca0baf42f4d16448e Author: Jason Dillaman Date: Fri Oct 20 09:32:14 2017 -0400 osdc/Objecter: delay initialization of hobject_t in _send_op Fixes: http://tracker.ceph.com/issues/21845 Signed-off-by: Jason Dillaman (cherry picked from commit eca4fafbcfa922d9e2e98564e69596f6c0b1b93c) commit dfebcce5a445793267ec8979bcb56ec04954d154 Author: Jason Dillaman Date: Thu Oct 19 22:24:31 2017 -0400 osdc/Objecter: skip sparse-read result decode if bufferlist is empty If the OSD does not execute sub-ops due to errors encountered prior to the sub-op, the sub-op result remains zeroed with empty out data. Attempting to decode the empty bufferlist results in large exception handling CPU overhead. Fixes: http://tracker.ceph.com/issues/21844 Signed-off-by: Jason Dillaman (cherry picked from commit dc9b309d03074862daad9ef05ef643da870f6722) commit 7012cf4b60c34dc7f41db9acbdeeeab20e8cd80a Author: Sage Weil Date: Tue Oct 24 15:20:49 2017 -0500 ceph_test_objectstore: do not change model for 0-length zero We almost fixed this in the pr merged at 2dbbb351e2e0e3703880023bf51f55790f763e04 but missed this piece. Signed-off-by: Sage Weil (cherry picked from commit 390779d1aa9b46648e4f5390dc431c255c70385d) commit 51b380a0a74f0335a04d1121042449909549c976 Author: Sage Weil Date: Fri Oct 6 15:29:32 2017 -0500 os/bluestore: 0-length zero should not change object size Fixes: http://tracker.ceph.com/issues/21712 Signed-off-by: Sage Weil (cherry picked from commit b588eaf2b0fdf06c94104d5a542bd571499f2b85) commit 45d70b1329a8c635d2ac0ea5ec171cbea0f6ce02 Author: Sage Weil Date: Fri Oct 6 15:29:15 2017 -0500 os/filestore: make 0-length zero avoid touching object length Fixes: http://tracker.ceph.com/issues/21712 Signed-off-by: Sage Weil (cherry picked from commit 979b7c9cd2e4919b8c7d5ed2418019310959f4bf) commit 0d921686080d0d1ce81c55a0e8926951fd9ffa58 Author: Sage Weil Date: Fri Oct 6 15:28:49 2017 -0500 os/ObjectStore: 0-length zero does not change length of object Signed-off-by: Sage Weil (cherry picked from commit 9ad1f4f10ff7bfe32d0a37361640fe5c65e56699) commit bacebf0fd7f841185236c1dc6754955da1dfb9b2 Author: Aleksei Gutikov Date: Mon Oct 16 17:31:12 2017 +0300 mgr: skip first non-zero incremental in PGMap::apply_incremental() After initialization of PGMap instance PGMap::stamp is zero and this cause huge first delta. Also after mgr restart first non-zero value coming to PGMap::apply_incremental() is current pg_sum value so it produces unreasonably huge pg_sum_delta. This patch introduces a workaround to save pg_sum and not update pg_sum_delta by first non-zero incremental. Signed-off-by: Aleksei Gutikov Fixes: http://tracker.ceph.com/issues/21773 (cherry picked from commit 013a3803c931ac2e5c390dcd209e1dd0a58418b9) Conflicts: src/mon/PGMap.cc commit b6f50a463a2757b5822ad5143c4ba8eb86e8d27e Author: Yanhu Cao Date: Wed Aug 9 10:22:19 2017 +0800 mgr/Mgr: implement completion of osd MetadataUpdate Fixes: http://tracker.ceph.com/issues/21159 Signed-off-by: Yanhu Cao (cherry picked from commit cb26eead3f195dd51ae79ec9e41700cdde674648) commit 948ca443fcdf876436a66cc0188cb7e99e1cb068 Author: John Spray Date: Fri Sep 29 07:02:21 2017 -0400 mgr/dashboard: redirect away if viewed filesystem is removed Previously this would sit there with stale data in the browser if you were viewing a filesystem and then deleted it in the background. Signed-off-by: John Spray (cherry picked from commit 83800cd7c11e229e291c699ed45bc65c7fcd9357) commit 2ed0f0918f4d2d02c57e0ed65c6d790aa185ad8d Author: John Spray Date: Fri Sep 29 07:01:48 2017 -0400 mgr/dashboard: 404 instead of 500 on missing filesystem Still not the most beautiful but lets reserve 500s for real errors. Signed-off-by: John Spray (cherry picked from commit 7e62cb703c64dbe177cc69d904ed4888b74cdb86) commit ed2dcad99b9d2ebb7d8d3b836b4761b0c10555f5 Author: John Spray Date: Fri Sep 29 06:43:24 2017 -0400 mds: fix FSMap copy constructor This was confusing ceph-mgr, which did a fsmap = new_fsmap and found that old filesystems were never disappearing after being removed. Fixes: http://tracker.ceph.com/issues/21599 Signed-off-by: John Spray (cherry picked from commit 516896d3c93ace4e610960f0f2b7a9824df3f7f2) commit 9a7e1f6a448e57f5ae24844d8a7edb0fd4963933 Author: John Spray Date: Wed Oct 25 09:41:56 2017 -0400 mds: set PRIO_USEFUL on num_sessions counter This is used by dashboard and status modules, so let's make sure we're sending it to the mgr. Fixes: http://tracker.ceph.com/issues/21927 Signed-off-by: John Spray (cherry picked from commit 1d0a7d1f35e1c1be472b0555323a9d2ac2d2d777) commit 5c9d212f94160715a44dca79f18be1c8b2f7ce55 Author: Casey Bodley Date: Tue Oct 31 12:26:07 2017 -0400 cmake/cls: add install() for ceph_test_cls_log Signed-off-by: Casey Bodley (cherry picked from commit 9e94244db2be48bc6ed198a093d7bbb8ae8a72b1) commit b228d35e1df2302215d751ed35988a34f1238bef Author: Casey Bodley Date: Mon Oct 30 15:31:03 2017 -0400 qa: add ceph_test_cls_log to cls workunit Signed-off-by: Casey Bodley (cherry picked from commit 83d6b493f71db9417c7b2b7db5f1da9c2d71eec3) commit 9f09b1bb1f032d626992fa9b63ac613ee0969f3d Author: Casey Bodley Date: Sat Oct 28 16:28:27 2017 -0400 osd: add processed_subop_count for cls_cxx_subop_version() cls_log_add() relies on cls_cxx_subop_version() to generate unique keys for log entries with the same timestamp. because cls calls back into do_osd_ops(), resetting current_osd_subop_num means that cls_log_add() will keep seeing the same subop version and generating the same keys. this causes the following failure in ceph_test_cls_log: [ RUN ] cls_rgw.test_log_add_same_time /home/cbodley/ceph/src/test/cls_log/test_cls_log.cc:144: Failure Expected: 10 To be equal to: (int)entries.size() Which is: 1 [ FAILED ] cls_rgw.test_log_add_same_time (1180 ms) Fixes: http://tracker.ceph.com/issues/21964 Signed-off-by: Casey Bodley (cherry picked from commit 95faac9ae2cd38b1ca25c90f90bd4a53fe6af4d4) commit 1d4435f0cda503b9742f1cad72930bf09bfce265 Author: Peter Keresztes Schmidt Date: Sun Oct 15 06:36:54 2017 +0200 rbd: fix crash during map Currently the iterator isn't advanced after the erase call leading to a second call on the iterator, which crashes due to a double free. Since C++11 the map::erase function returns an iterator pointing to the next element. Use the return value to set the iterator after erasing. Fixes: http://tracker.ceph.com/issues/21808 Signed-off-by: Peter Keresztes Schmidt (cherry picked from commit 9e49c4124422e58dd40dfb6038425430d3845412) commit 767ad8518d670e396b1a6a9d9574c5b8479673d1 Author: Jason Dillaman Date: Wed Oct 4 10:46:46 2017 -0400 librbd: list_children should not attempt to refresh image The snap_lock is being held when this method is invoked, which can result in a deadlock. Fixes: http://tracker.ceph.com/issues/21670 Signed-off-by: Jason Dillaman (cherry picked from commit 4c585d826f38ff97d3a484a30eca0588c79396b4) commit 22ff1477348f2fbfe94ef6a08b09687916734aa6 Author: Yan, Zheng Date: Thu Aug 24 17:06:30 2017 +0800 mds: fix StrayManager::truncate() old code does not set PurgeItem::action Fixes: http://tracker.ceph.com/issues/21091 Signed-off-by: "Yan, Zheng" (cherry picked from commit 45fd51254523f5707e5a67dd7c6ba6011f80e179)