commit e0354f9d3b1eea1d75a7dd487ba8098311be38a7 Author: Jenkins Build Slave User Date: Tue Aug 8 19:07:05 2017 +0000 11.2.1 commit 11d5c2b0aae372f9c993a7dea5a914ee74e7480c Merge: 397b553932 6b428b77f2 Author: Nathan Cutler Date: Wed Aug 2 15:08:48 2017 +0200 Merge pull request #14702 from ceph/wip-bp-kraken-systemd kraken: qa/tasks: misc systemd updates Reviewed-by: Nathan Cutler Date: Wed Aug 2 07:55:48 2017 +0200 Merge pull request #15509 from ceph/wip-ceph-disk-fix-kraken kraken: selinux: Do parallel relabel on package install Reviewed-by: Kefu Chai commit 0bab3f3a1026c41736e06f72f8ede68b3386d772 Merge: 646f2b3c26 a91253ec50 Author: Nathan Cutler Date: Tue Aug 1 20:19:15 2017 +0200 Merge pull request #14612 from smithfarm/wip-19651 tests: backport Sage's fixes to qa/suites/upgrade/jewel-x Reviewed-by: Kefu Chai commit 646f2b3c260ea5af3657d5f289753b114ba0d71e Merge: e9c255689f cf06edd36b Author: Nathan Cutler Date: Tue Aug 1 14:39:37 2017 +0200 Merge pull request #16298 from smithfarm/wip-20517-kraken kraken: rbd: cli: map with cephx disabled results in error message Reviewed-by: Jason Dillaman commit e9c255689fcf2a7afe1deda3b987591162767daa Merge: 143e431d76 1fc14857a9 Author: Nathan Cutler Date: Tue Aug 1 14:17:40 2017 +0200 Merge pull request #16175 from smithfarm/wip-20263-kraken kraken: rgw: datalog trim can't work as expected Reviewed-by: Orit Wasserman commit 143e431d769ad68101df436735154c1230f3732b Merge: c9a545dba6 b758348447 Author: Nathan Cutler Date: Tue Aug 1 14:16:54 2017 +0200 Merge pull request #15985 from prallabh/kraken kraken: rgw: Custom data header support Reviewed-by: Orit Wasserman commit c9a545dba68fe09e7aa971aec00c7e5aa1f217d2 Merge: 245bf9205d 45b4c86452 Author: Nathan Cutler Date: Tue Aug 1 14:15:33 2017 +0200 Merge pull request #16186 from smithfarm/wip-20264-kraken kraken: rbd: [cli] ensure positional arguments exist before casting Reviewed-by: Jason Dillaman commit 245bf9205dc811b4bbe99e53bf20b32daeb85f8d Merge: 3d19b8acf0 f2d61c199d Author: Nathan Cutler Date: Tue Aug 1 14:12:58 2017 +0200 Merge pull request #16342 from dillaman/wip-20630-kraken kraken: tests: qa/tasks: rbd-mirror daemon not properly run in foreground mode Reviewed-by: Nathan Cutler commit 3d19b8acf0f76794c2b9c48f6ab386b6e5d14139 Merge: a57772db99 f72ea68186 Author: Nathan Cutler Date: Tue Aug 1 13:14:09 2017 +0200 Merge pull request #16290 from smithfarm/wip-19759-kraken kraken: rgw: multisite: after CreateBucket is forwarded to master, local bucket may use different value for bucket index shards Reviewed-by: Casey Bodley commit a57772db99068c5e05b7b8cd1e346bdfb1fba292 Merge: a1ff4e61af af0a6df270 Author: Nathan Cutler Date: Tue Aug 1 13:12:37 2017 +0200 Merge pull request #16180 from smithfarm/wip-20347-kraken kraken: rgw: meta sync thread crash at RGWMetaSyncShardCR Reviewed-by: Casey Bodley commit a1ff4e61aff4497e84d07070de984e34c6539bf0 Merge: d0d98362e7 17fd055a4f Author: Nathan Cutler Date: Tue Aug 1 12:47:45 2017 +0200 Merge pull request #16190 from smithfarm/wip-20026-kraken kraken: mds: unresponsive when truncating a very large file Reviewed-by: Patrick Donnelly commit d0d98362e7363fd84334941085203aa3bfd7a2d2 Merge: 72e5d6116c 719ed0101b Author: Nathan Cutler Date: Tue Aug 1 11:08:47 2017 +0200 Merge pull request #15526 from badone/wip-async-sleep-timer-fix-kraken kraken: osd: Implement asynchronous scrub sleep Reviewed-by: Josh Durgin commit 72e5d6116cc1afd83f96e26e32c5949548c1dcbc Merge: 16ca3c7f40 533ff8a540 Author: Nathan Cutler Date: Mon Jul 31 16:47:45 2017 +0200 Merge pull request #16137 from smithfarm/wip-20024-kraken kraken: tests: HEALTH_WARN pool rbd pg_num 244 > pgp_num 224 during upgrade Reviewed-by: Kefu Chai commit 16ca3c7f40d37f1fb6e38665dd0c33fe3ab5a060 Merge: a1c2cbd074 4d1272e5c6 Author: Nathan Cutler Date: Mon Jul 31 15:31:12 2017 +0200 Merge pull request #14734 from smithfarm/wip-19670-kraken kraken: build/ops: logrotate is missing from debian package (kraken, master) Reviewed-by: Kefu Chai commit a1c2cbd074dbc2954fea0d2808a80958f17fe9c3 Merge: fd6816bb72 ccb33bab37 Author: Nathan Cutler Date: Mon Jul 31 15:21:02 2017 +0200 Merge pull request #16166 from smithfarm/wip-19840-kraken kraken: rgw: civetweb frontend segfaults in Luminous Reviewed-by: Radoslaw Zarzynski commit a91253ec50dd5c2ab85b00d52361fd22580e1ed3 Author: Sage Weil Date: Tue Mar 7 22:35:02 2017 -0500 qa/suites/upgrade/jewel-x: do not thrash cluster full during upgrade The mon command varies. Signed-off-by: Sage Weil (cherry picked from commit 39fdc53fe5f33678fbbd00cf8810b6d523d0040c) commit afb5f4b0b6eb0bbbfbdd628ff466e1fad6e6e4ef Author: Sage Weil Date: Mon Mar 6 15:16:13 2017 -0500 qa/suites/upgrade/jewel-x/parallel: expand workload matrix These should run independently against a racing upgrade. Signed-off-by: Sage Weil (cherry picked from commit 1a0ad2b48839fea75d4d3339f7d198404ff1ac37) commit a88bd845df01b9fa8d1b0482b1c4d6bfae301f40 Author: Sage Weil Date: Mon Mar 6 14:11:53 2017 -0500 qa/suites/upgrade/jewel-x/stress-split-erasure-code: box thrashosds Signed-off-by: Sage Weil (cherry picked from commit b2d354d56359d08b35cbea58f811c7bafb700d31) commit d4521fc2b9b13cee58abb4b578e8ba00c6aca0ae Author: Sage Weil Date: Mon Mar 6 14:07:53 2017 -0500 qa/suites/upgrade/jewel-x/stress-split: finish client.0 upgrade too Signed-off-by: Sage Weil (cherry picked from commit 56f9387736eed136b38c087a4805821063e9f8ab) commit 907888f881cc7224f9f9c17f38cc98773ca33906 Author: Sage Weil Date: Sun Mar 5 14:14:40 2017 -0500 qa/suites/upgrade/jewel-x: remove kraken references Signed-off-by: Sage Weil (cherry picked from commit 468285b65d6bef2d86c80ebdfecf0920294ca5cd) commit 70415451e90d5aba473dbb3b1bd00ce4de0d077b Author: Sage Weil Date: Wed Mar 1 19:01:20 2017 -0600 qa/suite/upgrade/jewel-x: drop x86 ec test Signed-off-by: Sage Weil (cherry picked from commit 1e0e53c80d8fa88970cee1d61aaa8514004547ac) commit fa7934b3abe395d52f127c6e170a54296d936211 Author: Sage Weil Date: Mon Feb 20 14:38:49 2017 -0500 qa/suites/upgrade/jewel-x: fix upgrade order; fix split grouping Signed-off-by: Sage Weil (cherry picked from commit 269eafb2027ea6dfbad730f6fb1481a1cabc5e60) Conflicts: stress-split/6-kraken.yaml (do not delete) stress-split/6-luminous.yaml (do not add) stress-split-erasure-code/6-kraken.yaml (do not delete) stress-split-erasure-code/6-luminous.yaml (do not add) commit b574cb341c9ce0f9d986c79c03951c7c9eafb0b8 Author: Sage Weil Date: Mon Mar 6 19:01:12 2017 -0500 qa/suites/upgrade/jewel-x/parallel: upgrade mons before osds Signed-off-by: Sage Weil (cherry picked from commit 3c80e15c3b34ac2adc4e70f09929e3bc01785594) commit 4eb3c2dbf321cf7a110b2ac7022d5d9cc80b8f3a Author: Sage Weil Date: Wed Mar 1 13:58:07 2017 -0600 qa/suites/upgrade/jewel-x/parallel: fix upgrade to luminous Signed-off-by: Sage Weil (cherry picked from commit 2e3ea53b41cacc4b4bbc4ffe7342fb00233009cf) Conflicts: 4-kraken.yaml (do not delete) 4-luminous.yaml (do not add) commit fd6816bb725f7699537276c13f68d815a7406228 Merge: a6fdfccd11 b7503d3a80 Author: Nathan Cutler Date: Mon Jul 31 12:32:45 2017 +0200 Merge pull request #13871 from smithfarm/wip-19162-kraken kraken: rgw: rgw_file: fix marker computation commit a6fdfccd117180cf3906e1349b7af19462eae0e1 Merge: b50909c861 a7af766aa8 Author: Nathan Cutler Date: Mon Jul 31 12:26:47 2017 +0200 Merge pull request #16133 from smithfarm/wip-18378-kraken kraken: msg/simple/SimpleMessenger.cc: 239: FAILED assert(!cleared) Reviewed-by: Josh Durgin commit b50909c861450f1aeb40d3fa1a0ca4d997d09a0c Merge: 938b723717 c6542ac7e6 Author: Nathan Cutler Date: Mon Jul 31 12:23:08 2017 +0200 Merge pull request #16134 from smithfarm/wip-19340-kraken kraken: An OSD was seen getting ENOSPC even with osd_failsafe_full_ratio passed Reviewed-by: David Zafman commit 17fd055a4f68cb3c51baa1ac1fb089d15b818312 Author: Yan, Zheng Date: Tue Apr 25 16:21:24 2017 +0800 osdc/Filer: truncate large file party by party Fixes: http://tracker.ceph.com/issues/19755 Signed-off-by: "Yan, Zheng" (cherry picked from commit 5fab215e461e5ecc36c0f9d9ea867f6c45e80263) Conflicts: src/osdc/Filer.h src/osdc/Filer.cc src/mds/MDCache.cc commit 938b723717745eff4e3ab18b9a0f11cfc5190876 Merge: e56d4c481d b8dfa2f73a Author: Nathan Cutler Date: Mon Jul 31 11:58:20 2017 +0200 Merge pull request #16131 from smithfarm/wip-swift-kraken tests: swift.py: clone the ceph-jewel branch Reviewed-by: Casey Bodley commit 719ed0101b1bfdd4b71ef84101515492597153f9 Author: Brad Hubbard Date: Mon May 22 13:21:25 2017 +1000 osd: Move scrub sleep timer to osdservice PR 14886 erroneously creates a scrub sleep timer for every pg resulting in a proliferation of threads. Move the timer to the osd service so there can be only one. Fixes: http://tracker.ceph.com/issues/19986 Signed-off-by: Brad Hubbard (cherry picked from commit f110a82437df79dc20207d296e8229fc0e9ce18b) commit 460a820a3b2fbd48c8a7966502b235aae8d5d298 Author: Brad Hubbard Date: Mon Apr 24 14:10:47 2017 +1000 osd: Implement asynchronous scrub sleep Rather than blocking the main op queue just do an async sleep. Fixes: http://tracker.ceph.com/issues/19497 Signed-off-by: Brad Hubbard (cherry picked from commit 7af3e86c2e4992db35637864b83832535c94d0e6) commit e56d4c481dead2f8ccb7baaae80db2f4acfc7bf0 Merge: 024272160d bdb16fd0f5 Author: Nathan Cutler Date: Tue Jul 25 07:14:37 2017 +0200 Merge pull request #16493 from smithfarm/wip-suppress-upgrade-fail tests: run certain upgrade/jewel-x tests on Xenial only Reviewed-by: Josh Durgin Reviewed-by: Gregory Farnum commit 024272160de37c3ed2a2df701fc1e8f4c11c2da5 Merge: d8e0ddc57b 838d7840f0 Author: Nathan Cutler Date: Tue Jul 25 07:13:48 2017 +0200 Merge pull request #14597 from gregsfortytwo/wip-kraken-snaptrim kraken: core: improve control and throttling of the snap trimmer Reviewed-by: Josh Durgin commit bdb16fd0f533691b7b7a01b176764ad907143814 Author: Nathan Cutler Date: Sat Jul 22 10:13:22 2017 +0200 tests: upgrade/jewel-x/parallel: drop duplicate kraken.yaml This yaml file has the effect of re-running "ceph osd set require_kraken_osds" at the very end of the test. Drop it. Signed-off-by: Nathan Cutler commit dc8c2231cefdbc8811cdea380d7c856b57c62251 Author: Nathan Cutler Date: Sat Jul 22 00:06:48 2017 +0200 tests: run certain upgrade/jewel-x tests on Xenial only This PR drops two upgrade/jewel-x test cases that are not compatible with https://github.com/ceph/ceph/pull/14597 Signed-off-by: Nathan Cutler commit b8dfa2f73aff0998a4aebf6f61865983f476b8d1 Author: Nathan Cutler Date: Sun Jun 25 09:27:47 2017 +0200 tests: swift.py: clone the ceph-jewel branch The master branch of ceph/swift.git contains tests that are incompatible with Jewel and Hammer. The ceph-jewel branch omits these tests. Signed-off-by: Nathan Cutler (cherry picked from commit a86ce728954a765797ce634025d43650d990e480) Conflicts: qa/tasks/swift.py: clone ceph-kraken branch instead of ceph-jewel commit d8e0ddc57b0e2632baf70bebf409d52808c202fa Merge: c539ea49a8 19c7524ace Author: Nathan Cutler Date: Fri Jul 21 08:22:25 2017 +0200 Merge pull request #16111 from smithfarm/wip-20497-kraken kraken: tests: insufficient timeout in radosbench task Reviewed-by: Gregory Farnum commit af0a6df270f3c469280cae877c177086ddf10245 Author: Nathan Cutler Date: Thu Jul 6 20:17:49 2017 +0200 rgw: lease_stack: use reset method instead of assignment It seems that the intent of 45877d38fd9a385b2f8b13e90be94d784898b0b3 was to change all instances of "lease_stack = ..." to "lease_stack.reset(...)", but this one was missed. Signed-off-by: Nathan Cutler (cherry picked from commit fc425afeb2e2a6ba3c98c612b3977aea619c9f73) Conflicts: src/rgw/rgw_sync.cc (trivial) commit c539ea49a8dab85bade53140d80f2a05f3f1860c Merge: 0a9badb3e2 640a7a2629 Author: Nathan Cutler Date: Thu Jul 20 16:38:27 2017 +0200 Merge pull request #16178 from smithfarm/wip-20268-kraken kraken: rgw: get wrong content when download object with specific range when compression was enabled Reviewed-by: Casey Bodley commit d24edde072ec63d35965e30a58aaaf02547f9c57 Author: Boris Ranto Date: Fri Jul 7 12:37:55 2017 +0200 rpm: Fix undefined FIRST_ARG If FIRST_ARG is undefined, the rpms will show an error on upgrade because the condition in the postun script gets broken. This was a regression introduced by commit for issue 20077 that moved ceph-disk unit files to ceph-base. Fixes: http://tracker.ceph.com/issues/20077 Signed-off-by: Boris Ranto (cherry picked from commit 562816914ccca8e4e7d9c31f333db2f0da6f7c99) commit eac6a0dd475e22512cc24bb3b8fe018c70cf7bc8 Author: Boris Ranto Date: Mon Jun 5 18:44:18 2017 +0200 selinux: Install ceph-base before ceph-selinux We need to have ceph-base installed before ceph-selinux to use ceph-disk in %post script. The default ordering is random and so the installation randomly failed to relabel the files. Fixes: http://tracker.ceph.com/issues/20184 Signed-off-by: Boris Ranto (cherry picked from commit e69086645d3411a2ed781609b670eb5f16ac4810) Conflicts: ceph.spec.in: No _epoch_prefix in kraken. commit 0a9badb3e2be41be5e7de6d865a8e323b3706fa8 Merge: 13b04089ce baa772372c Author: Nathan Cutler Date: Wed Jul 19 23:09:32 2017 +0200 Merge pull request #16114 from smithfarm/wip-20500-kraken kraken: cephfs: src/test/pybind/test_cephfs.py fails Reviewed-by: Patrick Donnelly commit 13b04089ceff7136785c37595f526841574cdcd9 Merge: 34c32b9d07 b941ca7eac Author: Nathan Cutler Date: Wed Jul 19 23:03:19 2017 +0200 Merge pull request #16108 from smithfarm/wip-19763-kraken kraken: cephfs: non-local quota changes not visible until some IO is done Reviewed-by: Patrick Donnelly commit 34c32b9d0721407511a8420ad3888cbee2fd8f5f Merge: bb95e12790 4241a6eef0 Author: Nathan Cutler Date: Wed Jul 19 23:02:09 2017 +0200 Merge pull request #16107 from smithfarm/wip-19710-kraken kraken: mds: enable daemon to start when session ino info is corrupt Reviewed-by: Patrick Donnelly commit bb95e127909052679a883375feb55935ccb127d5 Merge: 8202ddcf11 0b1dee0edc Author: Nathan Cutler Date: Wed Jul 19 23:01:01 2017 +0200 Merge pull request #16106 from smithfarm/wip-19680-kraken kraken: mds: damage reporting by ino number is useless Reviewed-by: Patrick Donnelly commit 8202ddcf1156093c58a0c161d6cdc3455fc440b5 Merge: e2efa6e9e7 f458d60838 Author: Nathan Cutler Date: Wed Jul 19 23:00:13 2017 +0200 Merge pull request #16105 from smithfarm/wip-19678-kraken kraken: cephfs: ceph-fuse does not recover after lost connection to MDS Reviewed-by: Patrick Donnelly commit e2efa6e9e742dba2b9779e8b7382c173abfea0af Merge: 055319bfe7 7674f84ddf Author: Nathan Cutler Date: Wed Jul 19 22:58:56 2017 +0200 Merge pull request #16104 from smithfarm/wip-19676-kraken kraken: cephfs: Test failure: test_data_isolated (tasks.cephfs.test_volume_client.TestVolumeClient) Reviewed-by: Patrick Donnelly commit 055319bfe7e6dd15417e30b708590a4040d2a884 Merge: 5c6a770308 bee73d2429 Author: Nathan Cutler Date: Wed Jul 19 22:56:04 2017 +0200 Merge pull request #16103 from smithfarm/wip-19674-kraken kraken: cephfs: mds is crushed, after I set about 400 64KB xattr kv pairs to a file Reviewed-by: Patrick Donnelly commit 5c6a770308200788231173c270833a15d04765ed Merge: 8d86192e34 32c7ebe3e6 Author: Nathan Cutler Date: Wed Jul 19 22:55:17 2017 +0200 Merge pull request #16102 from smithfarm/wip-19672-kraken kraken: mds: assert fail when shutting down Reviewed-by: Patrick Donnelly commit 8d86192e34dfd3d91a99a354545fb3e5b7b94f79 Merge: 99d0afda7f b267a1a8be Author: Nathan Cutler Date: Wed Jul 19 22:54:07 2017 +0200 Merge pull request #16101 from smithfarm/wip-19669-kraken kraken: mds: daemon goes readonly writing backtrace for a file whose data pool has been removed Reviewed-by: Patrick Donnelly commit 99d0afda7f8748b13940267c3d5286b8a98d5a8f Merge: 818ecc66c7 93e81a87ac Author: Nathan Cutler Date: Wed Jul 19 22:53:11 2017 +0200 Merge pull request #16100 from smithfarm/wip-19667-kraken kraken: cephfs: mount point break off problem after mds switch Reviewed-by: Patrick Donnelly commit 818ecc66c71e3ccb351353a36b32b3dd0c6c2123 Merge: c8ccfc72b3 e72d6362c1 Author: Nathan Cutler Date: Wed Jul 19 22:05:12 2017 +0200 Merge pull request #16099 from smithfarm/wip-19664-kraken kraken: mds: C_MDSInternalNoop::complete doesn't free itself Reviewed-by: Patrick Donnelly commit c8ccfc72b3556969722c24eacdd78828a3194b7f Merge: fbcef7d42a 3c4a5ea385 Author: Nathan Cutler Date: Wed Jul 19 22:03:52 2017 +0200 Merge pull request #14998 from jan--f/wip-19845-kraken kraken: cephfs: normalize file open flags internally used by cephfs Reviewed-by: Patrick Donnelly commit fbcef7d42a3832c2e404aa0d141238252045d3d9 Merge: dbaae75029 f32e26e909 Author: Nathan Cutler Date: Wed Jul 19 16:51:05 2017 +0200 Merge pull request #16168 from smithfarm/wip-20031-kraken kraken: rgw: Swift's at-root features (/crossdomain.xml, /info, /healthcheck) are broken Reviewed-by: Radoslaw Zarzynski commit dbaae750299d9169e2e1dda9d0af98521c8abb5e Merge: e26575be67 e773b304ee Author: Nathan Cutler Date: Wed Jul 19 16:15:21 2017 +0200 Merge pull request #16164 from smithfarm/wip-19777-kraken kraken: rgw: swift: disable revocation thread under certain circumstances Reviewed-by: Radoslaw Zarzynski commit e26575be67e00dbf674ff77ef6e7729f22750d8b Merge: 415174d29e 53024570e7 Author: Nathan Cutler Date: Wed Jul 19 15:43:16 2017 +0200 Merge pull request #16191 from smithfarm/wip-20028-kraken kraken: cephfs: Deadlock on two ceph-fuse clients accessing the same file Reviewed-by: Yan, Zheng commit 415174d29e20f73884e81773fd0b4d1055d439b4 Merge: 2879b80281 b27595ae9c Author: Nathan Cutler Date: Wed Jul 19 15:32:52 2017 +0200 Merge pull request #16183 from smithfarm/wip-20405-kraken kraken: rgw: Lifecycle thread will still handle the bucket even if it has been removed Reviewed-by: Casey Bodley Reviewed-by: Daniel Gryniewicz commit 2879b80281c1d4bd8b8daeb5662879a296274fca Merge: eed9a165bd 3105e2327e Author: Nathan Cutler Date: Wed Jul 19 15:31:50 2017 +0200 Merge pull request #16181 from smithfarm/wip-20363-kraken kraken: rgw: VersionIdMarker and NextVersionIdMarker are not returned when listing object versions Reviewed-by: Casey Bodley commit eed9a165bd1dd95f5afe8ccd78a742fbf16a4503 Merge: b759d4362b d0e742cafa Author: Nathan Cutler Date: Wed Jul 19 15:31:14 2017 +0200 Merge pull request #16179 from smithfarm/wip-20269-kraken kraken: rgw: wrong object size after copy of uncompressed multipart objects Reviewed-by: Casey Bodley commit b759d4362b7a3ae2bc4174eb0ef328f919644e6c Merge: 82e9c73fe7 90288afc10 Author: Nathan Cutler Date: Wed Jul 19 15:30:33 2017 +0200 Merge pull request #16174 from smithfarm/wip-20261-kraken kraken: rgw: 'radosgw-admin usage show' listing 0 bytes_sent/received Reviewed-by: Casey Bodley commit 82e9c73fe70b5e78bc170ad566fa7900191ece70 Merge: 2c5b063f43 dda4d912cd Author: Nathan Cutler Date: Wed Jul 19 15:29:57 2017 +0200 Merge pull request #16173 from smithfarm/wip-20156-kraken kraken: fix: rgw crashed caused by shard id out of range when listing data log Reviewed-by: Casey Bodley commit 2c5b063f4386663f00ff897beee0523fa6e95acd Merge: 423d6c319f f8235c5c0a Author: Nathan Cutler Date: Wed Jul 19 15:29:22 2017 +0200 Merge pull request #16165 from smithfarm/wip-19839-kraken kraken: rgw: reduce log level of 'storing entry at' in cls_log Reviewed-by: Casey Bodley commit 423d6c319f55fb3eddc0602954645b3b1f1d7bd8 Merge: 0af35a358e 68a853a14d Author: Nathan Cutler Date: Wed Jul 19 15:28:30 2017 +0200 Merge pull request #16163 from smithfarm/wip-19766-kraken kraken: rgw: when uploading the objects continuesly in the versioned bucket, some objects will not sync Reviewed-by: Casey Bodley commit 0af35a358ea8a8b3376faf4d77bd75edb1e808fe Merge: 872a8b53ea 9a7a73edf1 Author: Nathan Cutler Date: Wed Jul 19 15:27:38 2017 +0200 Merge pull request #16162 from smithfarm/wip-19725-kraken kraken: rgw: S3 v4 authentication issue with X-Amz-Expires Reviewed-by: Casey Bodley commit 872a8b53ea022923e9c442011432df5e403520ae Merge: 129345c319 d52aeec031 Author: Nathan Cutler Date: Wed Jul 19 15:25:48 2017 +0200 Merge pull request #16161 from smithfarm/wip-19614-kraken kraken: multisite: rest api fails to decode large period on 'period commit' Reviewed-by: Casey Bodley commit 129345c3196effdaea08f9678d1a9f1cd8265d81 Merge: 72c3a04f23 91569f6385 Author: Nathan Cutler Date: Wed Jul 19 15:24:59 2017 +0200 Merge pull request #16139 from smithfarm/wip-20147-kraken kraken: rgw: 'gc list --include-all' command infinite loop the first 1000 items Reviewed-by: Casey Bodley commit 72c3a04f23a227713f50527b8b8651580556d45a Merge: 762901080c 6d2f959735 Author: Nathan Cutler Date: Wed Jul 19 14:29:31 2017 +0200 Merge pull request #16096 from smithfarm/wip-19336-kraken kraken: rbd: refuse to use an ec pool that doesn't support overwrites Reviewed-by: Jason Dillaman commit 762901080c3e7982a0f38705cd09578be1c6d940 Merge: ac8bd5ea97 5dab0825d7 Author: Nathan Cutler Date: Wed Jul 19 14:28:08 2017 +0200 Merge pull request #16097 from smithfarm/wip-19609-kraken kraken: tests: [librados_test_stub] cls_cxx_map_get_XYZ methods don't return correct value Reviewed-by: Jason Dillaman commit ac8bd5ea97b7f87533d0320048335f80dcc66224 Merge: c5d53bd299 ac706f2b4a Author: Nathan Cutler Date: Wed Jul 19 14:27:11 2017 +0200 Merge pull request #16184 from smithfarm/wip-20154-kraken kraken: rbd: Potential IO hang if image is flattened while read request is in-flight Reviewed-by: Jason Dillaman commit c5d53bd299807d90d3311625554e055696898e71 Merge: f73412843d 18f1830e8a Author: Nathan Cutler Date: Wed Jul 19 14:24:05 2017 +0200 Merge pull request #16187 from smithfarm/wip-20266-kraken kraken: rbd: [api] is_exclusive_lock_owner shouldn't return -EBUSY Reviewed-by: Jason Dillaman commit f73412843db6cb9e3416d66a2ddf92e1b8d7038f Merge: 0d01813a8c ce874ab676 Author: Nathan Cutler Date: Wed Jul 19 14:22:57 2017 +0200 Merge pull request #16195 from dillaman/wip-20351-kraken kraken: tests: test/librbd: decouple ceph_test_librbd_api from libceph-common Reviewed-by: Nathan Cutler commit 0d01813a8c72e0d797d100e8e45d99f1ff99ba5a Merge: 771ebef418 379309b552 Author: Nathan Cutler Date: Wed Jul 19 12:56:47 2017 +0200 Merge pull request #15486 from dillaman/wip-20022-kraken kraken: rbd-mirror: ensure missing images are re-synced when detected Reviewed-by: Mykola Golub commit 771ebef418eb0d585ca9bd9dd565ea9f1caf3766 Merge: 9d6e0f2ef8 3173da6e3b Author: Nathan Cutler Date: Wed Jul 19 12:50:22 2017 +0200 Merge pull request #14540 from smithfarm/wip-18910-kraken kraken: rbd-nbd: check /sys/block/nbdX/size to ensure kernel mapped correctly Reviewed-by: Mykola Golub commit 9d6e0f2ef86b34b0bd2443f8c8a747bb15732e7c Merge: bb1b7c3b10 9a26882289 Author: Nathan Cutler Date: Wed Jul 19 08:05:11 2017 +0200 Merge pull request #16140 from smithfarm/wip-20271-kraken kraken: tests: LibRadosMiscConnectFailure.ConnectFailure hang Reviewed-by: Kefu Chai commit bb1b7c3b1060b983eeb8e3a3f9607306b47ead77 Merge: 5ea47f79c9 76fedab576 Author: Nathan Cutler Date: Tue Jul 18 23:14:59 2017 +0200 Merge pull request #16112 from smithfarm/wip-20499-kraken kraken: tests: ObjectStore/StoreTest.OnodeSizeTracking/2 fails on bluestore Reviewed-by: Sage Weil commit 5ea47f79c91229b57981bac77472102c868cfa89 Merge: 1913303660 47f751f48b Author: Yehuda Sadeh Date: Tue Jul 18 13:58:14 2017 -0700 Merge pull request #14960 from yehudasa/wip-19704 kraken: civetweb: move to post 1.8 version Reviewed-by: Yehuda Sadeh commit 838d7840f023c5802507727231983311182a1c05 Author: Greg Farnum Date: Mon Jul 10 13:33:26 2017 -0700 osd: do not default-abort on leaked pg refs Signed-off-by: Greg Farnum (cherry picked from commit 4caf2df0c380a1281db9509b3feb342705512b58) commit 4bf34200896ce120bba2e2d974c4f3dadb342821 Author: Greg Farnum Date: Thu May 25 22:14:38 2017 -0700 osd: shutdown our reserver_finisher earlier in the process This finisher thread has a lot of callbacks which can hold PGRefs. Make sure we drain them out before checking that all the PGs have finished and have no outstanding references. Moving this should be safe; we've already stopped the op thread et al and the only things still running are the OSDService's objecter_finisher, recovery_request_timer, and snap_sleep_timer (which has definitely been emptied by the time we get here as it's synchronously cleared out on PG shutdown). Signed-off-by: Greg Farnum (cherry picked from commit 66ea9c1f66ae31035e62bd4335b08948b1e8e5e2) commit 2d5cafdf6461c213aa2642210a3c4efabce61efd Author: Greg Farnum Date: Thu May 25 21:52:49 2017 -0700 osd: Reset() the snaptrimmer on shutdown We were failing to exit various wait states which held PGRefs. Error! Fixes: http://tracker.ceph.com/issues/19931 Signed-off-by: Greg Farnum (cherry picked from commit b0e9deeea8a8e90f6d7e9d56b6b4aed890e01d7b) commit cc0046a999600b2e684c17fddaebcf451c259443 Author: Greg Farnum Date: Wed May 24 18:43:34 2017 -0700 osd: print out pg debug ref counts on acquire/drop Signed-off-by: Greg Farnum (cherry picked from commit a3b028a6d1ba74ed975ebd665b1b50fb7e5039a4) commit c6542ac7e6408f26ce7376d3a58eed0661a17099 Author: David Zafman Date: Fri Jul 7 10:53:41 2017 -0700 mon: Fix status output warning for mon_warn_osd_usage_min_max_delta Fixes: http://tracker.ceph.com/issues/20544 Caued by: 489e810c37ed6fb9d32d1015634041a577501ee4 Signed-off-by: David Zafman (cherry picked from commit 56f9808016ab289bdc0ce7cfbb0503c78b509593) Conflicts: src/mon/PGMap.cc: kraken has no daa0793c393a21bd2dd2ec52a0efd181e1032400 so we manually apply the changes from 56f9808016ab289bdc0ce7cfbb0503c78b509593 to src/mon/PGMonitor.cc instead of to src/mon/PGMap.cc commit 2f26f8cbd3e81700c42202f8d933f1576772e212 Author: Sage Weil Date: Tue Apr 18 13:54:56 2017 -0400 mon/PGMonitor: clean up min/max span warning Clean up option naming. Signed-off-by: Sage Weil (cherry picked from commit 489e810c37ed6fb9d32d1015634041a577501ee4) commit 90b7fd8a0accc2f8f458d35199361344652d8859 Author: David Zafman Date: Fri Feb 17 12:27:36 2017 -0800 bluestore: Fix BlueStore::statfs available amount to not include bluefs min reserved This fixes OSD crashes because checking osd_failsafe_full_ratio won't work without accurate statfs information. Signed-off-by: David Zafman (cherry picked from commit 72d83f848a35e8831d66e8529c4e26f51e845da6) commit d69e4e5e6ad633e38fc62ca0859cea1f146744b1 Author: David Zafman Date: Thu Feb 16 22:23:06 2017 -0800 osd: Round fullness in message to correspond to df -h This really only works after journal drains because we adjust for the journal. Signed-off-by: David Zafman (cherry picked from commit 26dcb591f9af01ed444aa758c3d601e7f67261b2) commit eb887e3cb1f55f016ca70ac3d4441aa03478d934 Author: David Zafman Date: Thu Feb 16 17:25:12 2017 -0800 filestore: Account for dirty journal data in statfs Fixes: http://tracker.ceph.com/issues/16878 Signed-off-by: David Zafman (cherry picked from commit 78adb70c21c6b8e6a9191b76917919b125a9490f) commit bc324e0446d5fd6b35651b024926c5a3099a0aff Author: David Zafman Date: Tue Feb 14 16:37:07 2017 -0800 mon: Add warning if diff in OSD usage > config mon_warn_osd_usage_percent (10%) Signed-off-by: David Zafman (cherry picked from commit c8004e6558359fb542e45bb4b483a6c91afdc0b4) commit 2c2e0a353bd518f1e44435376ad1ad9a0ee6fe67 Author: David Zafman Date: Tue Feb 14 14:40:05 2017 -0800 mon: Bump min in ratio to 75% Signed-off-by: David Zafman (cherry picked from commit 830cc7aa7be1ccd8f54f056b6a58e923cadd1c2b) commit 899f3fca7f89ab37d20cb1ec06b911ba039c52a3 Author: David Zafman Date: Tue Feb 14 14:38:53 2017 -0800 osd: Fix ENOSPC crash message text Signed-off-by: David Zafman (cherry picked from commit 50cfe03fcba253c8380b21043ed03879134d6836) commit 19133036602d9dc9eb234b5e4dec6b845b4ade4b Merge: 7224ebc1f1 85cefa113b Author: Nathan Cutler Date: Tue Jul 18 19:50:46 2017 +0200 Merge pull request #16143 from smithfarm/wip-20365-kraken kraken: mon: osd crush set crushmap need sanity check Reviewed-by: Loic Dachary commit 7224ebc1f18fec1f472688f6494ed10d2b574f88 Merge: 47359c12e6 ea186807c3 Author: Nathan Cutler Date: Tue Jul 18 19:48:51 2017 +0200 Merge pull request #16138 from smithfarm/wip-20034-kraken kraken: ceph-disk: Racing between partition creation & device node creation Reviewed-by: Loic Dachary commit 47359c12e69fbde4c36403ccd7992fe4ecf9223b Merge: 9860e65818 f04eb66d36 Author: Nathan Cutler Date: Tue Jul 18 19:47:34 2017 +0200 Merge pull request #16135 from smithfarm/wip-20010-kraken kraken: ceph-disk: separate ceph-osd --check-needs-* logs Reviewed-by: Loic Dachary commit f2d61c199d7ba103290178573984db98e8c8ed21 Author: Jason Dillaman Date: Fri Jul 14 10:32:28 2017 -0400 qa/tasks: rbd-mirror daemon not properly run in foreground mode Fixes: http://tracker.ceph.com/issues/20630 Signed-off-by: Jason Dillaman (cherry picked from commit 4fa1918717b25a5ffafbf649eedcfe7d5ab829c2) commit cf06edd36b176a10e6ce3d62eb957bfdb0146703 Author: Jason Dillaman Date: Thu Jun 29 14:54:40 2017 -0400 rbd: do not attempt to load key if auth is disabled Fixes: http://tracker.ceph.com/issues/19035 Signed-off-by: Jason Dillaman (cherry picked from commit 8b9c8df6d7f0b75c5451953bb322bc1f9afb6299) commit f72ea6818691e1824d4c22630e856d3b280a008c Author: lu.shasha Date: Tue May 9 15:05:03 2017 +0800 rgw: when create_bucket use the same num_shards with info.num_shards pr #14388 only fix the num_shards in BucketInfo, "init_bucket_index" function still use local num_shards Fixes: http://tracker.ceph.com/issues/19745 Signed-off-by: Shasha Lu (cherry picked from commit 4ce64a190b4ff36985e785e574c077d39796feea) Conflicts: src/rgw/rgw_rados.cc - init_bucket_index() called earlier commit 9182c279cf2f2228471d506113c61566cbfea4bb Author: lu.shasha Date: Fri Apr 7 15:34:27 2017 +0800 rgw: using the same bucket num_shards as master zg when create bucket in secondary zg create bucket in secondary zonegroup will forward to master. The master may have different num_shards option. So when create bucket in local, should use master's num_shards instead of local num_shards option. Fixes: http://tracker.ceph.com/issues/19745 Signed-off-by: Shasha Lu (cherry picked from commit a34c4b8fb13dd5590eb3c6ecb5e55207ed8e3ee8) Conflicts: src/rgw/rgw_op.cc - no RGWBulkUploadOp:: methods in kraken; modifications to RGWBulkUploadOp::handle_dir() omitted commit b758348447e60af23b114aa47c28f151ffd97792 Author: Pavan Rallabhandi Date: Fri Apr 14 21:42:45 2017 +0530 rgw: add a field to store generic user data in the bucket index, that can be populated/fetched via a configurable custom http header Signed-off-by: Pavan Rallabhandi (cherry picked from commit abca7a86c3cfbb58fafb5d057d9d6f5017a53704) Conflicts: src/rgw/rgw_op.cc Signature fixes for RGWPutObjProcessor_Multipart::do_complete(). src/rgw/rgw_op.h A new member field `crypt_http_responses` is added in the class `RGWPutObj` in master version, which is not required in Kraken. src/rgw/rgw_rados.cc In RGWRados::Bucket::UpdateIndex::complete, RGWObjEnt has been removed in master, which has to be retained in Kraken. In RGWRados::cls_obj_complete_op, user_data is added to the rgw_bucket_dir_entry_meta In RGWRados::cls_bucket_list, the user_data field of RGWObjEnt is populated. src/rgw/rgw_rados.h In UpdateIndex::complete(), remove_objs is of type rgw_obj_key in Kraken instead of rgw_obj_index_key RGWPutObjProcessor_Multipart is not part of this file in Kraken. src/rgw/rgw_rest_swift.cc In RGWListBucket_ObjStore_SWIFT::send_response(), there is no meta struct in Kraken. src/rgw/rgw_common.h Add user_data field in RGWObjEnt structure. src/rgw/rgw_json_enc.cc Add user_data field while dumping RGWObjEnt. commit 4fa19ce669338bb2d4c3233c70da8a3829a468f6 Author: Greg Farnum Date: Mon Jul 10 13:31:21 2017 -0700 osd: repair the PG_DEBUG_REFS build Signed-off-by: Greg Farnum (cherry picked from commit ec4185d1980b48b01687a177248d0894f326dc37) Conflicts: src/osd/OSD.cc src/osd/Session.h commit 2a3e66c388892efcecebd8eb3d7bf082be485bab Author: Greg Farnum Date: Wed May 24 00:19:51 2017 -0700 osd: use PGRef instead of boost::intrusive_ptr everywhere Signed-off-by: Greg Farnum (cherry picked from commit 306ad85f0fb4597ba3bb3c04a67abdaba556ba74) commit 9860e658189e3ac775102b3f66676b0a77a80ba7 Merge: 6e1023ad24 c65eaaafb1 Author: Sage Weil Date: Sat Jul 8 21:39:59 2017 -0500 Merge pull request #16215 from liewegas/wip-17743 kraken: CMakeLists.txt: disable memstore make check test Reviewed-by: Josh Durgin commit c65eaaafb174ac1f62fe8fc08ae63deadeffd0ad Author: Sage Weil Date: Fri Jul 7 11:38:25 2017 -0400 CMakeLists.txt: disable memstore make check test This fails frequently on kraken, but is probably a wontfix. See http://tracker.ceph.com/issues/17743 Signed-off-by: Sage Weil commit 379309b552db51dcab9b08ba5437c1dae5cd45a7 Author: Jason Dillaman Date: Wed May 3 21:36:21 2017 -0400 rbd-mirror: ensure missing images are re-synced when detected Fixes: http://tracker.ceph.com/issues/19811 Signed-off-by: Jason Dillaman (cherry picked from commit 74bd4f230a0cb7b709f2cb5c6db3dc79f0d8dede) Conflicts: src/test/rbd_mirror/image_replayer/test_mock_BootstrapRequest.cc: trivial resolution src/tools/rbd_mirror/image_replayer/BootstrapRequest.h: trivial resolution commit 6e1023ad2482f5829c06a54adb4271643b1e53c4 Merge: 6353508134 076e4b7991 Author: Nathan Cutler Date: Fri Jul 7 09:49:51 2017 +0200 Merge pull request #14067 from asheplyakov/19322-bp-kraken kraken: rgw: fix break inside of yield in RGWFetchAllMetaCR Reviewed-by: Orit Wasserman commit 6353508134bd4d2bcbca7d0f9a42788895fe5c27 Merge: 425992f92d 87811ff161 Author: Nathan Cutler Date: Fri Jul 7 09:48:57 2017 +0200 Merge pull request #14509 from smithfarm/wip-18499-kraken kraken: rgw: Realm set does not create a new period Reviewed-by: Orit Wasserman commit 425992f92d30c8503053311471322a9849a37746 Merge: 39bf493984 6860fce424 Author: Nathan Cutler Date: Fri Jul 7 09:48:01 2017 +0200 Merge pull request #14511 from smithfarm/wip-18772-kraken kraken: rgw: crash when updating period with placement group Reviewed-by: Orit Wasserman commit 39bf493984cb98c5a3f7691458188fea7a246c70 Merge: eb397c1780 1cb9868456 Author: Nathan Cutler Date: Fri Jul 7 09:45:15 2017 +0200 Merge pull request #14513 from smithfarm/wip-18843-kraken kraken: rgw: usage stats and quota are not operational for multi-tenant users Reviewed-by: Orit Wasserman commit eb397c178028530d061e9edc32f84ddd6d826a89 Merge: 1ed342d75d 5f4c38fa6b Author: Nathan Cutler Date: Fri Jul 7 09:44:21 2017 +0200 Merge pull request #14515 from smithfarm/wip-18904-kraken kraken: rgw: first write also tries to read object Reviewed-by: Orit Wasserman commit 1ed342d75d0befb602597385b5f49d27a244d450 Merge: d3a2417cf1 935cc11595 Author: Nathan Cutler Date: Fri Jul 7 09:42:36 2017 +0200 Merge pull request #14516 from smithfarm/wip-18909-kraken kraken: rgw: the swift container acl does not support field .ref Reviewed-by: Orit Wasserman Reviewed-by: Radoslaw Zarzynski commit d3a2417cf11e6ccebf25de8fe12242c1e8eda626 Merge: ce95f0f7d8 ffe5bbfc6e Author: Nathan Cutler Date: Fri Jul 7 09:25:43 2017 +0200 Merge pull request #14517 from smithfarm/wip-19047-kraken kraken: rgw: fix use of marker in List::list_objects() Reviewed-by: Orit Wasserman commit ce95f0f7d81af4bfbad7afb4f429b9cfa2e37726 Merge: aab812c517 03e5be4498 Author: Nathan Cutler Date: Fri Jul 7 09:24:13 2017 +0200 Merge pull request #14524 from smithfarm/wip-19477-kraken kraken: rgw: S3 v4 authentication issue with X-Amz-Expires Reviewed-by: Orit Wasserman Reviewed-by: Radoslaw Zarzynski commit aab812c51706fc3395af3c34b0b3720e84c30b61 Merge: a441042e00 a5fe812483 Author: Nathan Cutler Date: Fri Jul 7 09:22:26 2017 +0200 Merge pull request #14525 from smithfarm/wip-19479-kraken kraken: rgw: zonegroupmap set does not work Reviewed-by: Orit Wasserman commit a441042e00cb752737e303c4c38322fadf13a2f6 Merge: 3b128e458f cac2647189 Author: Nathan Cutler Date: Fri Jul 7 09:21:22 2017 +0200 Merge pull request #14528 from smithfarm/wip-19534-kraken kraken: rgw: Error parsing xml when get bucket lifecycle Reviewed-by: Daniel Gryniewicz Reviewed-by: Orit Wasserman commit 3b128e458f86a23e8b6ff987b0f0774e7d5597bb Merge: 6366b2382b 8f81bb33c7 Author: Nathan Cutler Date: Fri Jul 7 09:19:08 2017 +0200 Merge pull request #14606 from asheplyakov/19608-kraken kraken: rgw: multisite: fetch_remote_obj() gets wrong version when copying from remote Reviewed-by: Orit Wasserman commit 6366b2382b8d457ab9e544cc889aa77f024a626f Merge: 19280ae91e 727653623f Author: Nathan Cutler Date: Fri Jul 7 09:18:10 2017 +0200 Merge pull request #15384 from asheplyakov/20015-bp-kraken kraken: cls/rgw: list_plain_entries() stops before bi_log entries Reviewed-by: Orit Wasserman commit 19280ae91e710cbb9569fc1dc3a6a24b6ea05ca4 Merge: eb56af6c0d 595c31f390 Author: Nathan Cutler Date: Fri Jul 7 09:03:16 2017 +0200 Merge pull request #14522 from smithfarm/wip-19472-kraken kraken: rgw: add the remove-x-delete feature to cancel swift object expiration Reviewed-by: Radoslaw Zarzynski commit eb56af6c0d8eea0e19a16da4e186e8edbfe13148 Merge: 4ccd32619f f8db63f24d Author: Nathan Cutler Date: Fri Jul 7 09:00:43 2017 +0200 Merge pull request #14523 from smithfarm/wip-19475-kraken kraken: rgw: multisite: EPERM when trying to read SLO objects as system/admin user Reviewed-by: Radoslaw Zarzynski commit 4ccd32619f39a4f6fa7a6700c8c90c622c83d67a Merge: e59eba0ff3 c3d8444890 Author: Nathan Cutler Date: Fri Jul 7 08:58:44 2017 +0200 Merge pull request #14519 from smithfarm/wip-19175-kraken kraken: swift API: cannot disable object versioning with empty X-Versions-Location Reviewed-by: Radoslaw Zarzynski commit e59eba0ff375011c18b3a8dcdb228c71c9085fe9 Merge: aaa7992ee5 faf80bc1dd Author: Nathan Cutler Date: Fri Jul 7 08:56:47 2017 +0200 Merge pull request #14526 from smithfarm/wip-19524-kraken kraken: rgw: 'radosgw-admin zone create' command with specified zone-id creates a zone with different id Reviewed-by: Radoslaw Zarzynski commit aaa7992ee5fd2767d8a144c47255489080f5fb47 Merge: 6c3daeb297 e5719234b4 Author: Nathan Cutler Date: Fri Jul 7 08:55:30 2017 +0200 Merge pull request #14529 from smithfarm/wip-19573-kraken kraken: rgw: Response header of swift API returned by radosgw does not contain x-openstack-request-id. But Swift returns it. Reviewed-by: Radoslaw Zarzynski commit ce874ab676e8b5a09e32db19f077c650cff9979b Author: Kefu Chai Date: Sat Jun 10 23:59:19 2017 +0800 test/librbd: decouple ceph_test_librbd_api from libceph-common Fixes: http://tracker.ceph.com/issues/20175 Signed-off-by: Kefu Chai (cherry picked from commit b7287fdc4d70c5ecedda78ae367b98e5d8f61c5b) Conflicts: src/test/librbd/CMakeLists.txt: trivial resolution commit 57f7213d9fd5a9a82e46c4ca3f3dda87c3dde717 Author: Kefu Chai Date: Sat Jun 10 23:31:53 2017 +0800 test/librbd: replace libcommon classes using standard library so ceph_test_librbd_api has less dependencies on libcommon Fixes: http://tracker.ceph.com/issues/20175 Signed-off-by: Kefu Chai (cherry picked from commit bc8fa0193c8b2fb4b749ce9976bf6efbea833e96) Conflicts: src/test/librbd/test_librbd.cc: trivial resolution commit 7d044c6fbdcabc92d4dae7d9759dc63f87b2e4f4 Author: Kefu Chai Date: Sat Jun 10 23:58:47 2017 +0800 test/librados: extract functions using libcommon in test.cc into test_common.cc Fixes: http://tracker.ceph.com/issues/20175 Signed-off-by: Kefu Chai (cherry picked from commit 82a848c2053ea69ebc6d3ec1003e18921e2d08d2) Conflicts: src/test/librados/CMakeLists.txt: trivial resolution src/test/librados/test.cc: trivial resolution commit 53024570e75f2963da84f3a045f1ac1f9ee4081e Author: Yan, Zheng Date: Sat Apr 22 12:27:12 2017 +0800 mds: issue new caps when sending reply to client After Locker::issue_new_caps() adds new Capability data struct, do not issue caps immediately. Let CInode::encode_inodestate() do the job instead. This can avoid various races that early reply is not allowed, caps that haven't been sent to client gets revoked. Fixes: http://tracker.ceph.com/issues/19635 Signed-off-by: "Yan, Zheng" (cherry picked from commit 799703a4acb49db0b6cc99a23e4326767e694c3a) commit 18f1830e8acaa386d0113171ec649b5b33698dd9 Author: Jason Dillaman Date: Mon Jun 5 08:17:05 2017 -0400 librbd: filter expected error codes from is_exclusive_lock_owner Fixes: http://tracker.ceph.com/issues/20182 Signed-off-by: Jason Dillaman (cherry picked from commit d4daaf54e6bc42cd4fb2111ea20b2042941b0c31) commit 45b4c86452d19eaf3078704ba951e2fdb2e49a30 Author: Jason Dillaman Date: Mon Jun 5 13:17:19 2017 -0400 rbd: properly decode features when using image name optional Fixes: http://tracker.ceph.com/issues/20185 Signed-off-by: Jason Dillaman (cherry picked from commit f1b05a2b062a59ec1b6682f7683bfd816433a931) commit d463de9b8ae357a4edbfcf63e65740ecd4845b26 Author: fang.yuxiang Date: Tue Jun 13 16:40:16 2017 +0800 rgw: meta sync thread crash at RGWMetaSyncShardCR Fixes: http://tracker.ceph.com/issues/20251 Signed-off-by: fang yuxiang fang.yuxiang@eisoo.com (cherry picked from commit 45877d38fd9a385b2f8b13e90be94d784898b0b3) Conflicts: src/rgw/rgw_data_sync.cc - kraken RGWContinuousLeaseCR() has slightly different options src/rgw/rgw_sync.cc - kraken RGWContinuousLeaseCR() has slightly different options - added "override" qualifier to ~RGWFetchAllMetaCR() definition commit d52aeec0310d3f1dab986a43bee2f4fc768efc06 Author: Casey Bodley Date: Sat Jun 10 18:12:52 2017 -0400 rgw: only log metadata on metadata master zone Fixes: http://tracker.ceph.com/issues/20244 Signed-off-by: Casey Bodley (cherry picked from commit b8272f3607074a2f7cbfd08f7bbc82f22cf120ba) commit 9651ed95894707d6d5517979b18a8acdbcc1b004 Author: Casey Bodley Date: Fri Apr 21 15:04:48 2017 -0400 radosgw-admin: warn that 'realm rename' does not update other clusters Fixes: http://tracker.ceph.com/issues/19746 Signed-off-by: Casey Bodley (cherry picked from commit 8a459c768ffecd689a53a79dfe33eb8f1bbc318f) commit 4e4cd5453670af8d87b217c60dc84dd39e192952 Author: Casey Bodley Date: Tue Apr 4 10:42:44 2017 -0400 rgw: fix for zonegroup redirect url local dest_url variable was shadowing the one in the enclosing scope, so the changes were not applied and no Location header was written on redirect Fixes: http://tracker.ceph.com/issues/19488 Signed-off-by: Casey Bodley (cherry picked from commit 542e188a40f0495720b48308372366951ae41e62) commit 693d88081358f2059a65de5ddb4d35272ef1f3fe Author: Casey Bodley Date: Tue Apr 4 10:41:51 2017 -0400 rgw: use zonegroup's master zone endpoints for bucket redirect if no zonegroup endpoints are set, fall back to master zone endpoints Fixes: http://tracker.ceph.com/issues/19488 Signed-off-by: Casey Bodley (cherry picked from commit 92f63c6392bdc4633a2e57cb3867051bb1a3fd55) commit f23df08430c6b62314898a4c95f81d8cb01f93c3 Author: Casey Bodley Date: Wed Apr 5 14:20:20 2017 -0400 rgw: allow larger payload for period commit testing with 3 zonegroups and 3 zones each, the period json grew larger than 4k and caused decode failures on period commit updated to use the new config variable rgw_max_put_param_size Fixes: http://tracker.ceph.com/issues/19505 Signed-off-by: Casey Bodley (cherry picked from commit 7f2871fe59d933b03f37fde40f1781b2320d0d50) commit dea9a69c3f85504070a56f8a2dbf1a4c129b9ef4 Author: Abhishek Lekshmanan Date: Fri Mar 3 16:50:33 2017 +0100 rgw: make a configurable size for requests with xml params We currently read all user input for a few apis accepting xml, avoid doing this and error out early, most s3 apis limit to about 1000 xml entries, make this configurable via a new config param, `rgw_max_put_param_size` defaulting to 1MB. Also modified `rgw_rest_read_all_input` with an additional param to not allow for chunked uploads, though we error out in a better way by responding with content length required rather than S3's 501 if chunked is set. This also adds the same behavior in RGWPutCORS reusing `rgw_rest_read_all_input`, and using a data_deleter to manage the char* Signed-off-by: Abhishek Lekshmanan rgw: rgw_rest_s3: make PUT CORS accept only a configured max input Signed-off-by: Abhishek Lekshmanan (cherry picked from commit 35375b75160dc93beeb04c7fe010dc0d21952534) commit ac706f2b4a4aee7b813a0b5ba0589431b84f3dcd Author: Mykola Golub Date: Tue May 23 12:07:45 2017 +0200 librbd: potential read IO hang when image is flattened Fixes: http://tracker.ceph.com/issues/19832 Signed-off-by: Jason Dillaman (cherry picked from commit 10d58618e7c632ef01b9537492239e0a042dc17e) commit b27595ae9c3490a4ff1d39b0d864a4a47a941981 Author: Zhang Shaowen Date: Wed Jun 14 10:29:53 2017 +0800 rgw: lifecycle thread shouldn't process the bucket which has been deleted. Fixes: http://tracker.ceph.com/issues/20285 Signed-off-by: Zhang Shaowen (cherry picked from commit a2b042fe9f7e6503273fa4c2a4c56e399d7c338f) commit 3105e2327ea29558fdd97dfb6da2c7ee9ec08f8f Author: Zhang Shaowen Date: Tue May 9 16:39:39 2017 +0800 rgw: VersionIdMarker and NextVersionIdMarker should be returned when listing object versions if necessary. Fixes: http://tracker.ceph.com/issues/19886 Signed-off-by: Zhang Shaowen (cherry picked from commit f805c3e08948e379b7d2c4f2faf9e7f550e4cb23) commit d0e742cafae325cf794a2160b6df19296ac398a4 Author: fang.yuxiang Date: Thu May 4 15:58:37 2017 +0800 rgw: set object accounted size correctly sometimes, object accounted size is set wrong, because we don't konw the object size if don't resort to the compression info or manifest. e.g, when i use s3cmd do copy object(bucket_A/obj_A -> bucket_B/obj_B, assume the size of obj_A is 4M). then i use s3cmd do list bucket, I got obj_B size is 512K, it is the head size apparently. Fixes: http://tracker.ceph.com/issues/20071 Signed-off-by: fang yuxiang (cherry picked from commit 539985a99eebdc72c8d2446acc1108664a162f68) commit 640a7a26296e652ffb8b8bfab87839d7bfd66d1b Author: fang.yuxiang Date: Sat May 27 15:20:30 2017 +0800 rgw: get wrong content when download object with specific range when compression was enabled look at the prototype: RGWGetObj_Decompress::handle_data(bufferlist& bl, off_t bl_ofs, off_t bl_len) we should trim the bl using bl_ofs and bl_len. Fixes: http://tracker.ceph.com/issues/20100 Signed-off-by: fang yuxiang (cherry picked from commit afe9d99570059b88837690950a4f4525e0e60262) commit 6c3daeb2970f52970145eb31d4bef56b33df8b40 Merge: 08b8a736e2 bd9aec00ec Author: Nathan Cutler Date: Thu Jul 6 17:46:06 2017 +0200 Merge pull request #16113 from smithfarm/wip-19807-kraken kraken: tests: remove hard-coded image name from TestLibRBD.Mirror Reviewed-by: Jason Dillaman Reviewed-by: Mykola Golub commit 1fc14857a9dd81b1b4726428f115ef5cfb495781 Author: Zhang Shaowen Date: Tue Jun 6 15:43:20 2017 +0800 rgw: datalog trim and mdlog trim handles the result returned by osd incorrectly. Fixes: http://tracker.ceph.com/issues/20190 Signed-off-by: Zhang Shaowen (cherry picked from commit 7fd6e031e5b0b1f3eca70c5b459d50f6f214171f) commit 90288afc10d878b59dacb99383ada309377e4bb4 Author: Pritha Srivastava Date: Mon Mar 20 11:41:48 2017 +0530 rgw: Added code to correctly account for bytes sent/ received during a 'PUT' operation. Currently, the bytes sent/ received are both set to zero after an object is uploaded to a bucket. Added code to correct the logic. Signed-off-by: Pritha Srivastava (cherry picked from commit 85735c972f5db1a110f55e01a4f6249202152553) commit dda4d912cd753e1443beac4e7f92d97f24b57992 Author: redickwang Date: Fri May 19 15:08:12 2017 +0800 rgw: segment fault when shard id out of range Fixes: http://tracker.ceph.com/issues/19732 Signed-off-by: redickwang (cherry picked from commit ff4c40fc2e3c092d17a218ae6132de4e6034c8ee) commit f32e26e909c3d56160e9a33d051d690fa4366bbd Author: Radoslaw Zarzynski Date: Thu Apr 6 17:08:23 2017 +0200 rgw: fix broken /crossdomain.xml, /info and /healthcheck of Swift API. Fixes: http://tracker.ceph.com/issues/19520 Signed-off-by: Radoslaw Zarzynski (cherry picked from commit fe2afd42aaf706d38d6ed611796a181ae9e02ae8) commit ccb33bab37e45bf47dad5577602e9be9d1d683c6 Author: Abhishek Lekshmanan Date: Tue Apr 25 14:46:09 2017 +0200 rgw: use a vector for options passed to civetweb Since the array we used needs additional check to ensure that the size is correct, and causes undefined behaviour in a few platforms, using a vector and passing the c array back to mg_start so that we don't go past the end of array. Fixes: http://tracker.ceph.com/issues/19749 Signed-off-by: Abhishek Lekshmanan Signed-off-by: Jesse Williamson (cherry picked from commit 3959a8b52c2910e4060f7d273c8939bbdc83e48a) commit f8235c5c0a64eb409957d80d4c4a22a87280d0b2 Author: Willem Jan Withagen Date: Sat Apr 29 13:36:07 2017 +0200 cls/log/cls_log.cc: reduce logging noise - The other reference in the source as already at 20. ./src/cls/timeindex/cls_timeindex.cc:85: CLS_LOG(20, "storing entry at %s", index.c_str()); And we need not always know where in the log items are stored. So it looks like a leftover debug feature. Fixes: http://tracker.ceph.com/issues/19835 Signed-off-by: Willem Jan Withagen (cherry picked from commit d76010900bf9012f2e66335787710531772766b7) commit e773b304eefa3d2ca7c1fe0817c89082bf574a38 Author: Marcus Watts Date: Thu Apr 13 05:33:55 2017 -0400 rgw: swift: disable revocation thread if sleep == 0 || cache_size == 0 Keystone tokens can be revoked. This causes them to fail validation. However, in ceph, we cache them. As long as they're in the cache we trust them. To find revoked tokens there's a call OSI-PKI/revoked but that's only useful for pki tokens. Installations using fernet/uuid may not even have the proper credentials to support the call, in which case the call blows up in various ways filling up logs with complaints. This code makes the revocation thread optional; by disabling it, the complaints go away. A further fix is in the works to use other more modern calls available in modern keystone installations to properly deal with non-PKI/PKIZ tokens. To disable the revocation thread, use at least one of these: rgw_keystone_token_cache_size = 0 using this will cause tokens to be validated on every call. You may instead want to set rgw_keystone_revocation_interval = 0 using just this will disable the revocation thread, but leaves the cache in use. That avoids the extra validation overhead, but means token revocation won't work very well. Fixes: http://tracker.ceph.com/issues/9493 Fixes: http://tracker.ceph.com/issues/19499 Signed-off-by: Marcus Watts (cherry picked from commit 003291a8cbca455c0e8731f66759395a0bb1f555) commit 68a853a14d89261fcd0a132a1a7a7151721b84c8 Author: lvshuhua Date: Wed Dec 7 15:47:47 2016 +0800 rgw: fix versioned bucket data sync fail when upload is busy Fixes: http://tracker.ceph.com/issues/18208 Signed-off-by: lvshuhua (cherry picked from commit ce7d00ac1642d84c1d9111156a544c37801c5adf) commit 9a7a73edf1f432df98984ab7ad60306aa6aca6e3 Author: liuchang0812 Date: Fri Feb 10 18:02:03 2017 +0800 rgw: don't return skew time in pre-signed url Fixes: http://tracker.ceph.com/issues/18828 Signed-off-by: liuchang0812 (cherry picked from commit dd8b348f4aad0124e8a4457117bf3f5f76af7bdb) commit 08b8a736e2b49bba39cf8fe51fe3435a7d381e72 Merge: f45de0f39d 405503472e Author: Nathan Cutler Date: Thu Jul 6 09:28:26 2017 +0200 Merge pull request #16132 from smithfarm/wip-20522-kraken kraken: FAILED assert(object_contexts.empty()) (live on master only from Jan-Feb 2017, all other instances are different) Reviewed-by: Josh Durgin commit f45de0f39db316219207d674198ecc578ed76f7e Merge: 4ca8f1f4a9 ff19977fbc Author: Nathan Cutler Date: Thu Jul 6 09:27:37 2017 +0200 Merge pull request #15962 from asheplyakov/20443-bp-kraken kraken: osd: unlock sdata_op_ordering_lock with sdata_lock hold to avoid missing wakeup signal Reviewed-by: Josh Durgin commit 4ca8f1f4a9c5cdb15bd273f74df7ac6cda218f66 Merge: d45c64d1b9 2aec591cfe Author: Nathan Cutler Date: Thu Jul 6 09:26:40 2017 +0200 Merge pull request #15066 from asheplyakov/19916-bp-kraken kraken: osd: osd/PrimaryLogPG: do not call on_shutdown() if (pg.deleting) Reviewed-by: Josh Durgin commit d45c64d1b94fb6a91c78e8333f924a2be08fcddb Merge: 4890ba0df0 62bb2086af Author: Nathan Cutler Date: Thu Jul 6 09:24:26 2017 +0200 Merge pull request #14942 from shinobu-x/wip-18293-kraken kraken: osd: leaked MOSDMap Reviewed-by: Sage Weil Reviewed-by: Josh Durgin commit 4890ba0df02403b8bc5348235d1aadb76ab8f4c1 Merge: c45570b4a6 d5aea7a68b Author: Nathan Cutler Date: Thu Jul 6 09:20:56 2017 +0200 Merge pull request #14736 from smithfarm/wip-19326-kraken kraken: bluestore bdev: flush no-op optimization is racy Reviewed-by: Josh Durgin commit c45570b4a61243839e7d4e64c5e1f1b017bb3948 Merge: 5a1116c0d5 e76725f4e9 Author: Nathan Cutler Date: Thu Jul 6 09:19:50 2017 +0200 Merge pull request #14732 from smithfarm/wip-19560-kraken kraken: objecter: full_try behavior not consistent with osd Reviewed-by: Josh Durgin commit 5a1116c0d5c1fa8810031104cd093ed1b1e8cb3a Merge: 7133d4db9b 1bc9cfff42 Author: Nathan Cutler Date: Thu Jul 6 09:18:24 2017 +0200 Merge pull request #13542 from shinobu-x/wip-18929-kraken kraken: osd: bogus assert when checking acting set on recovery completion in rados/upgrade Reviewed-by: Josh Durgin commit 7133d4db9b31b0627c6278f2f72b690ea366551e Merge: 6db580b6cf 56ca3b0190 Author: Nathan Cutler Date: Thu Jul 6 09:11:09 2017 +0200 Merge pull request #14939 from rzarzynski/wip-rgw-19754-kraken kraken: rgw: fix RadosGW hang during multi-chunk upload of AWSv4 Reviewed-by: Matt Benjamin commit 6db580b6cfdf3ef14403adbbcd27a2f16cbf15fc Merge: be8b7610d1 59391c40db Author: Nathan Cutler Date: Thu Jul 6 08:54:16 2017 +0200 Merge pull request #14530 from smithfarm/wip-19574-kraken kraken: rgw: unsafe access in RGWListBucket_ObjStore_SWIFT::send_response() Reviewed-by: Yehuda Sadeh commit be8b7610d164bf2798bb4b9676db330ef5fb6c27 Merge: e12eae92b0 1f86be6e19 Author: Nathan Cutler Date: Thu Jul 6 08:40:16 2017 +0200 Merge pull request #13868 from smithfarm/wip-19160-kraken kraken: rgw: multisite: RGWMetaSyncShardControlCR gives up on EIO Reviewed-by: Orit Wasserman commit 85cefa113b31c9dc6b8a7a88e3343de923278c7d Author: Loic Dachary Date: Sat Mar 18 10:04:20 2017 +0100 mon: osd crush set crushmap need sanity check The sanity check verifying the new crushmap does not remove crush rules that are in use is not exclusive to ceph setcrushmap. Fixes: http://tracker.ceph.com/issues/19302 Signed-off-by: Loic Dachary (cherry picked from commit ed760457bf154c10adf75c6df046eecab7eb8e4b) commit 9a26882289e1746fe8ca1443666ed0a583e07021 Author: Sage Weil Date: Tue Jun 6 15:13:50 2017 -0400 ceph_test_rados_api_misc: fix LibRadosMiscConnectFailure.ConnectFailure retry Fixes: http://tracker.ceph.com/issues/19901 Signed-off-by: Sage Weil (cherry picked from commit 087dff80cac707ee9bcb5bcfc98cb3ec047bd49f) commit 91569f63852f12d958175bf970967fa9d25b2cfc Author: lu.shasha Date: Thu Jan 5 11:50:42 2017 +0800 rgw: fix 'gc list --include-all' command infinite loop the first 1000 items When the items to gc over 1000, 'gc list --include-all' command will infinite loop the first 1000 items. Add next_marker to move to the next 1000 items. Fixes: http://tracker.ceph.com/issues/19978 Signed-off-by: fang yuxiang Signed-off-by: Shasha Lu (cherry picked from commit fc29f52ebca63104a05515484088ff136dfb0b15) commit ea186807c3fb7ac79f8ef1a8ae874ced69aec6e1 Author: Erwan Velu Date: Fri Mar 31 14:54:33 2017 +0200 ceph-disk: Adding retry loop in get_partition_dev() There is very rare cases where get_partition_dev() is called before the actual partition is available in /sys/block/. It appear that waiting a very short is usually enough to get the partition beein populated. Analysis: update_partition() is supposed to be enough to avoid any racing between events sent by parted/sgdisk/partprobe and the actual creation on the /sys/block//* entrypoint. On our CI that race occurs pretty often but trying to reproduce it locally never been possible. This patch is almost a workaround rather than a fix to the real problem. It offer retrying after a very short to be make a chance the device to appear. This approach have been succesful on the CI. Note his patch is not changing the timing when the device is perfectly created on time and just differ by a 1/5th up to 2 seconds when the bug occurs. A typical output from the build running on a CI with that code. command_check_call: Running command: /usr/bin/udevadm settle --timeout=600 get_dm_uuid: get_dm_uuid /dev/sda uuid path is /sys/dev/block/8:0/dm/uuid get_partition_dev: Try 1/10 : partition 2 for /dev/sda does not in /sys/block/sda get_partition_dev: Found partition 2 for /dev/sda after 1 tries get_dm_uuid: get_dm_uuid /dev/sda uuid path is /sys/dev/block/8:0/dm/uuid get_dm_uuid: get_dm_uuid /dev/sda2 uuid path is /sys/dev/block/8:2/dm/uuid fixes: #19428 Signed-off-by: Erwan Velu (cherry picked from commit 93e7b95ed8b4c78daebf7866bb1f0826d7199075) commit f7f6375b54233f1085ba8d64df1053c038ff3991 Author: Erwan Velu Date: Wed Mar 22 10:11:44 2017 +0100 ceph-disk: Reporting /sys directory in get_partition_dev() When get_partition_dev() fails, it reports the following message : ceph_disk.main.Error: Error: partition 2 for /dev/sdb does not appear to exist The code search for a directory inside the /sys/block/get_dev_name(os.path.realpath(dev)). The issue here is the error message doesn't report that path when failing while it might be involved in. This patch is about reporting where the code was looking at when trying to estimate if the partition was available. Signed-off-by: Erwan Velu (cherry picked from commit 413c9fcfbe8e6ab33d73b8428090ccacc33c5d15) commit 533ff8a540b0b30378a9562953f1da46291e9832 Author: Kefu Chai Date: Wed May 3 18:28:01 2017 +0800 qa/tasks/ceph_manager: always fix pgp_num when done with thrashosd task Fixes: http://tracker.ceph.com/issues/19771 Signed-off-by: Kefu Chai (cherry picked from commit da1161cbd8b50b8a980e8a3b48154a97f988426c) commit f04eb66d365c088e0e6985864d54bdc4fc3d3bfd Author: Loic Dachary Date: Tue May 9 12:32:51 2017 +0200 ceph-disk: separate ceph-osd --check-needs-* logs It is using the OSD id zero but have nothing to do with OSD zero and this is confusing to the user. The log themselves do not need to be kept around and are stored in the run directory so that they can be disposed of after reboot. Fixes: http://tracker.ceph.com/issues/19888 Signed-off-by: Loic Dachary (cherry picked from commit c7b3c46bd63b78475868e405bf20d9c142f0336a) commit a7af766aa89b9435c72cdb93e5d67a9e9331c635 Author: Sage Weil Date: Thu Dec 22 17:18:08 2016 -0500 msg/simple: clear_pipe when wait() is mopping up pipes When wait is mopping up connections it may hit one that is in the process of accepting. It will unregister it whilst the accept() thread is trying to set it up, aborting the accept and getting it reaped. However, the pipe mop-up does not clear_pipe() the way that mark_down(), mark_down_all(), and fault() do, which leads to this assert. Pipe is accepting... -161> 2016-12-22 17:31:45.460613 37353700 10 -- 172.21.15.14:6804/20738 >> 172.21.15.35:0/146098963 pipe(0x3e2a5c20 sd=31 :6804 s=0 pgs=0 cs=0 l=1 c=0x3e2a6f40).accept: setting up session_security. -160> 2016-12-22 17:31:45.460733 37353700 10 -- 172.21.15.14:6804/20738 >> 172.21.15.35:0/146098963 pipe(0x3e2a5c20 sd=31 :6804 s=0 pgs=0 cs=0 l=1 c=0x3e2a6f40).accept new session -159> 2016-12-22 17:31:45.460846 37353700 10 -- 172.21.15.14:6804/20738 >> 172.21.15.35:0/146098963 pipe(0x3e2a5c20 sd=31 :6804 s=2 pgs=7 cs=1 l=1 c=0x3e2a6f40).accept success, connect_seq = 1, sending READY -158> 2016-12-22 17:31:45.460959 37353700 10 -- 172.21.15.14:6804/20738 >> 172.21.15.35:0/146098963 pipe(0x3e2a5c20 sd=31 :6804 s=2 pgs=7 cs=1 l=1 c=0x3e2a6f40).accept features 1152921504336314367 wait() is shutting down... -156> 2016-12-22 17:31:45.461882 9506ac0 20 -- 172.21.15.14:6804/20738 wait: stopping accepter thread -155> 2016-12-22 17:31:45.462994 9506ac0 10 accepter.stop accept listening on: 15 ... -116> 2016-12-22 17:31:45.482137 9506ac0 10 -- 172.21.15.14:6804/20738 wait: closing pipes -115> 2016-12-22 17:31:45.482850 9506ac0 10 -- 172.21.15.14:6804/20738 >> 172.21.15.35:0/146098963 pipe(0x3e2a5c20 sd=31 :6804 s=2 pgs=7 cs=1 l=1 c=0x3e2a6f40).unregister_pipe -114> 2016-12-22 17:31:45.483421 9506ac0 10 -- 172.21.15.14:6804/20738 >> 172.21.15.35:0/146098963 pipe(0x3e2a5c20 sd=31 :6804 s=2 pgs=7 cs=1 l=1 c=0x3e2a6f40).stop ...which interrupts the accept()... -113> 2016-12-22 17:31:45.484164 37353700 10 -- 172.21.15.14:6804/20738 >> 172.21.15.35:0/146098963 pipe(0x3e2a5c20 sd=31 :6804 s=4 pgs=7 cs=1 l=1 c=0x3e2a6f40).accept fault after register and makes accept() return failure, and reader() to exit and reap... -110> 2016-12-22 17:31:45.486103 9506ac0 10 -- 172.21.15.14:6804/20738 wait: waiting for pipes 0x3e2a5c20 to close -109> 2016-12-22 17:31:45.487146 37353700 10 -- 172.21.15.14:6804/20738 queue_reap 0x3e2a5c20 -108> 2016-12-22 17:31:45.487658 9506ac0 10 -- 172.21.15.14:6804/20738 reaper -107> 2016-12-22 17:31:45.487722 9506ac0 10 -- 172.21.15.14:6804/20738 reaper reaping pipe 0x3e2a5c20 172.21.15.35:0/146098963 -106> 2016-12-22 17:31:45.487816 9506ac0 10 -- 172.21.15.14:6804/20738 >> 172.21.15.35:0/146098963 pipe(0x3e2a5c20 sd=31 :6804 s=4 pgs=7 cs=1 l=1 c=0x3e2a6f40).discard_queue -105> 2016-12-22 17:31:45.494742 37353700 10 -- 172.21.15.14:6804/20738 >> 172.21.15.35:0/146098963 pipe(0x3e2a5c20 sd=31 :6804 s=4 pgs=7 cs=1 l=1 c=0x3e2a6f40).reader done ... -92> 2016-12-22 17:31:45.527589 9506ac0 -1 /mnt/jenkins/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/11.1.0-6151-ge1781dd/rpm/el7/BUILD/ceph-11.1.0-6151-ge1781dd/src/msg/simple/SimpleMessenger.cc: In function 'void SimpleMessenger::reaper()' thread 9506ac0 time 2016-12-22 17:31:45.488264 /mnt/jenkins/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos7/DIST/centos7/MACHINE_SIZE/huge/release/11.1.0-6151-ge1781dd/rpm/el7/BUILD/ceph-11.1.0-6151-ge1781dd/src/msg/simple/SimpleMessenger.cc: 235: FAILED assert(!cleared) Fixes: http://tracker.ceph.com/issues/15784 Signed-off-by: Sage Weil (cherry picked from commit 948f97b3bdd39269a38277238a61f24e5fec6196) commit 6d2f95973561c9f0911fce04dff2036f73a77442 Author: Jason Dillaman Date: Mon Nov 28 13:36:01 2016 -0500 test: added missing IoCtx copy/assignment methods in librados_test_stub Signed-off-by: Jason Dillaman (cherry picked from commit c35d307c62961072b4c00f94e37486e3371c1a21) commit 405503472ea6b3a2965455c8c064dc0e4d713ad4 Author: Samuel Just Date: Tue Feb 14 12:47:37 2017 -0800 ReplicatedBackend: don't queue Context outside of ObjectStore with obc We only flush the ObjectStore callbacks, not everything else. Thus, there isn't a guarrantee that the obc held by pull_complete_info will be cleaned up before the Flush callback is triggered. Instead, just defer clearing the pull state until the callback (it'll be cleaned up during the interval change) and remove the ObjectContext from pull_complete_info. Introduced: 68defc2b0561414711d4dd0a76bc5d0f46f8a3f8 Fixes: http://tracker.ceph.com/issues/18927 Signed-off-by: Samuel Just (cherry picked from commit 51eee55c475a3d931844831e040ed3d66ee59af4) commit 5257d1a643fb1af36b80650ea2b014b4887dc10f Author: Samuel Just Date: Fri Feb 3 13:12:47 2017 -0800 osd/: don't leak context for Blessed*Context or RecoveryQueueAsync This has always been a bug, but until 68defc2b0561414711d4dd0a76bc5d0f46f8a3f8, nothing deleted those contexts without calling complete(). Fixes: http://tracker.ceph.com/issues/18809 Bug shadowed until: 68defc2b0561414711d4dd0a76bc5d0f46f8a3f8 Signed-off-by: Samuel Just (cherry picked from commit 91b74235027c8a4872dcab6b37767b12c3267061) commit e12eae92b04d30d322740a6c7a33d86e448bf9c5 Merge: 7bdc3ef238 73fe367bc4 Author: Nathan Cutler Date: Wed Jul 5 22:04:41 2017 +0200 Merge pull request #15792 from liewegas/wip-bluestore-leaks-kraken kraken: os/bluestore: deep decode onode value Reviewed-by: Josh Durgin commit 7bdc3ef238228bb156f8f97867a66138fcc93807 Merge: 0a71e1899b 27e9644ceb Author: Nathan Cutler Date: Wed Jul 5 22:01:58 2017 +0200 Merge pull request #15729 from asheplyakov/20315-bp-kraken kraken: mon: fail to form large quorum; msg/async busy loop Reviewed-by: Josh Durgin commit 0a71e1899bd8ee84640059e18126461d3a1748a3 Merge: d7598df528 d38a752999 Author: Nathan Cutler Date: Wed Jul 5 22:00:17 2017 +0200 Merge pull request #15084 from asheplyakov/19928-bp-kraken kraken: mon crash on shutdown, lease_ack_timeout event Reviewed-by: Josh Durgin commit d7598df5281e1648c5890a4b18b7b64d74b8dc95 Merge: 0dbbd5001e 47a4544176 Author: Nathan Cutler Date: Wed Jul 5 21:59:03 2017 +0200 Merge pull request #14735 from smithfarm/wip-19685-kraken kraken: osd: Give requested scrubs a higher priority Reviewed-by: Josh Durgin commit 0dbbd5001ea84906d15bb27c99b73493d20eab75 Merge: 0d177d5c99 7a275769b6 Author: Nathan Cutler Date: Wed Jul 5 21:57:53 2017 +0200 Merge pull request #14733 from smithfarm/wip-19561-kraken kraken: api_misc: [ FAILED ] LibRadosMiscConnectFailure.ConnectFailure Reviewed-by: Josh Durgin commit 0d177d5c992ca8cd99afa8e9be3f429a933d490a Merge: 11ed483791 0227920438 Author: Nathan Cutler Date: Wed Jul 5 21:51:26 2017 +0200 Merge pull request #13883 from shinobu-x/wip-19119-kraken kraken: pre-jewel "osd rm" incrementals are misinterpreted Reviewed-by: Josh Durgin commit 11ed483791849905910c879152d1eea41d95604e Merge: 74b0c9da46 fa2753245d Author: Nathan Cutler Date: Wed Jul 5 19:39:54 2017 +0200 Merge pull request #14646 from gregsfortytwo/wip-17916-kraken kraken: osd: pglog: with config, don't assert in the presence of stale diverg… Reviewed-by: Josh Durgin commit 74b0c9da461825ef53ddf79ec00d08168e33025c Merge: 7a87735243 648c6adf52 Author: Nathan Cutler Date: Wed Jul 5 18:41:59 2017 +0200 Merge pull request #15237 from asheplyakov/20035-bp-kraken kraken: mon/PGMap: factor mon_osd_full_ratio into MAX AVAIL calc Reviewed-by: Josh Durgin commit 7a877352430e49f94dff614dc0a697a4697f6a77 Merge: 4a7c26083f 5c7a07677a Author: Nathan Cutler Date: Wed Jul 5 18:19:05 2017 +0200 Merge pull request #14331 from shinobu-x/wip-19396-kraken kraken: Objecter::epoch_barrier isn't respected in _op_submit() Reviewed-by: Josh Durgin commit 4a7c26083feebca273044691c7f7ef0dbf8305eb Merge: f978b2514c 071f111abb Author: Nathan Cutler Date: Wed Jul 5 17:54:30 2017 +0200 Merge pull request #16083 from smithfarm/wip-18677-kraken kraken: osd: metadata reports filestore when using bluestore Reviewed-by: Josh Durgin commit f978b2514cb17dabd341b5c801bd541a489071a7 Merge: 56c65b120f 7054ff69d0 Author: Nathan Cutler Date: Wed Jul 5 17:51:36 2017 +0200 Merge pull request #16098 from smithfarm/wip-19621-kraken kraken: rbd-nbd: add signal handler Reviewed-by: Jason Dillaman commit 56c65b120f1ef7323109889c6a82287e5e4f5371 Merge: e13f5467fa 27cbff1114 Author: Nathan Cutler Date: Wed Jul 5 17:50:21 2017 +0200 Merge pull request #15612 from smithfarm/wip-19611-kraken kraken: rbd: Issues with C API image metadata retrieval functions Reviewed-by: Jason Dillaman Reviewed-by: Mykola Golub commit e13f5467fac1e296d2d56b8ed4dc89d5cdfa0f5d Merge: 19deb31d39 b19d6eb311 Author: Nathan Cutler Date: Wed Jul 5 17:48:24 2017 +0200 Merge pull request #14833 from smithfarm/wip-19794-kraken kraken: tests: test_notify.py: assert(not image.is_exclusive_lock_owner()) on line 147 Reviewed-by: Jason Dillaman commit 19deb31d399715716b02c8fdf58ea2845b42d58c Merge: cc5241e4aa c79d45ff52 Author: Nathan Cutler Date: Wed Jul 5 17:26:58 2017 +0200 Merge pull request #14543 from smithfarm/wip-19173-kraken kraken: rbd: rbd_clone_copy_on_read ineffective with exclusive-lock Reviewed-by: Jason Dillaman commit cc5241e4aab227b8b272774bb5d4ff1f37ed3446 Merge: e705528331 9efeb6bc2d Author: Nathan Cutler Date: Wed Jul 5 09:37:01 2017 +0200 Merge pull request #14974 from dillaman/wip-19872-kraken kraken: rbd-mirror: failover and failback of unmodified image results in split-brain Reviewed-by: Mykola Golub commit e70552833137eed1f9ede6c3eb59d345a50ac9ce Merge: 2b73b570e0 d51b755391 Author: Nathan Cutler Date: Wed Jul 5 09:36:14 2017 +0200 Merge pull request #14934 from dillaman/wip-19833-kraken kraken: cls_rbd: default initialize snapshot namespace for legacy clients Reviewed-by: Mykola Golub commit 2b73b570e060d2c77b6c86579cecc84221b166f9 Merge: dbe8a8ba84 e0f90f03dd Author: Nathan Cutler Date: Wed Jul 5 09:30:07 2017 +0200 Merge pull request #14539 from smithfarm/wip-18771-kraken kraken: rbd: Improve compatibility between librbd + krbd for the data pool Reviewed-by: Jason Dillaman commit baa772372c52ca2f5b27bad59798734def8656f1 Author: Yan, Zheng Date: Fri May 12 10:38:51 2017 +0800 pybind: fix cephfs.OSError initialization Traceback (most recent call last): File "", line 1, in File "cephfs.pyx", line 672, in cephfs.LibCephFS.open (/home/zhyan/Ceph/ceph-2/build/src/pybind/cephfs/pyrex/cephfs.c:10160) File "cephfs.pyx", line 155, in cephfs.OSError.__init__ (/home/zhyan/Ceph/ceph-2/build/src/pybind/cephfs/pyrex/cephfs.c:1889) TypeError: __init__() takes exactly 3 positional arguments (2 given) Signed-off-by: "Yan, Zheng" (cherry picked from commit e6493f64ba4592b8dca54ece4464efa6c7f331a7) commit ac9aed9c2c46bd64384ae047f461b990cb3f6ffe Author: Yan, Zheng Date: Wed May 10 08:13:52 2017 +0800 pybind: fix open flags calculation (O_WRONLY | O_RDWR) is invaild open flags Fixes: http://tracker.ceph.com/issues/19890 Signed-off-by: "Yan, Zheng" (cherry picked from commit 2c25c99cb4572ffae97555a56b24a4c4097dcdec) commit bd9aec00ec284e2e828625b5e92165465aee38b6 Author: Jason Dillaman Date: Thu Apr 27 16:21:33 2017 -0400 test: remove hard-coded image name from RBD metadata test Fixes: http://tracker.ceph.com/issues/19798 Signed-off-by: Jason Dillaman (cherry picked from commit 8f72e745e157cc12d76ca6babe956c5698ee297f) commit 474d504dd67378588d2dd243b06b844737ab64d5 Author: Mykola Golub Date: Thu Mar 2 17:18:18 2017 +0100 librbd: relax "is parent mirrored" check when enabling mirroring for pool If the parent is in the same pool and has the journaling feature enabled we can assume the mirroring will eventually be enabled for it. Fixes: http://tracker.ceph.com/issues/19130 Signed-off-by: Mykola Golub (cherry picked from commit fe31bca22f90ce02f461d6421a4f66539db888d3) commit 44c116dfd6f519538b6c1401e3c3bc283d31b78a Author: Mykola Golub Date: Mon Jan 9 10:40:46 2017 +0100 rbd-mirror: check image mirroring state when bootstrapping Fixes: http://tracker.ceph.com/issues/18447 Signed-off-by: Mykola Golub (cherry picked from commit 5fc5a8ac895524f05eed6e7db20b0dda3a8cb60f) commit d5fa6a65b52c3f2cf5363ebb55af7e96d8be90ab Author: Mykola Golub Date: Mon Jan 9 09:23:19 2017 +0100 rbd-mirror: async request to test if image is primary Signed-off-by: Mykola Golub (cherry picked from commit 0a1cb35caacdf85029f31a0364dc07a5d7462f5f) commit 4035dc6fdaf9df06ffeb8b86813046a1b5e0cff2 Author: Mykola Golub Date: Mon Jan 9 15:02:02 2017 +0100 rbd-mirror: hold owner lock when testing if lock owner Signed-off-by: Mykola Golub (cherry picked from commit 82aa89668d666f434cd19ff444223017b5512c6a) commit 76fedab57612ea0e15d18729e779ec43bf830704 Author: xie xingguo Date: Tue Dec 27 19:39:51 2016 +0800 os/bluestore: fix OnodeSizeTracking testing [ FAILED ] 1 test, listed below: [ FAILED ] ObjectStore/StoreTest.OnodeSizeTracking/2, where GetParam() = "bluestore" 1 FAILED TEST The above test failure happens as the bluestore mount() process will try to load all collections and put them into the coll_map, which will be also tracked as mempool::bluestore_meta_other. So total_bytes from mempool won't be equal to zero. Signed-off-by: xie xingguo (cherry picked from commit 1d97862f716ce24bfe227b192ecc5e1eb3f76757) commit 19c7524ace7aeb4f1bc7986162191f4a88bcb4d2 Author: Sage Weil Date: Fri Jun 23 11:12:01 2017 -0400 qa/tasks/radosbench: increase timeout The current timeout isn't enough in some cases (powercycle thrashing leaves osds down for a long time because rebooting is so slow). Signed-off-by: Sage Weil (cherry picked from commit dfa95db57d2c73923918ecf7014c801a925ddfb7) commit fa88fc77b4df0c548cda8783334862e33867abac Author: Jason Dillaman Date: Wed Mar 15 14:49:13 2017 -0400 librbd: image create validates that pool supports overwrites Fixes: http://tracker.ceph.com/issues/19081 Signed-off-by: Jason Dillaman (cherry picked from commit a16beba7843539bef4e5035148253a7b594a5cfd) Conflicts: src/librbd/image/CreateRequest.cc - kraken uses create_rados_ack_callback; in master, this has been renamed ceph_rados_callback commit d5aea7a68b8b2603cb9fa57df3a3ae5a44ec450a Author: Sage Weil Date: Fri Mar 24 12:15:50 2017 -0400 os/bluestore/KernelDevice: fix uninit value Signed-off-by: Sage Weil (cherry picked from commit 42db0c70bc7ef595f0925657c043ce081799b2b9) Conflicts: src/os/bluestore/KernelDevice.h - flush_lock mutex is still used in kraken, so keep it commit c08720553f6ae787fe3b0edbdd1497859cdfe0d4 Author: Sage Weil Date: Thu Mar 9 16:51:21 2017 -0500 os/bluestore/BlueFS: fix flush_bdev placement We need to flush any new writes on any fsync(). Notably, this includes the rocksdb log. However, previously _fsync was only doing a bdev flush if we also had a dirty bluefs journal and called into _sync_and_flush_journal. If we didn't, we weren't doing a flush() at all, which could lead to corrupted data. Fix this by moving the first flush_bdev *out* of _sync_and_flush_log. (The second one is there to flush the bluefs journal; the first one was to ensure prior writes are stable.) Instead, flush prior writes in all of the callers prior to calling _sync_and_flush_log. This includes _fsync (and fixes the bug by covering the non-journal-flush path) as well as several other callers. Fixes: http://tracker.ceph.com/issues/19250 Signed-off-by: Sage Weil (cherry picked from commit 2924a96493d8570317e55854a25fc64911ecf151) commit 594332631e442f3da1b7e6aa49248a6e3c46a30f Author: Sage Weil Date: Thu Mar 9 16:51:05 2017 -0500 os/bluestore/KernelDevice: make flush() thread safe flush() may be called from multiple racing threads (notably, rocksdb can call fsync via bluefs at any time), and we need to make sure that if one thread sees the io_since_flush command and does an actual flush, that other racing threads also wait until that flush is complete. This is accomplished with a simple mutex! Also, set the flag on IO *completion*, since flush is only a promise about completed IOs, not submitted IOs. Document. Fixes: http://tracker.ceph.com/issues/19251 Signed-off-by: Sage Weil (cherry picked from commit 6b3c52643c8e5fa820c53d96608862b7649c3fd0) commit dbe8a8ba848d5567db935d7cc945fdd85d8e9d3f Merge: ccde4da15c 5dcf60702c Author: Nathan Cutler Date: Tue Jul 4 13:32:46 2017 +0200 Merge pull request #14065 from asheplyakov/19212-bp-kraken kraken: rgw: "cluster [WRN] bad locator @X on object @X...." in cluster log Reviewed-by: Orit Wasserman commit b941ca7eac3090bf219b6ced4a9239e315f663de Author: John Spray Date: Wed Mar 15 19:36:08 2017 +0000 qa/cephfs: use getfattr/setfattr helpers Signed-off-by: John Spray (cherry picked from commit dd43d3bc646aeab88486b0963fc83de0b18800c4) Conflicts: qa/tasks/cephfs/test_data_scan.py - the master commit drops p = self._mount.run_shell(...) assignment which isn't present in kraken (trivial resolution) commit f8843445aa1b027c7a2ab5729b715d0da616bfb7 Author: John Spray Date: Wed Mar 15 19:26:30 2017 +0000 qa: add test for reading quotas from different clients Fixes: http://tracker.ceph.com/issues/17939 Signed-off-by: John Spray (cherry picked from commit 61617f8f10a6322603a9add77980865cd972ef97) commit b7f46973b03bc9e113ccd55ede0f766a1b1eb0d3 Author: John Spray Date: Wed Mar 15 17:51:44 2017 +0000 client: _getattr on quota_root before using in statfs ...so that after someone adjusts the quota settings on an inode that another client is using as its mount root, the change is visible immediately on the other client. Signed-off-by: John Spray (cherry picked from commit 3d25941aadd223669448d0f5d3c0bd1fefa72308) commit 5a69e33057982d78157b75bce3fc458e03260d9e Author: John Spray Date: Wed Mar 15 15:32:47 2017 +0000 client: getattr before read on ceph.* xattrs Previously we were returning values for quota, layout xattrs without any kind of update -- the user just got whatever happened to be in our cache. Clearly this extra round trip has a cost, but reads of these xattrs are fairly rare, happening on admin intervention rather than in normal operation. Fixes: http://tracker.ceph.com/issues/17939 Signed-off-by: John Spray (cherry picked from commit 532dc4b68e538c189ef828f67cecd0d647a62250) commit 4241a6eef03e93918d18e4a4c2ac38d4a55b82b6 Author: John Spray Date: Mon Mar 27 12:56:31 2017 +0100 mds: validate prealloc_inos on sessions after load Mitigates http://tracker.ceph.com/issues/16842 Signed-off-by: John Spray (cherry picked from commit c39aaf90ed1b23343eba2b341bb8ee6a50a4ea74) commit 41be67e80909962a004c66491815195e9a6fbf39 Author: John Spray Date: Mon Mar 27 12:33:59 2017 +0100 mds: operator<< for Session Use this to get a nice human readable name when available (also including the session id in parentheses) Signed-off-by: John Spray (cherry picked from commit 0f89787d8312f132ebb621f16c44e950b17a395a) commit ccde4da15c9ea50a4663b0697d0cef827c26f9c7 Merge: a1ed9bac52 52ee6a4c49 Author: Nathan Cutler Date: Tue Jul 4 13:18:48 2017 +0200 Merge pull request #16092 from smithfarm/wip-20150-kraken kraken: ceph-disk: fails if OSD udev rule triggers prior to mount of /var Reviewed-by: Loic Dachary commit 0b1dee0edcf3fe38289c10262655875d61a0dbbf Author: John Spray Date: Thu Mar 23 09:07:32 2017 -0400 mds: include advisory `path` field in damage This will just be whatever path we were looking at at the point that damage was notified -- no intention whatsoever of providing any up to date path or resolution when there are multiple paths to an inode. Fixes: http://tracker.ceph.com/issues/18509 Signed-off-by: John Spray (cherry picked from commit c0bff51ef409eb6e4b2fc248e06e5a7e43faf51e) Conflicts: src/mds/CDir.cc - trivial resolution (kraken does not have cb86740a5f4aa3eed43c7f09ac5e7e525a5c1d67) commit f458d60838628c4ed08448998956bc6ce7228f9b Author: Henrik Korkuc Date: Sun Feb 19 11:44:20 2017 +0200 client/Client.cc: add feature to reconnect client after MDS reset Client.cc marks session as stale instead of reconecting after received reset from MDS. On MDS side session is closed so MDS is ignoring cap renew. This adds option to reconnect stale client sessions instead of just marking sessions stale. Fixes: http://tracker.ceph.com/issues/18757 Signed-off-by: Henrik Korkuc (cherry picked from commit e0bbc704676ef4aed510daff075ef63c9e73b7b3) commit 7674f84ddf8f8f8f0a3fe8c3b336bacc0718eeed Author: Yan, Zheng Date: Wed Feb 22 17:33:05 2017 +0800 client: wait for lastest osdmap when handling set file/dir layout Fixes: http://tracker.ceph.com/issues/18914 Signed-off-by: "Yan, Zheng" (cherry picked from commit 76f5eb86cdd61dde4e6c7cfeb5cf34f0c0334f21) commit bee73d2429628e7d27cf9b1ca67eb5d5f049f285 Author: Yang Honggang Date: Thu Apr 13 20:09:07 2017 +0800 cephfs: fix write_buf's _len overflow problem After I have set about 400 64KB xattr kv pair to a file, mds is crashed. Every time I try to start mds, it will crash again. The root reason is write_buf._len overflowed when doing Journaler::append_entry(). This patch try to fix this problem through the following changes: 1. limit file/dir's xattr size 2. throttle journal entry append operations Fixes: http://tracker.ceph.com/issues/19033 Signed-off-by: Yang Honggang joseph.yang@xtaotech.com (cherry picked from commit eb915d0eeccbe523f8f70f6571880003ff459459) commit 32c7ebe3e6d19d37f198a8ae6bdc46d68fb3ed05 Author: John Spray Date: Wed Mar 8 12:13:46 2017 +0000 mds: shut down finisher before objecter Some of the finisher contexts would try to call into Objecter. We mostly are protected from this by mds_lock+the stopping flag, but at the Filer level there's no mds_lock, so in the case of file size probing we have a problem. Fixes: http://tracker.ceph.com/issues/19204 Signed-off-by: John Spray (cherry picked from commit 177a97d5c55ee6a2d5dcd3cf0893546190b10f7a) Conflicts: src/mds/MDSRank.cc - kraken has no 7189b53b410424d4a662486c1081181b2bd662ff so we omit the purge_queue.shutdown() call commit b267a1a8be0ea4893abb77695f1d1b2a0dac2c90 Author: John Spray Date: Tue Mar 28 14:13:33 2017 -0400 mds: ignore ENOENT on writing backtrace We get ENOENT when a pool doesn't exist. This can happen because we don't prevent people deleting former cephfs data pools whose files may not have had their metadata flushed yet. http://tracker.ceph.com/issues/19401 Signed-off-by: John Spray (cherry picked from commit 3fccc2372f2715d075b05e459140360cf6e7ca96) commit 93e81a87ac99080a8670b7485dc9c8c801477518 Author: YunfeiGuan Date: Mon Apr 10 05:48:47 2017 +0000 cephfs: fix mount point break off problem after mds switch occured The hot-standby become active as we expected but the mount piont broken strangely when the active mds is down. The root reason is the new mds use last_cap_renews decoded from ESesson::replay in find_idle_sessions and wrongly killed the session. Maybe we should reset session->last_cap_renew to the current time when server send OPEN to client in reconnect stage. Fixes: http://tracker.ceph.com/issues/19437 Signed-off-by: Guan yunfei (cherry picked from commit 4ef830c5d6f22bf0d4f82a8624c772ecbbda44a6) commit 78ccba49d3a1f53c3fd7ffe0d6b6bb45eb66c801 Author: Sage Weil Date: Sun Mar 5 21:59:45 2017 -0500 common: remove \n on clog messages Confirmed these aren't needed, and are inconsistently included. Remove them where they do appear. Signed-off-by: Sage Weil (cherry picked from commit 693132eb00b1803d5e97a79908521d5a6903e9f8) Conflicts: src/mon/OSDMonitor.cc - trivial resolution src/osd/PrimaryLogPG.cc - trivial resolution commit e72d6362c15668e2340212dcdeaa07fc94d12ba7 Author: Yan, Zheng Date: Wed Apr 5 21:29:10 2017 +0800 mds: make C_MDSInternalNoop::complete() delete 'this' Fixes: http://tracker.ceph.com/issues/19501 Signed-off-by: "Yan, Zheng" (cherry picked from commit 424e0c6744e7f63459ca0ff7deab751726aa30cd) Conflicts: src/mds/MDSContext.h - omit "override" because kraken does not have 1a91aeab987870b3ccbcf2f1e476fac8b534d449 commit a1ed9bac523d3115fe5120b2f74f450a739d89b1 Merge: 4c8d3ed4dc 8d64dd4f29 Author: Nathan Cutler Date: Tue Jul 4 11:37:29 2017 +0200 Merge pull request #14604 from asheplyakov/19647-kraken kraken: ceph-disk: enable directory backed OSD at boot time Reviewed-by: Loic Dachary commit 7054ff69d0450ecd86aff5c9c78a0a93f536d179 Author: Pan Liu Date: Fri Mar 31 00:23:12 2017 +0800 rbd-nbd: remove debug message from do_unmap Global context is not initialized when do_unmap is called. Signed-off-by: Pan Liu (cherry picked from commit cd748f9b260a3c388b45091ff0bfc041257acd0c) commit 36e2199f0bfd869054506476f64af7f603ee6541 Author: Kefu Chai Date: Wed Mar 29 19:07:34 2017 +0800 rbd-nbd: s/cpp_error/cpp_strerror/ to fix FTBFS the build failure was introduced by ff4dcf0 Signed-off-by: Kefu Chai (cherry picked from commit c12ecd984df0f9ac0d7de81b9d689f6182a3646a) commit 4d310c24af51bb0539426a5a7ca6f2b1ca31a1d0 Author: Pan Liu Date: Tue Mar 28 16:48:21 2017 +0800 rbd-nbd: polish the output info before and after ioctl NBD_DISCONNECT. Signed-off-by: Pan Liu (cherry picked from commit ff4dcf029028e8a3636ac71a6c5ac5380bf274e4) commit e6a01244e6ae2862b1c6e78be67b3c2be6899364 Author: Pan Liu Date: Tue Mar 28 16:33:25 2017 +0800 rbd-nbd: support signal handle for SIGHUP, SIGINT, and SIGTERM. Fixes: http://tracker.ceph.com/issues/19349 Signed-off-by: Pan Liu (cherry picked from commit 3ba01aa6ce052d1afa42132feffc2353d73caae6) commit 5dab0825d705d84926fd5f2f106fa945492bda34 Author: Jason Dillaman Date: Wed Apr 12 10:47:28 2017 -0400 test/librados_test_stub: fixed cls_cxx_map_get_keys/vals return value Fixes: http://tracker.ceph.com/issues/19597 Signed-off-by: Jason Dillaman (cherry picked from commit 9ffd464dac102f684d6dfa78e58d2cb45e165ed6) commit 8ed81b033a7a9120a24611c9e193d85e109ad2e5 Author: Jason Dillaman Date: Wed Mar 15 16:33:58 2017 -0400 librbd: clean up debug messages within image create state machine Signed-off-by: Jason Dillaman (cherry picked from commit d4e27c48aa3f96f650d6cbfe103be5aa980e0f56) commit fbd296933bb64239f49909a123fcd654000f910f Author: Jason Dillaman Date: Wed Mar 15 14:12:35 2017 -0400 librbd: removed legacy state machine callbacks from CreateRequest Signed-off-by: Jason Dillaman (cherry picked from commit 8d79bf2bcba1d09d8e5589c5a8ee6dca1d27f800) Conflicts: src/librbd/image/CreateRequest.cc - kraken does not have 5f2689b62ae commit f1cf9465984801b17ce536a6a94b2789b482d1a7 Author: Jason Dillaman Date: Thu Mar 2 10:29:36 2017 -0500 librbd: avoid duplicating librados IoCtx objects if not needed This introduces the potential for shutdown race conditions within the unit tests. Signed-off-by: Jason Dillaman (cherry picked from commit cc5ac6aa66f4c97cbe1c7d6334b3f710610f6742) commit 4c8d3ed4dc7fc51d7116bb65915ed5cf8ac468bd Merge: 8452c0c134 d186951730 Author: Nathan Cutler Date: Tue Jul 4 10:26:43 2017 +0200 Merge pull request #14616 from smithfarm/wip-18723-kraken kraken: osd: calc_clone_subsets misuses try_read_lock vs missing Reviewed-by: Kefu Chai commit b7503d3a80b14dd99f6e46fe63414815afcff814 Author: Matt Benjamin Date: Fri Apr 14 15:56:37 2017 -0400 rgw_file: fix readdir after dirent-change Also, fixes link count computation off-by-one, update of state.nlink after computation, link computation reset at start, and a time print in debug log. Fixes: http://tracker.ceph.com/issues/19634 Signed-off-by: Matt Benjamin link count Signed-off-by: Matt Benjamin (cherry picked from commit e0f80266ecd424bf9466579b3edc03911a7c5719) commit ea3aec3334871c5952986fcf14aac61348f1cc00 Author: Matt Benjamin Date: Tue Apr 11 06:42:07 2017 -0400 rgw_file: don't expire directories being read If a readdir expire event turns out to be older than last_readdir, just reschedule it (but actually, we should just discard it, as another expire event must be in queue. Fixes: http://tracker.ceph.com/issues/19625 Signed-off-by: Matt Benjamin (cherry picked from commit 007b7451c26716c51207c161dc347e9a00da53f1) commit 247f897c31f15c48fa0521311b1afc557e5a10ae Author: Matt Benjamin Date: Tue Apr 11 05:56:13 2017 -0400 rgw_file: chunked readdir Adjust readdir callback path for new nfs-ganesha chunked readdir, including changes to respect the result of callback to not continue. Pending introduction of offset name hint, our caller will just be completely enumerating, so it is possible to remove the offset map and just keep a last offset. Fixes: http://tracker.ceph.com/issues/19624 Signed-off-by: Matt Benjamin (cherry picked from commit e0191d74e3aef06bf300df045a53a3952a71f651) commit ac1dd8d323c5d90e20fd88e56031441f704ee95a Author: Matt Benjamin Date: Tue Feb 28 20:24:12 2017 -0500 rgw_file: RGWFileHandle dtor must also cond-unlink from FHCache Formerly masked in part by the reclaim() action, direct-delete now substitutes for reclaim() iff its LRU lane is over its high-water mark, and in particular, like reclaim() the destructor is certain to see handles still interned on the FHcache when nfs-ganesha is recycling objects from its own LRU. Fixes: http://tracker.ceph.com/issues/19112 Signed-off-by: Matt Benjamin (cherry picked from commit d51a3b1224ba62bb53c6c2c7751fcf7853c35a4b) commit 78e251632dd3103c5476544ea7386203bd5ff7a2 Author: Gui Hecheng Date: Thu Mar 2 17:21:57 2017 +0800 rgw_file: posix style atime,ctime,mtime As an ganesha FSAL backend, rgw_file should properly maintain the atime,ctime,mtime properly against operations such as: (read,write) for file (create,unlink,mkdir,rmdir,rename) for dir (setattr) for file and dir Signed-off-by: Gui Hecheng (cherry picked from commit ac25da2479b9be876cbdb820560ac46a6e2b17d7) commit 59a965a8e017da3dc6c79a25d55441b945cd50bc Author: Gui Hecheng Date: Wed Mar 8 16:23:11 2017 +0800 rgw_file: fix reversed return value of getattr When ::getattr returns -ESTALE, rgw_getattr returns ESTALE, which is a not expected postive. Signed-off-by: Gui Hecheng (cherry picked from commit 39203cf872b8f4af86eb0e4a0f96dffd9cc92b41) commit e5c78f8bf2733b803f4e8e051884c8130809e506 Author: Gui Hecheng Date: Wed Mar 15 15:01:05 2017 +0800 rgw_file: fix double unref on rgw_fh for rename Skip unref after unlink to fix the problem. Signed-off-by: Gui Hecheng (cherry picked from commit bff228734c73b536d2482e2e2fa4ad38b206ebff) commit f97ab40539385494f9d4a66806c2bfc08d3654bb Author: Matt Benjamin Date: Wed Feb 22 14:57:59 2017 -0500 rgw_file: rgw_lookup: don't ref for "/" or ".." These refs won't be returned by nfs-ganesha, and are sufficiently magical that other consumers should be persuaded to understand their specialness. Fixes: http://tracker.ceph.com/issues/19060 Signed-off-by: Matt Benjamin (cherry picked from commit dea8d1ee373399a21851690a9753388b659b8ede) commit 6e0b260154328b14c45ecbea00007e537ce220ac Author: Matt Benjamin Date: Thu Feb 23 16:02:07 2017 -0500 rgw_file: ensure valid_s3_object_name for directories, too The logic in RGWLibFS::mkdir() validated bucket names, but not object names (though RGWLibFS::create() did so). The negative side effect of this was not creating illegal objects (we won't), but in a) failing with -EIO and b) more importantly, not removing up the proposed object from FHCache, which produced a boost assert when recycled. Fixes: http://tracker.ceph.com/issues/19066 Signed-off-by: Matt Benjamin (cherry picked from commit eb1cd3b30c0504385f05bf2d2dd5e2251b7efed7) commit 349de8a5b30691b0579708ce95f91ce8b929a393 Author: Matt Benjamin Date: Fri Feb 3 13:44:45 2017 -0500 rgw_file: fix RGWLibFS::setattr for directory objects Fixes: http://tracker.ceph.com/issues/18808 Signed-off-by: Matt Benjamin (cherry picked from commit 4ad5a9226852d6d564baf2e63278ed6c4c185ecb) commit 2b9a77edf203ef39049677a94e13db0fdf0fed8d Author: Matt Benjamin Date: Tue Apr 4 20:16:13 2017 -0400 rgw_file: introduce rgw_lookup type hints The new type hints optimize object type deduction, when the rgw_lookup is called from an rgw_readdir callback. Fixes: http://tracker.ceph.com/issues/19623 Signed-off-by: Matt Benjamin (cherry picked from commit 2e66c7a7cc763c5c0d6f5db04855f60f2b2ceed3) commit 64cce346c0fcc1402d4dab548f6f7536430703e7 Author: Matt Benjamin Date: Thu Feb 23 10:21:38 2017 -0500 rgw_file: return of RGWFileHandle::FLAG_EXACT_MATCH Allow callers of rgw_lookup() on objects attested in an rgw_readdir() callback the ability to bypass exact match in RGWLibFS::stat_leaf() case 2, but restore exact match enforcement for general lookups. This preserves required common_prefix namespace behavior, but prevents clients from eerily permitting things like "cd sara0" via partial name match on "sara01." Fixes: http://tracker.ceph.com/issues/19059 Signed-off-by: Matt Benjamin (cherry picked from commit 70ef7d45e0abf2661bd4e23161d4e70cf5178079) commit fa208008dad8cb165ee72a8cf96ead36d855ba30 Author: Matt Benjamin Date: Tue Feb 28 15:49:06 2017 -0500 rgw_file: use fh_hook::is_linked() to check residence Previously we assumed that !deleted handles were resident--there is an observed case where a !deleted handle is !linked. Since we currently use safe_link mode, an is_linked() check is available, and exhaustive. Fixes: http://tracker.ceph.com/issues/19111 Signed-off-by: Matt Benjamin (cherry picked from commit c0aa515f8d8c57ec5ee09e3b60df3cac60453c40) commit 29fe4cfeb71a1b0a74fb7a073c2348cf2d1d143d Author: Matt Benjamin Date: Sun Feb 12 18:20:43 2017 -0500 rgw_file: refcnt bugfixes This change includes 3 related changes: 1. add required lock flags for FHCache updates--this is a crash bug under concurrent update/lookup 2. omit to inc/dec refcnt on root filehandles in 2 places--the root handle current is not on the lru list, so it's not valid to do so 3. based on observation of LRU behavior during creates/deletes, update (cohort) LRU unref to move objects to LRU when their refcount falls to SENTINEL_REFCNT--this cheaply primes the current reclaim() mechanism, so very significanty improves space use (e.g., after deletes) in the absence of scans (which is common due to nfs-ganesha caching) Signed-off-by: Matt Benjamin (cherry picked from commit beaeff059375b44188160dbde8a81dd4f4f8c6eb) commit 99941105a982f77513c8df576176f4a707daf26a Author: Matt Benjamin Date: Sat Feb 11 16:38:05 2017 -0500 rgw_file: add refcount dout traces at debuglevel 17 These are helpful for checking RGWFileHandle refcnt invariants. Signed-off-by: Matt Benjamin (cherry picked from commit 462034e17f919fb783ee33e2c9fa8089f93fd97d) commit a446c724aa2404f4f4ff8efc2da3e12ca2bf6892 Author: Matt Benjamin Date: Fri Feb 10 17:14:16 2017 -0500 rgw_file: add pretty-print for RGWFileHandle Signed-off-by: Matt Benjamin (cherry picked from commit ef330f385d3407af5f470b5093145f59cc4dcc79) commit 1553877b1673b21694a4a3b3ef9811800e67144e Author: Matt Benjamin Date: Fri Dec 30 23:30:16 2016 -0500 rgw_file: interned RGWFileHandle objects need parent refs RGW NFS fhcache/RGWFileHandle operators assume existence of the full chain of parents from any object to the its fs_root--this is a consequence of the weakly-connected namespace design goal, and not a defect. This change ensures the invariant by taking a parent ref when objects are interned (when a parent ref is guaranteed). Parent refs are returned when objects are destroyed--essentially by the invariant, such a ref must exist. The extra ref is omitted when parent->is_root(), as that node is not in the LRU cache. Fixes: http://tracker.ceph.com/issues/18650 Signed-off-by: Matt Benjamin (cherry picked from commit 0e5299f3f43e633a5d8a9360893b4b11f6217d81) commit 4af9f6fef77597a826cfdf5d0b3a814ce4629bc3 Author: Matt Benjamin Date: Wed Feb 22 10:24:29 2017 -0500 rgw_file: avoid stranding invalid-name bucket handles in fhcache To avoid a string copy in the common mkdir path, handles for proposed buckets currently are staged in the handle table, before being rejected. They need to be destaged, not just marked deleted (because deleted objects are now assumed not to be linked, as of beaeff059375b44188160dbde8a81dd4f4f8c6eb). This triggered an unhandled Boost assert when deleting staged handles, as current safe_link mode requires first removing from the FHCache. Fixes: http://tracker.ceph.com/issues/19036 Signed-off-by: Matt Benjamin (cherry picked from commit 6cde812c92e5bba9f85fbf8486ebe69b55952370) commit e03a5cba8ff4a2ceebf85a2697e90d525185b2d2 Author: Gui Hecheng Date: Fri Mar 31 10:42:40 2017 +0800 rgw_file: fix missing unlock in unlink Fixes: http://tracker.ceph.com/issues/19435 Signed-off-by: Gui Hecheng (cherry picked from commit cb6808a6366a70f54d0cc16437d16aa1b7819c84) commit 34553f919ccf0324138b990ff928b32dc0e4baf2 Author: Matt Benjamin Date: Wed Mar 15 16:35:16 2017 -0400 rgw_file: remove unused rgw_key variable Signed-off-by: Matt Benjamin (cherry picked from commit 1100a1c26e76485569cfebcf863b18cf908f6161) commit b04d051fd765a88a270041c5238a65481180528c Author: Matt Benjamin Date: Wed Mar 15 16:40:35 2017 -0400 rgw_file: rgw_readdir: return dot-dirs only when *offset is 0 Signed-off-by: Matt Benjamin (cherry picked from commit 61482c2b85a07519f2256b1a3f2b6d8aa99d5f06) commit b350b61bd2c75bc176f854bb3a39a453a2faec97 Author: Matt Benjamin Date: Mon Mar 13 21:52:08 2017 -0400 rgw_file: implement reliable has-children check (unlink dir) Bug report and discussion provided by Gui Hecheng in nfs-ganesha upstream github. Briefly, while a reliable check is potentially costly, it is necessary. Fixes: http://tracker.ceph.com/issues/19270 Signed-off-by: Matt Benjamin (cherry picked from commit b05f1c6d61aa4501a971e87de6dcaf3e58c3d9b4) commit 7faa917cdb1c484018dd47559c5f25b6b33b5529 Author: Matt Benjamin Date: Mon Feb 20 15:05:18 2017 -0500 rgw_file: fix marker computation Fixes: http://tracker.ceph.com/issues/19018 Signed-off-by: Matt Benjamin (cherry picked from commit 4454765e7dd08535c50d24205858e18dba4b454c) commit 50955a5aef10f18f050735af03734189f519f585 Author: Matt Benjamin Date: Thu Jan 19 18:14:30 2017 -0500 rgw_file: add timed namespace invalidation With change, librgw/rgw_file consumers can provide an invalidation callback, which is used by the library to invalidate directories whose contents should be forgotten. The existing RGWLib GC mechanism is being used to drive this. New configuration params have been added. The main configurable is rgw_nfs_namespace_expire_secs, the expire timeout. Updated post Yehuda review. Fixes: http://tracker.ceph.com/issues/18651 Signed-off-by: Matt Benjamin (cherry picked from commit deb2c1ea985fcb906e47b93fd3d0117794e2d0a1) Conflicts: src/rgw/rgw_lib_frontend.h - in class RGWLibProcess : public RGWProcess there was no public method stop() in kraken (now there is) commit c613ee7e5853c40c89215efdc231c9e5274aec58 Author: Matt Benjamin Date: Sun Feb 19 20:34:31 2017 -0500 rgw_file: rgw_readdir can't list multi-segment dirs This issue has one root cause in librgw, namely that the marker argument to these requests was incorrectly formatted (though the marker cache was working as intended). Secondarily, for nfs-ganesha users, there is a compounding issue that the RGW fsal was required by "temporary" convention to populate the entire dirent cache for a directory on a single readdir() invocation--the cache_inode/mdcache implementations invariantly pass (before future 2.5 changesets, currently in progress) a null pointer for the start cookie offset, intended to convey this. Fixes: http://tracker.ceph.com/issues/18991 Signed-off-by: Matt Benjamin (cherry picked from commit 2cd60ee9712291b906123aca1704288b18a9742b) commit 2e27aa9bb2484512620d885c0a19e35b1de7b652 Author: Matt Benjamin Date: Sun Feb 19 18:21:06 2017 -0500 rgw_file: allow setattr on placeholder directories When a POSIX path /foo/ is known only as an implicit path segment from other objects (e.g., /foo/bar.txt), a case that would usually arise from S3 upload of such an object, an RGWFileHandle object representing "/foo/" will be constructed as needed, with no backing in RGW. This is by design, but subsequently, if a setattr is performed on such a handle, we must be ready to create the object inline with storing the attributes. Fixes: http://tracker.ceph.com/issues/18989 Signed-off-by: Matt Benjamin (cherry picked from commit 55eec1c0a0e136736961423b7b6244d0f3693c1a) commit 70ebe961de881734b637c7d13e7dd63bc09066c6 Author: Matt Benjamin Date: Sun Feb 19 17:43:17 2017 -0500 rgw_file: invalid use of RGWFileHandle::FLAG_EXACT_MATCH The change which introduced this flag also caused it to be given as the flags argument to RGWLibFS::stat_leaf() when called from rgw_lookup(). This was incorrect: in particular, when a directory is known only as a common prefix of other objects, the AWS namespace mapping convention requires lookup("foo") to match a non-materialized instance of "foo/" (case 2 in RGWLibFS::stat_leaf's stat loop). Fixes: http://tracker.ceph.com/issues/18992 Signed-off-by: Matt Benjamin (cherry picked from commit e31e9eb980f958640150e8d7f17de1b9e5478b1e) commit 52ee6a4c49e5749463811b57f4af4a0824875b70 Author: Loic Dachary Date: Thu Jun 1 11:37:20 2017 +0200 ceph-disk: do not setup_statedir on trigger trigger may run when statedir is unavailable and does not use it. Fixes: http://tracker.ceph.com/issues/19941 Signed-off-by: Loic Dachary (cherry picked from commit 16bfbdd3d9988523bba31aace516c303057daa58) commit 8452c0c134a1cae30ac9e80eb7e08c6e54b16239 Merge: e8312132ff 75cdc3f3a1 Author: Nathan Cutler Date: Tue Jul 4 00:02:05 2017 +0200 Merge pull request #14852 from yehudasa/wip-rgw-support-ragweed-kraken kraken: rgw: add apis to support ragweed suite Reviewed-by: Nathan Cutler commit e8312132ff8caee9b0f83b641bfc0a1875bacf3f Merge: 7540396a34 6e5ac14ab6 Author: Nathan Cutler Date: Mon Jul 3 22:28:08 2017 +0200 Merge pull request #13514 from rzarzynski/wip-rgw-controllable-204-kraken kraken: rgw: make sending Content-Length in 204 and 304 controllable Reviewed-by: Yehuda Sadeh commit 7540396a34147fbb34558356fde18f0a11798d8f Merge: f31033cb5b 910b989970 Author: Nathan Cutler Date: Mon Jul 3 17:05:34 2017 +0200 Merge pull request #14403 from shinobu-x/wip-17331-kraken kraken: ceph-disk list reports mount error for OSD having mount options with SELinux context Reviewed-by: Loic Dachary commit 071f111abbd7d6c7a90fc58f52c84907cf4d9860 Author: Wido den Hollander Date: Mon Jan 23 21:06:05 2017 +0100 osd: Return correct osd_objectstore in OSD metadata Do not simply read the configuration value as it might have changed during OSD startup by reading the type from disk. Fixes: http://tracker.ceph.com/issues/18638 Signed-off-by: Wido den Hollander (cherry picked from commit 8fe6a0303b02ac1033f5bfced9f94350fe3e33de) Conflicts: src/osd/OSD.cc - g_conf->osd_objectstore was changed to cct->_conf->osd_objectstore by 1d5e967a05ddbcceb10efe3b57e242b3b6b7eb8c which is not in kraken commit ffe5bbfc6e09a47009dd281a38b35a8a3528812e Author: Yehuda Sadeh Date: Thu Jan 26 16:26:42 2017 -0800 rgw: fix use of marker in List::list_objects() Fixes: http://tracker.ceph.com/issues/18331 List marker is an index key, so treat it as such. This fixes infinite loop in orphans find command. Signed-off-by: Yehuda Sadeh (cherry picked from commit a5d1fa0587184f43c69d8e03114b58d43f320781) commit 56ca3b01906ad7ee25b4bc2db0b6df51d2b162cb Author: Radoslaw Zarzynski Date: Tue Apr 25 12:22:54 2017 +0200 rgw: fix RadosGW hang during multi-chunk upload of AWSv4. Fixes: http://tracker.ceph.com/issues/19754 Signed-off-by: Radoslaw Zarzynski (cherry picked from commit 72c1e2e351d984d0425a20f2c772951cbc36f13e) commit cac26471897b6126a9b95e02f1a93a3a5386dae1 Author: liuchang0812 Date: Mon Mar 27 13:08:12 2017 +0800 rgw/lifecycle: do not send lifecycle rules when GetLifeCycle failed Now, RGW will send two HTTP responses when GetLifeCycle failed. The first one is Error Respnse like 404, and the second is lifecycle rules. It will breaks s3 sdk and s3 utilities. Fixes: http://tracker.ceph.com/issues/19363 Signed-off-by: liuchang0812 (cherry picked from commit c3c0c828da5a64ca896475c1b0c369fde1bbd76a) commit e5719234b48a90bbe59a693c96da3634884a0578 Author: tone-zhang Date: Thu Apr 6 17:56:05 2017 +0800 rgw: fix response header of Swift API Response header of Swift API returned by radosgw does not contain "x-openstack-request-id", but Swift returns it. Enhance the compatibility of radosgw. Fixes: http://tracker.ceph.com/issues/19443 Signed-off-by: tone-zhang (cherry picked from commit e96db213079ab5e026156ab4b38418d1d4c23d27) commit 59391c40db969f2a9e97505cfcd2997d02ea86a3 Author: Yehuda Sadeh Date: Wed Mar 8 14:52:34 2017 -0800 rgw: fix crash when listing objects via swift Fixes: http://tracker.ceph.com/issues/19249 Signed-off-by: Yehuda Sadeh (cherry picked from commit a9ec5e8ce184e19c009863db4d3519f9d8af91bd) Conflicts: src/rgw/rgw_rest_swift.cc ("key" element of RGWObjEnt struct is not a reference; fix) commit e0f90f03ddf2824b448f6affc34ed40d4cead5d2 Author: Jason Dillaman Date: Thu Jan 26 14:02:11 2017 -0500 librbd: introduce new constants for tracking max block name prefix Fixes: http://tracker.ceph.com/issues/18653 Signed-off-by: Jason Dillaman (cherry picked from commit 2c08629c99d90aa7676b59263c055c9f1f577039) commit 3173da6e3bc7083ed2bdeea3a25bb3fa790bbc86 Author: Pan Liu Date: Thu Feb 16 22:17:52 2017 +0800 rbd-nbd: no need create asok file for unmap and list-mapped commands. Fixes: http://tracker.ceph.com/issues/17951 Signed-off-by: Pan Liu (cherry picked from commit 72352653d585ef89043a4ece371b5c0cb3f6f32a) commit 61c5957b30f807f5eab4c072deb00f19433f0a46 Author: Mykola Golub Date: Thu Feb 2 11:11:35 2017 +0100 rbd-nbd: check /sys/block/nbdX/size to ensure kernel mapped correctly Fixes: http://tracker.ceph.com/issues/18335 Signed-off-by: Mykola Golub (cherry picked from commit 596e5ea8a5df72002672eef0a6d20572ca6f60f0) commit c79d45ff52a636c136e0c5f8aec7911c04601b6f Author: Venky Shankar Date: Mon Feb 20 12:04:10 2017 +0530 librbd: acquire exclusive-lock during copy on read Fixes: http://tracker.ceph.com/issues/18888 Signed-off-by: Venky Shankar (cherry picked from commit 7dba5311b12011a4a6e8564e68150e54c5af5ddd) Conflicts: src/librbd/AioImageRequestWQ.h: - in master this file has morphed into src/librbd/io/ImageRequestWQ.h - kraken has AioImageRequest instead of ImageRequest src/librbd/image/RefreshRequest.cc: - rename image context element to "aio_work_queue" (from "io_work_queue") because kraken doesn't have de95d862f57b56738e04d77f2351622f83f17f4a src/test/librbd/image/test_mock_RefreshRequest.cc: - rename image context element to "aio_work_queue" (from "io_work_queue") because kraken doesn't have de95d862f57b56738e04d77f2351622f83f17f4a commit d186951730d947881fea0e638a6e112718820ae0 Author: Samuel Just Date: Wed Jan 18 10:24:13 2017 -0800 PrimaryLogPG::try_lock_for_read: give up if missing The only users calc_*_subsets might try to read_lock an object which is missing on the primary. Returning false in those cases is perfectly reasonable and avoids the problem. Fixes: http://tracker.ceph.com/issues/18583 Signed-off-by: Samuel Just (cherry picked from commit 3833440adea6f8bcb0093603c3a9d16360ed57ec) commit 30ff11c84fd6d53c6439aaed8f7742b98a5b6218 Author: Samuel Just Date: Wed Nov 23 15:41:13 2016 -0800 ReplicatedBackend: take read locks for clone sources during recovery Otherwise, we run the risk of a clone source which hasn't actually come into existence yet being used if we grab a clone which *just* got added the the ssc, but has not yet actually had time to be created (can't rely on message ordering here since recovery messages don't necessarily order with client IO!). Fixes: http://tracker.ceph.com/issues/17831 Signed-off-by: Samuel Just (cherry picked from commit 68defc2b0561414711d4dd0a76bc5d0f46f8a3f8) commit e76725f4e9fb4a6cd7c76480d0313a8e0264eb73 Author: Sage Weil Date: Thu Mar 30 13:50:41 2017 -0400 osd/PrimaryLogPG: do not expect FULL_TRY ops to get resent The objecter will not resend FULL_TRY requests that were sent prior to becoming full, so we should not discard them. Fixes: http://tracker.ceph.com/issues/19430 Signed-off-by: Sage Weil (cherry picked from commit 3f7acdbc9a942fd18937dbcf07fbc7b752c50ba3) commit 7a275769b6a281eaec1a659975ec50bfe847e679 Author: Sage Weil Date: Fri Mar 31 10:06:42 2017 -0400 ceph_test_librados_api_misc: fix stupid LibRadosMiscConnectFailure.ConnectFailure test Sometimes the cond doesn't time out and it wakes up instead. Just repeat the test many times to ensure that at least once it times out (usually it doesn't; it's pretty infrequent that it doesn't). Fixes: http://tracker.ceph.com/issues/15368 Signed-off-by: Sage Weil (cherry picked from commit 8bc197400d94ee2716d3f2fa454247379a676cf9) commit 4d1272e5c654c32a844a791b08a97ea8c28c5e5d Author: Kefu Chai Date: Tue Apr 18 14:07:04 2017 +0800 debian: package ceph.logroate properly see also "man dh_installlogrotate" Fixes: http://tracker.ceph.com/issues/19390 Signed-off-by: Kefu Chai (cherry picked from commit 17ca501fe8927d541d50a502da53a4d7628d2b4a) commit 47a45441762e610eef548273c719cf70643bfeac Author: David Zafman Date: Mon Apr 17 14:58:02 2017 -0700 osd: Give requested scrub work a higher priority Once started we now queue scrub work at higher priority than scheduled scrubs. Fixes: http://tracker.ceph.com/issues/15789 Signed-off-by: David Zafman (cherry picked from commit ebab8b1f4f67fbdec1e147c580329c1e2b5cf7cd) Conflicts: src/osd/OSD.h - in kraken, the PGScrub() call is enclosed within op_wq.queue(make_pair(...)) instead of enqueue_back() commit f31033cb5bc7764b44871c15717aed5a34921881 Merge: b7a2e67460 795538210c Author: Nathan Cutler Date: Mon Jul 3 11:39:42 2017 +0200 Merge pull request #16069 from smithfarm/wip-20345-kraken kraken: make check fails with Error EIO: load dlopen(build/lib/libec_FAKE.so): build/lib/libec_FAKE.so: cannot open shared object file: No such file or directory Reviewed-by: Kefu Chai commit 795538210ce76adc7521fac2caf5de27856aee39 Author: Kyr Shatskyy Date: Wed Feb 8 16:02:52 2017 +0100 ceph.spec.in, debian/control: Add bc to build dependencies The bc is missing for ceph-helpers.sh Fixes: http://tracker.ceph.com/issues/18876 Signed-off-by: Kyr Shatskyy (cherry picked from commit 3ff4be6d6896346b7b9ec7f158fcde8866faeb38) commit 977770d828252c25004cb6344c43d85abd8ee6f5 Author: Kefu Chai Date: Wed Apr 12 17:31:52 2017 +0800 qa/workunits/ceph-helpers: do not error out if is_clean it would be a race otherwise, because we cannot be sure that the cluster pgs are not all clean or not when run_osd() returns, but we can be sure that they are expected to active+clean after a while. that's what wait_for_clean() does. Signed-off-by: Kefu Chai (cherry picked from commit 6cb4503a40ae4ebee5690fda024cae8d1a506bce) commit b1d3c1a322dd611b3608732e7ef31cf15e671845 Author: Kefu Chai Date: Wed Apr 12 12:33:53 2017 +0800 qa/workunits/ceph-helpers: display rejected string Signed-off-by: Kefu Chai (cherry picked from commit 0196e154ed2e164cf55b0d7ed9f9cdd1f4f50100) commit 75cdc3f3a16d3799ae417862b9b5ee41cfcaee3e Author: Yehuda Sadeh Date: Mon Jan 9 13:04:43 2017 -0800 rgw: new rest api to retrieve object layout Signed-off-by: Yehuda Sadeh (cherry picked from commit 2768583dc486109e49d209243675b99fdd39e92c) commit df30ac30112a0c15798c868dc1fb6bd58bcc05a2 Author: Yehuda Sadeh Date: Thu Jan 5 13:47:24 2017 -0800 rgw: rest api to read zone config params Signed-off-by: Yehuda Sadeh (cherry picked from commit a220a2efbfd675e6abf14ae33c21005bcbf6dadf) commit 47f751f48ba972f2c93ecad919c66ee03b4aec70 Author: Yehuda Sadeh Date: Thu May 4 10:59:07 2017 -0700 civetweb: move to post 1.8 version Fixes: http://tracker.ceph.com/issues/19704 Version that went into kraken was pre 1.8, and had a few issues. Signed-off-by: Yehuda Sadeh commit ff19977fbc4cf4a787af80b6c0457df932004fd3 Author: Alexey Sheplyakov Date: Tue Jun 27 16:07:01 2017 +0400 kraken: osd: unlock sdata_op_ordering_lock with sdata_lock hold to avoid missing wakeup signal Based on commit bc683385819146f3f6f096ceec97e1226a3cd237. The OSD code has been refactored a lot since Kraken, hence cherry-picking that patch introduces a lot of unrelated changes, and is much more difficult than reusing the idea. Fixes: http://tracker.ceph.com/issues/20443 Signed-off-by: Alexey Sheplyakov commit 9efeb6bc2df559f8d91c480c396b4dd97b981d34 Author: Jason Dillaman Date: Thu May 4 17:56:22 2017 -0400 librbd: add no-op event when promoting an image The rbd-mirror process needs an event in the journal to properly detect the transition between primary and non-primary state between peers. Fixes: http://tracker.ceph.com/issues/19858 Signed-off-by: Jason Dillaman (cherry picked from commit 4031555dda7597d24e9eb04b9ff29173909586f7) Conflicts: src/librbd/journal/DemoteRequest.cc: logic exists in Journal.cc commit 1d71475ff561c7f7d4936774d0ff821786b15fe6 Author: Jason Dillaman Date: Thu May 4 17:57:34 2017 -0400 rbd-mirror: prevent infinite loop when computing replay status If the image had a non-primary predecessor epoch whose tag tid duplicates an epoch within its own journal, an infinite loop would result. Signed-off-by: Jason Dillaman (cherry picked from commit 3f179bf86216540d8e25aad469c604f96f6aecd8) commit b7a2e674601133f481449d3f8cdd3512ccbad51c Merge: 2f5c65bfc2 9fd233c2d4 Author: Zack Cerza Date: Mon Jun 26 14:11:56 2017 -0600 Merge pull request #15869 from smithfarm/wip-swift-task-move-kraken tests: move swift.py task from teuthology to ceph, phase one (kraken) commit 9fd233c2d406a31bfc35993539d60e0b41a53cc0 Author: Nathan Cutler Date: Sun Jun 25 12:42:36 2017 +0200 tests: swift.py: tweak imports The ".." form only works within the teuthology repo. With swift.py now in the Ceph repo, we have to be explicit. Error message was: "ValueError: Attempted relative import beyond toplevel package Signed-off-by: Nathan Cutler commit 5d8ffee4d07d21f12a748af2a86820b1307f2e7c Merge: 2f5c65bfc2 7b58ac97e9 Author: Nathan Cutler Date: Fri Jun 23 08:34:53 2017 +0200 Merge branch 'master' of /home/smithfarm/src/ceph/upstream/teuthology into wip-swift-task-move-kraken commit 7b58ac97e9dd195f4170e9e0ea00bae76d1f3ccd Author: Nathan Cutler Date: Fri Jun 23 08:27:42 2017 +0200 tests: move swift.py task to qa/tasks In preparation for moving this task from ceph/teuthology.git into ceph/ceph.git The move is necessary because jewel-specific changes are needed, yet teuthology does not maintain a separate branch for jewel. Also, swift.py is a Ceph-specific task so it makes more sense to have it in Ceph. Signed-off-by: Nathan Cutler commit 73fe367bc4bf27730479bdc46931dd7c3c710c39 Author: Sage Weil Date: Mon May 29 21:55:33 2017 -0400 os/bluestore: deep decode onode value In particular, we want the attrs (map) to be a deep decode so that we do not pin this buffer, and so that any changed attr will free the previous memory. Signed-off-by: Sage Weil (cherry picked from commit f53f56350b893bfdd47fe730a4339ca5793791a0) commit 2f5c65bfc229cb43ac5c193fed7c9f51bd20cf79 Merge: 4b1e8bcf4c f96262fc9c Author: Nathan Cutler Date: Tue Jun 20 22:59:27 2017 +0200 Merge pull request #13181 from smithfarm/wip-18548-kraken kraken: rgw: multisite: segfault after changing value of rgw_data_log_num_shards Reviewed-by: Orit Wasserman commit 4b1e8bcf4c86168ec66808a9a88a6fa556e97445 Merge: 7706a386e2 291ffe1801 Author: Nathan Cutler Date: Tue Jun 20 22:56:59 2017 +0200 Merge pull request #13838 from smithfarm/wip-19049-kraken kraken: rgw: multisite: some yields in RGWMetaSyncShardCR::full_sync() resume in incremental_sync() Reviewed-by: Orit Wasserman Reviewed-by: Casey Bodley commit 7706a386e255eec46fafa58ea1435293581218cc Merge: 2f4a775d5f d1ec59544b Author: Nathan Cutler Date: Tue Jun 20 22:52:13 2017 +0200 Merge pull request #13224 from smithfarm/wip-18780-kraken kraken: rgw: Swift API: spurious newline after http body causes weird errors Reviewed-by: Orit Wasserman commit d38a7529995cd13b1a3d13a8e00d22ce0172aa5c Author: Kefu Chai Date: Fri May 5 12:02:05 2017 +0800 mon: check is_shutdown() in timer callbacks introduce a helper class: C_MonContext, and initialize all timer events using it, to ensure that they do check is_shutdown() before doing their work. Fixes: http://tracker.ceph.com/issues/19825 Signed-off-by: Kefu Chai (cherry picked from commit 561cbded0c7e28231b1c7ce18663b8d7d40aad6d) commit e7361176f4ad1d4b43a393e6e34c50233a73e829 Author: Kefu Chai Date: Thu May 4 22:49:04 2017 +0800 mon/Elector: call cancel_timer() in shutdown() instead of doing it manually. Signed-off-by: Kefu Chai (cherry picked from commit 12139ae529a49b6caedea89f910d034ddca094b6) commit bf347e92a2d67032f4365a125f3bb9975930e912 Author: Alexey Sheplyakov Date: Wed May 17 17:50:10 2017 +0400 kraken: mon: add override annotation to callback classes The only purpose of this patch is to avoid merge conflicts while cherry-picking commit 561cbded0c7e28231b1c7ce18663b8d7d40aad6d. Alternatively one could cherry-pick 1effdfe19bf9fd6d546620b96eaf452e889b15dc, but that one brings a lot of unrelated changes. Signed-off-by: Alexey Sheplyakov commit 3c4a5ea385c7d2f2dfe88328e2c5f7778928d92d Author: Jan Fajerski Date: Thu Apr 20 18:38:43 2017 +0200 fs: normalize file open flags internally used by cephfs The file open flags (O_foo) are platform specific. Normalize these flags before they are send to the MDS. For processing of client messages the MDS should only compare to these normalized flags. Otherwise this can lead to bogus flags getting transmitted on ppc64. Signed-off-by: Jan Fajerski (cherry picked from commit 88d2da5e93198e69435e288ce00d216d5fe27f80) Conflicts: src/client/Client.cc - trivial resolution, add ceph_flags_sys2wire when flags are logged commit b19d6eb3110cdf121c086a754176005a0e62797f Author: Mykola Golub Date: Mon Apr 24 16:23:21 2017 +0200 test/librbd/test_notify.py: don't disable feature in slave On jewel it will have stolen the exclusive lock. Instead, ensure that object map and fast diff are already disabled on the clone before the start of the test. Fixes: http://tracker.ceph.com/issues/19716 Signed-off-by: Mykola Golub (cherry picked from commit e009e1bdd4b3997462feb9a050bd2eb201e028ba) commit 2f4a775d5ffd170903c4412e4274ab2a188922ef Merge: 358081d27a 4776067797 Author: Nathan Cutler Date: Tue Jun 20 08:46:28 2017 +0200 Merge pull request #13174 from smithfarm/wip-18711-kraken kraken: rgw: slave zonegroup cannot enable the bucket versioning Reviewed-by: Orit Wasserman commit f8db63f24dcb0d3aa93eba84e80a3450fb4bd666 Author: Casey Bodley Date: Wed Mar 8 16:31:34 2017 -0500 rgw: data sync skips slo data when syncing the manifest object Fixes: http://tracker.ceph.com/issues/19027 Signed-off-by: Casey Bodley (cherry picked from commit 8b69847d7b3e92c70090d1dddf7cea5c44fb6b20) Conflicts: src/rgw/rgw_rest_conn.cc (kraken is missing 50c522ea89a756123bf74ab615138cf8478b2cee) src/rgw/rgw_rest_s3.h (omitted override because kraken is missing 3b247475a2c35526c129535021adfa621ecb5327) commit e3d4d5791867009c8c4b4bc867b65a23116653ae Author: Casey Bodley Date: Mon Mar 13 11:33:02 2017 -0400 rgw: RGWGetObj applies skip_manifest flag to SLO Signed-off-by: Casey Bodley (cherry picked from commit 987377ae34382e107e1d54f0bfc1121fcedb4513) commit 6c1ba33a7bdffb7fdaf288596acf628ea8ed471c Author: Casey Bodley Date: Tue Feb 21 10:27:13 2017 -0500 rgw: allow system users to read SLO parts multisite data sync relies on fetching the object as the system user Fixes: http://tracker.ceph.com/issues/19027 Signed-off-by: Casey Bodley (cherry picked from commit d50d18c500fd5dd89e7cada1162cf453b36df370) commit 6860fce4243f0a02bf424a477bf554da608cf90c Author: Orit Wasserman Date: Sun Jan 22 15:05:30 2017 +0200 rgw: add check for update return value Signed-off-by: Orit Wasserman (cherry picked from commit 125955e0625461065dc4755b900e51c3598cadb4) commit b9c7c6503eff88bccea3bb19b4edeba89cb73718 Author: Orit Wasserman Date: Sun Jan 22 14:42:14 2017 +0200 rgw: we need to reinit the zonegroup after assignment to avoid invalid cct and store Fixes: http://tracker.ceph.com/issues/18631 Signed-off-by: Orit Wasserman (cherry picked from commit ac9a7565ddf801121f22476cf3f66668f311833e) commit db6fb3c005e05f0c6f503a367919e31943044e6b Author: Orit Wasserman Date: Sun Jan 22 14:40:16 2017 +0200 rgw: fix init_zg_from_period when default zone is not set as default Signed-off-by: Orit Wasserman (cherry picked from commit 5393077e07bed45b9fc007591d365f1229d3e815) commit 5f4c38fa6b2162cb97162089a8a22095a7cde99d Author: Yehuda Sadeh Date: Fri Jan 20 17:05:24 2017 -0800 rgw: don't update bucket index multiple times in overwrite Instead of this for overwrites: prepare (index), write (head) [-EEXIST] cancel (index) read (head) prepare (index) write (head) complete (index) We now do: prepare (index), write (head) [-EEXIST] read (head) write (head) complete (index) Signed-off-by: Yehuda Sadeh (cherry picked from commit 7f4818f9611ea8f7b3fac7df8c5c314964f657a2) commit 96c8161a786783b84a164afef162dd24c46abdc4 Author: Yehuda Sadeh Date: Fri Jan 20 16:17:49 2017 -0800 rgw: assume obj write is a first write if fails and object already exists then retry. This improves first obj write performance on the expense of overwrites. Fixes: http://tracker.ceph.com/issues/18622 Signed-off-by: Yehuda Sadeh (cherry picked from commit 6f27f6089238d2a20e7a0f13066eddfc31192dc8) commit f2e39fec64df7592bc3b6191f122cdfb5d407d6a Author: Yehuda Sadeh Date: Fri Jan 20 14:40:58 2017 -0800 rgw: configurable write obj window size Fixes: http://tracker.ceph.com/issues/18623 Signed-off-by: Yehuda Sadeh (cherry picked from commit 66a82b4266acedfdd71c64394d68d9e50ed11b20) commit 8c67dd10c63009e5bc67c20493de732f7bf4abf7 Author: Yehuda Sadeh Date: Fri Jan 20 12:03:54 2017 -0800 rgw: change default chunk size to 4MB Fixes: http://tracker.ceph.com/issues/18621 Signed-off-by: Yehuda Sadeh (cherry picked from commit 54ef04bc957a3fb034dce903d62a42d77950dd72) commit f96262fc9c3805aa5fa04348f520dba8be1e4ae8 Author: Casey Bodley Date: Wed Jan 11 09:32:59 2017 -0500 rgw: fix off-by-one in RGWDataChangesLog::get_info Fixes: http://tracker.ceph.com/issues/18488 Signed-off-by: Casey Bodley (cherry picked from commit a0974fdcf62e60cf31bc15588e7b718da6f6ade3) commit 935cc11595f4a0b56313c8c98aae0c7684bfff05 Author: Radoslaw Zarzynski Date: Fri Feb 3 14:41:50 2017 +0100 rgw: improve handling of illformed Swift's container ACLs. Fixes: http://tracker.ceph.com/issues/18796 Signed-off-by: Radoslaw Zarzynski (cherry picked from commit f780fc6ec40395ad0941d4e0309d464fe33836b1) commit 500eac722d3e59ca8408d1bf9a23cc7368164c58 Author: Jing Wenjun Date: Wed Jan 11 05:28:43 2017 +0800 rgw: the swift container acl should support field .ref On the openstack-swift. The container acl supports .ref, which is ignored on ceph swift. Fixes: http://tracker.ceph.com/issues/18484 Signed-off-by: Jing Wenjun (cherry picked from commit b06f9cd9f0900db7b0d0fbcaea69cdd0d4b10132) commit d1ec59544b3894f8ce7a9d00b5e8fb937fb20b7a Author: Marcus Watts Date: Wed Jan 11 00:06:15 2017 -0500 radosgw/swift: clean up flush / newline behavior. The current code emits a newline after swift errors, but fails to account for it when it calculates 'content-length'. This results in some clients (go github.com/ncw/swift) producing complaints about the unsolicited newline such as this, Unsolicited response received on idle HTTP channel starting with "\n"; err= This logic eliminates the newline on flush. This makes the content length calculation correct and eliminates the stray newline. There was already existing separator logic in the rgw plain formatter that can emit a newline at the correct point. It had been checking "len" to decide if previous data had been emitted, but that's reset to 0 by flush(). So, this logic adds a new per-instance variable to separately track state that it emitted a previous item (and should emit a newline). Fixes: http://tracker.ceph.com/issues/18473 Signed-off-by: Marcus Watts Signed-off-by: Matt Benjamin (cherry picked from commit 5f229d6a33eae4906f22cdb90941835e47ee9f02) commit 1cb98684565427a28b286dcaaa6cce48626ba79b Author: Radoslaw Zarzynski Date: Wed Feb 1 19:05:50 2017 +0100 rgw: be aware abount tenants on cls_user_bucket -> rgw_bucket conversion. Fixes: http://tracker.ceph.com/issues/18364 Fixes: http://tracker.ceph.com/issues/16355 Signed-off-by: Radoslaw Zarzynski (cherry picked from commit 871e1f51afe9d6c8b88debc07460b4316121f999) commit 87811ff161d3c6273050a6be5d7eb7582ecca23c Author: Orit Wasserman Date: Sun Jan 1 12:56:44 2017 +0200 radosgw-admin: check for name mistmatch in realm set Signed-off-by: Orit Wasserman (cherry picked from commit 4facc5f4f3e842d371115a9a04d86257280014f0) commit ff888f8f9a4b81347b78ae6681a868cc9a7910e9 Author: Orit Wasserman Date: Sun Jan 1 12:40:37 2017 +0200 radosgw-admin: relam set can use input redirection Signed-off-by: Orit Wasserman (cherry picked from commit b8b3ae3be3f8e4c05cb23062d25c701b15900475) commit 68d2d4bb91d6c508c0e1c154aff3b5de3dc347eb Author: Orit Wasserman Date: Sun Jan 1 12:36:04 2017 +0200 radosgw-admin: realm set should create a new realm Fixes: http://tracker.ceph.com/issues/18333 Signed-off-by: Orit Wasserman (cherry picked from commit e23339c9ef34f6b9df90b1ab64b550af9b541d9e) commit 291ffe18018029f1418afb2dc038c605a1e34157 Author: Casey Bodley Date: Tue Nov 29 11:29:41 2016 -0500 rgw: fix for broken yields in RGWMetaSyncShardCR Fixes: http://tracker.ceph.com/issues/18076 Signed-off-by: Casey Bodley (cherry picked from commit e62d48a9bf2e309eab1a863f167af5267ebcc371) commit 4776067797b46b4d38d987caa5da4e6954d392aa Author: Orit Wasserman Date: Mon Dec 12 14:00:05 2016 +0100 rgw: complete versioning enablement after sending it to meta master Fixes: http://tracker.ceph.com/issues/18003 Signed-off-by: Orit Wasserman (cherry picked from commit 2d8aafb9dbe64bd9dd2b7d5ed50c6e9550cbe1ab) commit 27e9644ceb469fb2f54cea468d873ad13c1fe6c4 Author: Haomai Wang Date: Tue Jun 13 10:19:55 2017 +0800 msg/async: go to open new session when existing already closed Fixes: http://tracker.ceph.com/issues/20230 Signed-off-by: Haomai Wang (cherry picked from commit 99f580a3959240f99061a9ad48ec591b39a9fd46) commit 67ee45bbed3975f338b798caf81cd89f80ceb785 Author: Haomai Wang Date: Tue Jun 13 10:16:47 2017 +0800 msg/async: fix accept_conn not remove entry in conns when lazy delete Signed-off-by: Haomai Wang (cherry picked from commit bf98babb3289a7714543ff3cbd3872d80f0dc196) commit 27cbff111458e77baae617c7a222610c8ff0abee Author: Mykola Golub Date: Tue Apr 11 22:31:43 2017 +0200 librbd: fix rbd_metadata_list and rbd_metadata_get - properly check for val_len in rbd_metadata_list - don't expect output buffers are zero pre-filled Fixes: http://tracker.ceph.com/issues/19588 Signed-off-by: Mykola Golub (cherry picked from commit 75afc74ea681402e22b6dec8b83276d145fc786b) commit 30e3ae27e0b99ae9b7f3234dbb93260ebda91466 Author: Boris Ranto Date: Fri May 26 09:52:25 2017 +0200 rpm: Move ceph-disk to ceph-base The SELinux package now requires the ceph-disk binary but that one was part of the ceph-osd package. The ceph-disk python library is already packaged in ceph-base so moving ceph-disk to ceph-base seems like a reasonable next step. Signed-off-by: Boris Ranto (cherry picked from commit 6991764f3bff7b8f6be699603927aff882377878) Conflicts: ceph.spec.in: ceph-disk-udev is present in kraken commit fa100ed57c3002b357d56398c3589f33cc651fcf Author: Boris Ranto Date: Thu May 25 14:36:13 2017 +0200 ceph-disk: Fix the file ownership, skip missing This commit fixes the file ownership for the /usr/bin/ and /etc/ceph files and skips missing files as some of the files that we do specify now can be missing from the system (not installed, e.f. radosgw). Fixes: http://tracker.ceph.com/issues/20077 Signed-off-by: Boris Ranto (cherry picked from commit 077038b4393a28ccbd38ca4a90105dbd4c1ffcd5) commit 4e0cfecd593b267811e6bf92e81ba98dbede5317 Author: Boris Ranto Date: Fri Apr 28 12:29:46 2017 +0200 selinux: Do parallel relabel on package install We can take advantage of ceph-disk fix subcommand when doing a package install. We will keep using the differential fixfiles command otherwise. We also need to add relabel for /usr/bin/ daemons so that we could use this. Fixes: http://tracker.ceph.com/issues/20077 Signed-off-by: Boris Ranto (cherry picked from commit 1cecddf031991f1c64ea203f173189624f11940e) commit 358081d27a9b2c39e9daed10244819df5291da48 Merge: ae0eab53f0 bf26c56555 Author: David Zafman Date: Fri Jun 2 09:54:45 2017 -0700 Merge pull request #15421 from dzafman/wip-20125 Reviewed-by: Josh Durgin commit bf26c5655505369753142646ac357df2b8897ab7 Author: David Zafman Date: Wed May 31 15:39:19 2017 -0700 osd: Object level shard errors are tracked and used if no auth available Shards with object mismatch are tracked to mark them inconsistent Fix test because storing omap_digest in object_info not behaving as before Fixes: http://tracker.ceph.com/issues/20089 Signed-off-by: David Zafman (cherry picked from commit 1cacbea763c7aabfeaaf4bd5e878301044184117) commit 727653623f6cc616bdeeab2f0ce98f41a649ceee Author: Casey Bodley Date: Fri May 5 14:56:40 2017 -0400 cls/rgw: list_plain_entries() stops before bi_log entries list_plain_entries() was using encode_obj_versioned_data_key() to set its end_key, which gives a prefix of BI_BUCKET_OBJ_INSTANCE_INDEX[=2] that range between start_key and end_key would not only span the BI_BUCKET_OBJS_INDEX[=0] prefixes, but BI_BUCKET_LOG_INDEX[=1] prefixes as well. this can result in list_plain_entries() trying and failing to decode a rgw_bi_log_entry as a rgw_bucket_dir_entry Fixes: http://tracker.ceph.com/issues/19876 Signed-off-by: Casey Bodley (cherry picked from commit b29a1633a57abf443d5790c13d680d2917f86037) commit 648c6adf52a7847a5b46d1523da45a29504a68da Author: Sage Weil Date: Fri Feb 3 10:08:33 2017 -0500 mon/PGMap: factor mon_osd_full_ratio into MAX AVAIL calc If we only fill OSDs to 95%, we should factor that into the MAX AVAIL calculation for the pool. Fixes: http://tracker.ceph.com/issues/18522 Signed-off-by: Sage Weil (cherry picked from commit f223ac92917f4bc18e5b9b3ad61afa155e4d088a) commit 2aec591cfe1ba69631860b4886d34de80f752bd7 Author: Kefu Chai Date: Thu May 11 13:13:39 2017 +0800 osd/PrimaryLogPG: do not call on_shutdown() if (pg.deleting) when a callback is called, it could be facing a PG already shut down by OSD. but if that callback wants to shut that PG down. it should check the PG's status first. Fixes: http://tracker.ceph.com/issues/19902 Signed-off-by: Kefu Chai (cherry picked from commit f3c44a0dfc859f6f625a92e727e0e521ed4a9207) commit ae0eab53f0bb05c954fc98e019e2d5a054dbba45 Merge: 9df2d772ff 28e1fd0f5d Author: Boris Ranto Date: Thu May 11 08:06:47 2017 +0200 Merge pull request #14345 from ceph/wip-ceph-disk-fix-kraken ceph-disk: Add fix subcommand kraken back-port Reviewed-by: Loic Dachary Reviewed-by: Nathan Cutler commit 28e1fd0f5d1442d4b56194082014853db551def4 Author: Boris Ranto Date: Wed Mar 8 09:38:39 2017 +0100 ceph-disk: Add --system option for fix command This adds the ability to restore the labels of the underlying system data in addition to ceph data. Signed-off-by: Boris Ranto (cherry picked from commit 8d81af42fd507c7b92c8279eb114b0a733ac1da6) commit dfa721f48df5eac565d27388adc17c40991502c6 Author: Boris Ranto Date: Thu Feb 16 11:34:27 2017 +0100 ceph-disk: Add more fix targets It turns out I forgot several more directories that needs to be fixed by this script. We need to fix /var/log/ceph, /var/run/ceph and /etc/ceph as well. Signed-off-by: Boris Ranto (cherry picked from commit ae139307d6b2bfba47e21d29d6dbd3c8dc01b5b5) commit 2db2fc81835ea8d0b35f563dd09691b030c920e9 Author: Boris Ranto Date: Thu Feb 9 19:17:12 2017 +0100 ceph-disk: Add unit test for fix command This will simulate the command* functions to not actually run anything thus excercising the python code directly. It also checks that the proper (sub-strings) are in the output. Signed-off-by: Boris Ranto (cherry picked from commit 1ec53dee9a690134936bdc3a09c9a02fecf13a9d) commit 82764ca8e986c8efc31488f2b30d2a2706f9826c Author: Boris Ranto Date: Tue Jan 31 13:19:33 2017 +0100 ceph-disk: Add fix subcommand This subcommand will fix the SELinux labels and/or file permissions on ceph data (/var/lib/ceph). The command is also optimized to run the commands in parallel (per sub-dir in /var/lib/ceph) and do restorecon and chown at the same time to take advantage of the caching mechanisms. Signed-off-by: Boris Ranto (cherry picked from commit 6d5d30f6ed7538271579cc2ef4e2e364f01a4a6f) commit 9df2d772ff272386bff6b1657f3f1528fcaf4a8a Merge: 6b1782959e 04f8186f37 Author: Sage Weil Date: Mon May 8 11:33:59 2017 -0500 Merge pull request #14983 from liewegas/wip-denc-dump-kraken ceph-object-corpus: kraken objects (kraken) commit 04f8186f37d5add0e68ff47a025f0296377e96d1 Author: Sage Weil Date: Mon Feb 27 16:10:34 2017 -0500 test/encoding/readable.sh: join children at finish Signed-off-by: Sage Weil (cherry picked from commit c8a349eb4603be295a6f0e1683105cb93bfcd066) commit ba249885d1a37e703f37a08958af2f85eb227bd6 Author: Sage Weil Date: Mon Feb 27 16:10:24 2017 -0500 test/encoding/readable: fix passing of results to parent Signed-off-by: Sage Weil (cherry picked from commit 1325ba672d5b5c8a12722a395ad029c98d1ff236) commit d97154e61bb615faf566a42a878b2e7a303d99ae Author: Sage Weil Date: Fri May 5 16:48:25 2017 -0400 messages/MCommand: fix type on decode Wow, this has been broken since v0.38, but apparently the message never made it into the object corpus so we never noticed! In reality the bug is harmless: decode_message() will set_header which clobbers whatever version the default ctor fills in, so this only affects ceph-dencoder's test. Signed-off-by: Sage Weil (cherry picked from commit 3018b907c1d0375b5ae20802811591c0a46b61be) commit b05fa6cd2ff07349d6cc76be0732e627f0290308 Author: Sage Weil Date: Fri May 5 16:32:48 2017 -0400 ceph-object-corpus: kraken objects Signed-off-by: Sage Weil commit 4ff0cb953e190806f7b9e6791d300bd22ad281d9 Author: Sage Weil Date: Fri May 5 16:31:02 2017 -0400 test/encoding/generate-corpus-objects: fix typo Signed-off-by: Sage Weil (cherry picked from commit b89dff472c86c435fcee767ec9d992b308c4d816) commit 34e04de108004eccab82d6f791fbffc9ec97ab77 Author: Sage Weil Date: Fri May 5 09:19:23 2017 -0400 test/encoding/generate-corpus-objects.sh: simplify object corpus population Script that generates corpus objects. Signed-off-by: Sage Weil (cherry picked from commit b38b663a71fdc37a2d5c3341836b6716d576cecc) commit 260352405e82009132755105d78a885ae327a418 Author: Sage Weil Date: Fri May 5 15:19:05 2017 -0400 qa/workunits/rgw/run-s3tests.sh: run s3-tests This works out of the box with a vstart environment and RGW=1 ../src/vstart.sh -n -l PATH=bin:$PATH ../qa/workunits/rgw/run-s3tests.sh Signed-off-by: Sage Weil (cherry picked from commit 40fe1d181cf8d31b80db6f8f99394bd66d2affd5) commit 2b107f91cefbdc6ab173890c3ccf3693768bc76c Author: Sage Weil Date: Thu May 4 18:33:21 2017 -0400 include/denc: support ENCODE_DUMP Signed-off-by: Sage Weil (cherry picked from commit 97987b4ace737b86a4dc2dce94ce9d330fbe903c) commit 62bb2086af2f0d63c081a8a0b9d1de8d1c1179a7 Author: Sage Weil Date: Fri Apr 14 13:21:38 2017 -0400 osd: fix occasional MOSDMap leak _committed_osd_maps() may return early (without putting the ref) on shutdown. Fixes: http://tracker.ceph.com/issues/18293 Signed-off-by: Sage Weil (cherry picked from commit a74632f964e10a57fb8065aec90dc7b69bddba5b) commit d51b7553919233d2f18d5fa79a0ff3c09e71ba8a Author: Jason Dillaman Date: Mon May 1 21:06:19 2017 -0400 cls_rbd: default initialize snapshot namespace for legacy clients Creating a snapshot on >=Kraken OSDs using <=Jewel clients can result in an improperly initialized snapshot namespace. As a result, attempting to remove the snapshot using a >=Kraken client will result in an -EINVAL error. Fixes: http://tracker.ceph.com/issues/19413 Signed-off-by: Jason Dillaman (cherry picked from commit 03b0b03071f3e04754896664c69f73759ddb907a) Conflicts: src/cls/rbd/cls_rbd.h: trivial resolution commit 6b1782959ee29de6eff9a2d2c81a108485347bbd Author: Sage Weil Date: Fri Apr 28 18:03:59 2017 -0400 mgr/DaemonServer: do not crash on bad authorizer Signed-off-by: Sage Weil commit 33c211bf9868156e41d7196cbac5dc6bff418248 Author: Greg Farnum Date: Mon Apr 17 14:32:38 2017 -0700 PendingReleaseNotes: discuss snap trim improvements Signed-off-by: Greg Farnum commit 911a894d64616b2cb5bfe92e3366e8033fb0f59e Author: Greg Farnum Date: Fri Apr 7 15:45:12 2017 -0700 PrimaryLogPG: reimplement osd_snap_trim_sleep within the state machine Rather than blocking the main op queue, just pause for that amount of time between state machine cycles. Also, add osd_snap_trim_sleep to a few of the thrasher yamls. Signed-off-by: Samuel Just (cherry picked from commit 2ed7759cfeb03e71f0fbd98fe7c2db2bb741861c) Conflicts: src/osd/PrimaryLogPG.cc Signed-off-by: Greg Farnum commit 02a2ef41ae54dac7d903bc5753e6702582d55dd4 Author: Samuel Just Date: Thu Jan 26 15:41:21 2017 -0800 rados: check that pool is done trimming before removing it Signed-off-by: Samuel Just (cherry picked from commit 4aebf59d906fa3e03d21bdac182f89fe3cd4c802) commit 95306559db139ad3cda3ee2eac36195582ff9329 Author: Samuel Just Date: Thu Jan 26 15:05:14 2017 -0800 osd/: add PG_STATE_SNAPTRIM[_WAIT] to expose snap trim state to user Signed-off-by: Samuel Just (cherry picked from commit c2eac34c86517e410eb4842d8b8085da7d8d7973) commit 995ef4a6b01635ef09532487f7a4ea5fc90e721d Author: Greg Farnum Date: Fri Apr 7 15:41:18 2017 -0700 osd/PrimaryLogPG: limit the number of concurrently trimming pgs This patch introduces an AsyncReserver for snap trimming to limit the number of pgs on any single OSD which can be trimming, as with backfill. Unlike backfill, we don't take remote reservations on the assumption that the set of pgs with trimming work to do is already well distributed, so it doesn't seem worth the implementation overhead to get reservations from the peers as well. Signed-off-by: Samuel Just (cherry picked from commit 21cc515adfb225ba70f1d80b1b76f0345c214c22) Conflicts: src/osd/PrimaryLogPG.cc src/osd/PrimaryLogPG.h Signed-off-by: Greg Farnum commit 49f99c3b4d9051bccb34117075348e3ea58804db Merge: 0d6953ec50 e9a10eaccd Author: Nathan Cutler Date: Wed Apr 26 19:53:49 2017 +0200 Merge pull request #14506 from smithfarm/wip-19119-kraken kraken: doc: PendingReleaseNotes: warning about 'osd rm ...' and #13733 Reviewed-by: Sage Weil commit 0d6953ec501f0ec6c9a5a72b532e99554dfe5d09 Merge: 557fee8d0d 8d0c2297f1 Author: Nathan Cutler Date: Wed Apr 26 19:28:58 2017 +0200 Merge pull request #14692 from smithfarm/wip-fix-hammer-jewel-x qa/suites/upgrade: add tiering test to hammer-jewel-x Reviewed-by: Kefu Chai commit 557fee8d0dad8847728820b1d345e2d9ed9ab433 Merge: e85b09f415 543e4b9060 Author: Nathan Cutler Date: Mon Apr 24 22:22:14 2017 +0200 Merge pull request #14425 from smithfarm/wip-19564-kraken kraken: build/ops: ceph-base missing dependency for psmisc in Ubuntu Xenial Reviewed-by: Ken Dreyer commit e85b09f4158296dbc559d3b2e0816364ca1c448c Merge: d2c17adc14 8c2f590338 Author: Nathan Cutler Date: Sat Apr 22 10:55:16 2017 +0200 Merge pull request #13494 from shinobu-x/wip-18516-kraken kraken: build/ops: systemd: Start OSDs after MONs Reviewed-by: Nathan Cutler commit d2c17adc140d2358512c4c00e079299ae7f31f74 Merge: 9aa180b7f1 02afe9855a Author: Nathan Cutler Date: Fri Apr 21 17:36:42 2017 +0200 Merge pull request #14588 from asheplyakov/19618-kraken kraken: mon/MonClient: make get_mon_log_message() atomic Reviewed-by: Kefu Chai commit 9aa180b7f13674a3de28115347534540ecf6ae38 Merge: 8e2e3d8292 914d8a6109 Author: Nathan Cutler Date: Fri Apr 21 17:12:29 2017 +0200 Merge pull request #13543 from shinobu-x/wip-18954-kraken kraken: ceph-disk prepare get wrong group name in bluestore Reviewed-by: Loic Dachary commit 8e2e3d8292f93812933c049f9489871f249c9dfa Merge: e68d3e14b8 d8946abf07 Author: Nathan Cutler Date: Fri Apr 21 08:53:25 2017 +0200 Merge pull request #12746 from SUSE/wip-18387-kraken kraken: tests: use ceph-kraken branch for s3tests Reviewed-by: Orit Wasserman commit 6b428b77f287b3a2b9b7061e3da3279381f7d3aa Author: Vasu Kulkarni Date: Tue Apr 11 13:51:47 2017 -0700 use sudo to check check health Signed-off-by: Vasu Kulkarni (cherry picked from commit 7af157ad4ce7f7e2b8de97ee10eeaf64b9099bc0) commit 80c47c87e5c060d45ea704ac31709bb061d79db6 Author: Vasu Kulkarni Date: Wed Mar 29 09:27:20 2017 -0700 Add reboot case for systemd test test systemd units restart after reboot Signed-off-by: Vasu Kulkarni (cherry picked from commit 7b587304a54d9b21041ffdfbc85fad8d87859c49) commit a6c2d7feb3248a0bbe0399323bb2fe11e23d15e1 Author: Vasu Kulkarni Date: Wed Mar 29 09:56:11 2017 -0700 Fix distro's, point to latest version Signed-off-by: Vasu Kulkarni (cherry picked from commit 1947648669971c1bd1ca189870ed9b25bbd48d3a) commit 8d0c2297f1dbe95c236357861026263677aea0c5 Author: Kefu Chai Date: Mon Mar 6 15:50:59 2017 +0800 qa/suites/upgrade: add tiering test to hammer-jewel-x Related: http://tracker.ceph.com/issues/19185 Signed-off-by: Kefu Chai (cherry picked from commit e8c04a027b5d108b30e63ab759760add57d43198) commit e68d3e14b8ea42baa30ca4dd836c1651deb99dcd Merge: 766a55ac87 1067764418 Author: Nathan Cutler Date: Thu Apr 20 21:12:42 2017 +0200 Merge pull request #14620 from smithfarm/wip-19659-kraken kraken: librbd: corrected resize RPC message backwards compatibility Reviewed-by: Jason Dillaman commit 766a55ac87cb7c08f1316cf7f0fd6bca834ad269 Merge: 01804c78e0 55a15b18e7 Author: Nathan Cutler Date: Thu Apr 20 19:06:35 2017 +0200 Merge pull request #14574 from smithfarm/wip-19620-kraken kraken: cephfs: MDS server crashes due to inconsistent metadata. Reviewed-by: John Spray commit 01804c78e0af6fd518e8ad5de2bf39e6743e3039 Merge: 1866332928 93efc1e948 Author: Nathan Cutler Date: Thu Apr 20 19:05:44 2017 +0200 Merge pull request #14573 from smithfarm/wip-19483-kraken kraken: cephfs: No output for ceph mds rmfailed 0 --yes-i-really-mean-it command Reviewed-by: John Spray commit 18663329283c9df32bd42e1dc41cff10ba2e7826 Merge: 9ff3560ab4 941a356fa7 Author: Nathan Cutler Date: Thu Apr 20 19:04:34 2017 +0200 Merge pull request #14572 from smithfarm/wip-19335-kraken kraken: cephfs: MDS heartbeat timeout during rejoin, when working with large amount of caps/inodes Reviewed-by: John Spray commit 9ff3560ab4c4dfa35983beb845331f8fb261270b Merge: 5554f5a82a 473ad4ea2a Author: Nathan Cutler Date: Thu Apr 20 19:03:42 2017 +0200 Merge pull request #14571 from smithfarm/wip-19045-kraken kraken: cephfs: buffer overflow in test LibCephFS.DirLs Reviewed-by: John Spray commit 5554f5a82ad2c78b93651aeef5f37e4ffb4586f1 Merge: cc49802d76 64616bb2a5 Author: Nathan Cutler Date: Thu Apr 20 19:02:54 2017 +0200 Merge pull request #14570 from smithfarm/wip-18950-kraken kraken: cephfs: mds/StrayManager: avoid reusing deleted inode in StrayManager::_purge_stray_logged Reviewed-by: John Spray commit cc49802d769e3a59664fc5ef80d20171c07693df Merge: 0c51569478 2d5d8f2767 Author: Nathan Cutler Date: Thu Apr 20 19:02:02 2017 +0200 Merge pull request #14569 from smithfarm/wip-18899-kraken kraken: cephfs: test_open_inode fails Reviewed-by: John Spray commit 0c51569478ebb75defb6ba059f60007db597016a Merge: 63fd2b9884 a7831ccf79 Author: Nathan Cutler Date: Thu Apr 20 19:00:51 2017 +0200 Merge pull request #14568 from smithfarm/wip-18706-kraken kraken: cephfs: fragment space check can cause replayed request fail Reviewed-by: John Spray commit 63fd2b9884eeb7551e08875cb6421a79d22e9674 Merge: f1affa93d1 eb12e3a752 Author: Nathan Cutler Date: Thu Apr 20 18:59:25 2017 +0200 Merge pull request #14567 from smithfarm/wip-18700-kraken kraken: cephfs: client: fix the cross-quota rename boundary check conditions Reviewed-by: John Spray commit f1affa93d15f5f37910debe73eee2b4c8774fe2c Merge: 2887fb06aa 95335098d5 Author: Nathan Cutler Date: Thu Apr 20 18:58:08 2017 +0200 Merge pull request #14566 from smithfarm/wip-18616-kraken kraken: cephfs: segfault in handle_client_caps Reviewed-by: John Spray commit 2887fb06aa25aeb1b7a845b288ba4a556cab74bc Merge: 655ab2a76f 91bec618f8 Author: Nathan Cutler Date: Thu Apr 20 18:57:08 2017 +0200 Merge pull request #14565 from smithfarm/wip-18566-kraken kraken: cephfs: MDS crashes on missing metadata object Reviewed-by: John Spray commit 655ab2a76f29c3a06e7d7e9bacd3b51b2725952e Merge: bb08c04f26 a51e9b5af9 Author: Nathan Cutler Date: Thu Apr 20 18:56:04 2017 +0200 Merge pull request #14564 from smithfarm/wip-18562-kraken kraken: cephfs: test_client_recovery.TestClientRecovery fails Reviewed-by: John Spray commit bb08c04f26f1d05eedbbdc7cd23f51314edc13cf Merge: e3db135268 ba7b6ced0a Author: Nathan Cutler Date: Thu Apr 20 18:55:01 2017 +0200 Merge pull request #14563 from smithfarm/wip-18552-kraken kraken: cephfs: ceph-fuse crash during snapshot tests Reviewed-by: John Spray commit e3db1352688fc68063707882cf0c26f34db4fabf Merge: 7fda19ee7e 3bbd3c5dc3 Author: Nathan Cutler Date: Thu Apr 20 18:18:59 2017 +0200 Merge pull request #14521 from smithfarm/wip-19462-kraken kraken: doc: rgw: admin ops: fix the quota section Reviewed-by: Casey Bodley commit 7fda19ee7e8a80f0b34e9d539746c973c8c45707 Merge: d1f066b7b5 d5398effcf Author: Nathan Cutler Date: Thu Apr 20 14:52:45 2017 +0200 Merge pull request #14641 from smithfarm/wip-19693-kraken kraken: [test] test_notify.py: rbd.InvalidArgument: error updating features for image test_notify_clone2 Reviewed-by: Jason Dillaman commit d1f066b7b58343e48ddaff398be7c799323fac4d Merge: d82f0cd250 a0ad2d11ce Author: Nathan Cutler Date: Thu Apr 20 14:51:45 2017 +0200 Merge pull request #14533 from smithfarm/wip-18501-kraken kraken: rbd-mirror: potential race mirroring cloned image Reviewed-by: Jason Dillaman commit d82f0cd2505dd06507e1a08c30fa20fdbb024847 Merge: 7638d3d0cd d826600ee2 Author: Nathan Cutler Date: Thu Apr 20 14:50:56 2017 +0200 Merge pull request #14534 from smithfarm/wip-18549-kraken kraken: rbd: 'metadata_set' API operation should not change global config setting Reviewed-by: Jason Dillaman commit 7638d3d0cd0b50bc7ae9cb537c7465152cf9e476 Merge: 2c1d5db604 b9258ccfd8 Author: Nathan Cutler Date: Thu Apr 20 14:50:11 2017 +0200 Merge pull request #14536 from smithfarm/wip-18557-kraken kraken: rbd: 'rbd bench-write' will crash if --io-size is 4G Reviewed-by: Jason Dillaman commit 2c1d5db6048ed6dd239af0bdbfa79cbc14432fcc Merge: 8794053597 e3a230cb19 Author: Nathan Cutler Date: Thu Apr 20 14:49:24 2017 +0200 Merge pull request #14537 from smithfarm/wip-18601-kraken kraken: rbd: Add missing parameter feedback to 'rbd snap limit' Reviewed-by: Jason Dillaman commit 87940535979da874b9b44e037b27d869972efae2 Merge: b457cbd768 bafa2b0c83 Author: Nathan Cutler Date: Thu Apr 20 14:23:08 2017 +0200 Merge pull request #14538 from smithfarm/wip-18632-kraken kraken: rbd: [qa] crash in journal-enabled fsx run Reviewed-by: Jason Dillaman commit b457cbd768b12701532bfa57304e5c25e6b49528 Merge: 24e6d0e440 3112ab5726 Author: Nathan Cutler Date: Thu Apr 20 14:19:59 2017 +0200 Merge pull request #14622 from smithfarm/wip-19037-kraken kraken: rbd-mirror: deleting a snapshot during sync can result in read errors Reviewed-by: Jason Dillaman commit 24e6d0e44080558f956d2c8446cf60f7137649bc Merge: 13de280cfe 551ce273a1 Author: Nathan Cutler Date: Thu Apr 20 14:18:48 2017 +0200 Merge pull request #14545 from smithfarm/wip-19324-kraken kraken: rbd: [api] temporarily restrict (rbd_)mirror_peer_add from adding multiple peers Reviewed-by: Jason Dillaman commit fa2753245d056f39f0f6996988e193bf4917bc52 Author: Greg Farnum Date: Fri Apr 7 14:33:20 2017 -0700 osd: pglog: with config, don't assert in the presence of stale divergent_priors Fixes: http://tracker.ceph.com/issues/17916 Signed-off-by: Greg Farnum commit d5398effcfacb6d5436ae32e1ee40872efd5b504 Author: Jason Dillaman Date: Wed Apr 19 09:26:31 2017 -0400 test: rbd master/slave notify test should test active features Fixes: http://tracker.ceph.com/issues/19692 Signed-off-by: Jason Dillaman (cherry picked from commit 0dcba41cba96566d0b8da54cf0316d523b88ded2) commit d02d0e5d97fd7d9b3eac1e2ba55776d2c9f7f8ee Author: Jason Dillaman Date: Wed Apr 19 09:12:04 2017 -0400 qa/suites: client-upgrade/jewel-client-x should test multiple features Signed-off-by: Jason Dillaman (cherry picked from commit 7ab93b7779b293ceda325a6d1cdb446cf624a6e3) commit 13de280cfe7379440185fefcb71a6d7d4a30e65f Merge: 5d604d058b b8d988f7f1 Author: Zack Cerza Date: Tue Apr 18 14:17:02 2017 -0600 Merge pull request #14487 from zmc/wip-kraken-workunit qa/tasks/workunit: Backport repo fixes from master commit 5d604d058b0b1f7b7659390093437f109fcb9af7 Merge: 959248d249 c2232643d0 Author: Nathan Cutler Date: Tue Apr 18 22:09:19 2017 +0200 Merge pull request #14531 from smithfarm/wip-18493-kraken kraken: rbd-mirror: sporadic image replayer shut down failure Reviewed-by: Jason Dillaman commit 959248d2497a48f98c0a4fd818fcce79c994ca61 Merge: c195418b4f e91de89b0d Author: Nathan Cutler Date: Tue Apr 18 22:08:34 2017 +0200 Merge pull request #14532 from smithfarm/wip-18495-kraken kraken: rbd: Possible deadlock performing a synchronous API action while refresh in-progress Reviewed-by: Jason Dillaman commit 10677644181356f561cc10a19d76833d55917a43 Author: Jason Dillaman Date: Tue Apr 18 10:05:53 2017 -0400 librbd: corrected resize RPC message backwards compatibility Commit d1f2c557 incorrectly changed the order of variables within the payload. This resulted in breaking the resize RPC message with older versions of Ceph. Fixes: http://tracker.ceph.com/issues/19636 Signed-off-by: Jason Dillaman (cherry picked from commit 9db305a99fd945059a173f5ae8ae61744dd28615) commit c195418b4f240a8f4bf1b1739af9a64ba046c2d2 Merge: 432bc8b416 81f3d86032 Author: Nathan Cutler Date: Tue Apr 18 16:58:39 2017 +0200 Merge pull request #14095 from shinobu-x/wip-19319-kraken kraken: RadosImport::import should return an error if Rados::connect fails Reviewed-by: David Zafman commit 8f81bb33c7791c46db5e4db1ee45ab02a70580d9 Author: Casey Bodley Date: Wed Apr 5 16:19:57 2017 -0400 rgw: fix for null version_id in fetch_remote_obj() commit 8b43c9781206c22d9aedb4beb8d669bf1e23169f fixed the wrong use of the dest_obj's version, but removed the check for "null" version Signed-off-by: Casey Bodley (cherry picked from commit 915370776df5b964c2ee8d9f9329562919eef8d5) commit ad2b140e01462d3dfe58fd03879f3868e561f165 Author: Zhang Shaowen Date: Fri Mar 17 16:26:56 2017 +0800 rgw: version id doesn't work in fetch_remote_obj Signed-off-by: Zhang Shaowen (cherry picked from commit 8b43c9781206c22d9aedb4beb8d669bf1e23169f) Conflicts: src/rgw/rgw_rados.cc: trivial: dest_obj.key.instance in master versus dest_obj.get_instance() in Jewel commit 8d64dd4f29e1b0af51e7938d1aa363e4ff710154 Author: Loic Dachary Date: Thu Apr 13 23:49:50 2017 +0200 ceph-disk: enable directory backed OSD at boot time https://github.com/ceph/ceph/commit/539385b143feee3905dceaf7a8faaced42f2d3c6 introduced a regression preventing directory backed OSD from starting at boot time. For device backed OSD the boot sequence starts with ceph-disk@.service and proceeds to systemctl enable --runtime ceph-osd@.service where the --runtime ensure ceph-osd@12 is removed when the machine reboots so that it does not compete with the ceph-disk@/dev/sdb1 unit at boot time. However directory backed OSD solely rely on the ceph-osd@.service unit to start at boot time and will therefore fail to boot. The --runtime flag is selectively set for device backed OSD only. Fixes: http://tracker.ceph.com/issues/19628 Signed-off-by: Loic Dachary (cherry picked from commit f425a127b7487d2093c8c943f0bcdec3d673d601) commit 02afe9855a4d04fe56951a9e9ffd3e32537ff77e Author: Kefu Chai Date: Mon Apr 10 14:53:46 2017 +0800 mon/MonClient: make get_mon_log_message() atomic * LogClient: move reset_session() into get_mon_log_message() and add a "flush" param to the latter. so it can get_mon_log_message() atomically. otherwise another call changing the log queue could sneak in between reset_session() and get_mon_log_message(). * MonClient: add a "flush" param to do_send() so we can reset the LogClient session once we are connected to a monitor. Fixes: http://tracker.ceph.com/issues/19427 Signed-off-by: Kefu Chai (cherry picked from commit 5215e291da2b527d85e129eda86043490843178e) Conflicts: src/mon/MonClient.cc: handle_auth: replaced 'log_client->reset_session(); send_log();' sequence with newly introduced 'send_log(true);' like the original patch does commit 432bc8b4168e70a68e53795dea538882f83a9a8e Merge: 8eb10440ce c2cc96e6c1 Author: Nathan Cutler Date: Sun Apr 16 04:13:12 2017 +0200 Merge pull request #14323 from shinobu-x/wip-19371-kraken kraken: common: monitor creation with IPv6 public network segfaults Reviewed-by: Kefu Chai Reviewed-by: Nathan Cutler commit 8eb10440ce144ddab2997ac0b8a868d28de932fe Merge: 74907feb3f 1d25327b00 Author: Nathan Cutler Date: Sat Apr 15 10:08:15 2017 +0200 Merge pull request #13284 from shinobu-x/wip-18599-kraken kraken: osd: os/bluestore: fix statfs to not include DB partition in free space Reviewed-by: Sage Weil Reviewed-by: Nathan Cutler commit 74907feb3fda64f9e9a1f7410ce71683a40f417c Merge: fc5c4cd2c0 0e0d149895 Author: Nathan Cutler Date: Sat Apr 15 09:57:21 2017 +0200 Merge pull request #13046 from SUSE/wip-18554-kraken kraken: mon: peon wrongly delete routed pg stats op before receive pg stats ack Reviewed-by: Kefu Chai commit fc5c4cd2c01fa97f65b4bbad0d2cfe6dc7e3f9a9 Merge: 19b8a9f1f8 7e30b630e2 Author: Nathan Cutler Date: Sat Apr 15 00:30:39 2017 +0200 Merge pull request #14480 from dillaman/wip-19467-kraken kraken: librbd: is_exclusive_lock_owner API should ping OSD Reviewed-by: Jason Dillaman Reviewed-by: Mykola Golub commit 19b8a9f1f87b2b9ff69eb67b38e1f2e161ea1c1d Merge: 91fbfaf3bd d4a43e1fd5 Author: Nathan Cutler Date: Sat Apr 15 00:28:19 2017 +0200 Merge pull request #13877 from smithfarm/wip-19178-kraken kraken: rgw: anonymous user error code of getting object is not consistent with SWIFT Reviewed-by: Radoslaw Zarzynski commit 91fbfaf3bdbb9dfcf6704effc271d5038263676d Merge: abcf1b1e28 b39bef0961 Author: Nathan Cutler Date: Sat Apr 15 00:25:51 2017 +0200 Merge pull request #13866 from smithfarm/wip-19157-kraken kraken: rgw: health check errors out incorrectly Reviewed-by: Radoslaw Zarzynski commit abcf1b1e28c75feb702b100bbc0ca0ff9c4d14f6 Merge: cbcb932d3d 47cf8f6699 Author: Nathan Cutler Date: Sat Apr 15 00:22:22 2017 +0200 Merge pull request #13843 from smithfarm/wip-19146-kraken kraken: rgw: a few cases where rgw_obj is incorrectly initialized Reviewed-by: Orit Wasserman commit cbcb932d3d40118101df350694eacbe2cfb22232 Merge: 6702056568 03b1ab4b13 Author: Nathan Cutler Date: Sat Apr 15 00:20:29 2017 +0200 Merge pull request #13829 from smithfarm/wip-18898-kraken kraken: no http referer info in container metadata dump in swift API Reviewed-by: Radoslaw Zarzynski commit 670205656847742f1e9d46b152c66f725dfba5dc Merge: 841216b781 c8d70ee012 Author: Nathan Cutler Date: Sat Apr 15 00:19:00 2017 +0200 Merge pull request #13780 from smithfarm/wip-18896-kraken kraken: should parse the url to http host to compare with the container referer acl Reviewed-by: Radoslaw Zarzynski commit 841216b781e229205476f633eae75355f87c27ae Merge: eb84959bac bfc058d8f9 Author: Nathan Cutler Date: Sat Apr 15 00:15:15 2017 +0200 Merge pull request #12986 from Werkov/rgw-fix-tempurl-url-encoding kraken: rgw: Use decoded URI when verifying TempURL Reviewed-by: Radoslaw Zarzynski commit 55a15b18e7a2188f5ff74b5fa26ef0316ec9c005 Author: John Spray Date: Wed Mar 29 19:38:37 2017 +0100 tools/cephfs: set dir_layout when injecting inodes When we left this as zero, the MDS would interpret it was HASH_LINUX rather than the default HASH_RJENKINS. Potentially that could cause problems if there perhaps were already dirfrags in the metadata pool that were set up using rjenkins. Mainly it just seems more appropriate to explicitly set this field rather than hit the fallback behaviour. Related: http://tracker.ceph.com/issues/19406 Signed-off-by: John Spray (cherry picked from commit 7d6d542885bd29b71214f9ca52bd26e9183c5d01) commit 93efc1e9484ea41f3eb831dcb3d19d63b044ef70 Author: John Spray Date: Thu Mar 9 13:15:46 2017 +0000 mon: fix hiding mdsmonitor informative strings Local `stringstream ss` declarations were hiding the real variable used to feed back to the user. Fixes: http://tracker.ceph.com/issues/16709 Signed-off-by: John Spray (cherry picked from commit 00404ae9bd4cce0518a44d36d2d6a5612f4f9d04) commit 941a356fa77ffc2f41d5054a8e3f43b46523008e Author: John Spray Date: Mon Mar 6 11:51:31 2017 +0000 mds: reset heartbeat in export_remaining_imported_caps This loop can be very long. Fixes: http://tracker.ceph.com/issues/19118 Signed-off-by: John Spray (cherry picked from commit 85071f1509beba4a390730e6a3a4332484646d63) commit 5e19692bb68049ea8fed0c39534fe8abb10a5378 Author: John Spray Date: Mon Mar 6 11:24:50 2017 +0000 mds: heartbeat_reset in dispatch Previously we only heartbeated in tick. However, our locking is not guaranteed to be fair, so on a super-busy dispatch queue it may be possible for the heartbeat to time out while the tick() function is waiting for mds_lock. Fixes: http://tracker.ceph.com/issues/19118 Signed-off-by: John Spray (cherry picked from commit 819394549af10532419d88742fae3a69d2ea487d) commit 473ad4ea2a35c74d89605b5579a4ff3866134c2d Author: Yan, Zheng Date: Wed Feb 15 11:45:26 2017 +0800 test/libcephfs: avoid buffer overflow when testing ceph_getdents() The buffer size should be at least "2 * sizeof(struct dirent)". Otherwise, the code that checks dentry '..' overflow. Fixes: http://tracker.ceph.com/issues/18941 Signed-off-by: "Yan, Zheng" (cherry picked from commit fa6671345b8f3a82dcd232f99e55a982b0a641f1) commit 64616bb2a5b84f8518ecbb9878a5ba3abaaf87ed Author: Zhi Zhang Date: Fri Feb 10 10:56:46 2017 +0800 mds/StrayManager: aviod reusing deleted inode in StrayManager::_purge_stray_logged Signed-off-by: Zhi Zhang (cherry picked from commit 4978e57419482384279d7e784a625f5e5c10961a) commit 2d5d8f27679d40ff962c8329026b2d75a3fa7c78 Author: John Spray Date: Wed Feb 1 00:38:08 2017 +0000 tasks/cephfs: switch open vs. write in test_open_inode Do the write after opening the file, so that we get good behaviour wrt the change in Mount.open_background that uses file existence to confirm that the open happened. Signed-off-by: John Spray (cherry picked from commit a027dba78fc8bc84ae39d7998b386ce21c01e1bf) commit 284cd5065f871fc7aeb134d0f9dd48ba9d1f12bf Author: John Spray Date: Thu Jan 26 16:48:58 2017 +0000 qa: fix race in Mount.open_background Previously a later remote call could end up executing before the remote python program in open_background had actually got as far as opening the file. Fixes: http://tracker.ceph.com/issues/18661 Signed-off-by: John Spray (cherry picked from commit c6d91dd91252e703d08b8ac62ac6a47ee82c0bed) commit a7831ccf795b31705a66be89beef63b0bbe817d3 Author: Yan, Zheng Date: Wed Jan 25 15:28:23 2017 +0800 mds: don't purge strays when mds is in clientreplay state MDS does not trim log when it's in clientreplay state. If mds hang at clientreplay state (due to bug), purging strays can submit lots of log events and create very large mds log. Signed-off-by: "Yan, Zheng" (cherry picked from commit 86bbc7fff02668077f27d0924ba3efe6544b77f6) commit 224745491e5c9e495adce5abbf7731f7fae1ddc5 Author: Yan, Zheng Date: Wed Jan 25 11:03:45 2017 +0800 mds: skip fragment space check for replayed request when handling replayed request, stray directory can be different from the stray directory used by the original request. The fragment space check for stray directory can fail. Fixes: http://tracker.ceph.com/issues/18660 Signed-off-by: "Yan, Zheng" (cherry picked from commit afe889cbc5baab196567c2aad01f49fe90901fda) commit eb12e3a7524fcbc009cabda333a6a958390743bd Author: Greg Farnum Date: Wed Dec 14 12:09:44 2016 -0800 client: fix the cross-quota rename boundary check conditions We were previously rejecting a rename if either of the involved directories was a quota root, even if the other directory was part of the same quota "tree". What we really want to do is identify the correct quota root (whether local or ancestral) for each directory and compare them. So now we do. Signed-off-by: Greg Farnum (cherry picked from commit 8e8892aa46accb519faa4bb9fecf66618f1b11b2) commit 95335098d5fa1a8e308f697a6bcef6ce3e090ac9 Author: Yan, Zheng Date: Fri Jan 6 15:42:52 2017 +0800 mds: fix null pointer dereference in Locker::handle_client_caps Locker::handle_client_caps delays processing cap message if the corresponding inode is freezing or frozen. When the message gets processed, client can have already closed the session. Fixes: http://tracker.ceph.com/issues/18306 Signed-off-by: Yan, Zheng (cherry picked from commit e281a0b9c1fdeaf09f1b01f34cecd62e4f49d02e) commit 91bec618f87d4fc5dc0619d1b980c14ec7527d48 Author: Yan, Zheng Date: Wed Jan 11 15:50:52 2017 +0800 qa/tasks: add test_open_ino_errors Validate that errors encountered during opening inos are properly propagated Signed-off-by: Yan, Zheng (cherry picked from commit 6526ecc084733b34129aa1f21085fa41fb53b785) commit 4d5775092d29c2eb3c2d88d1fff93ce5af4779dd Author: Yan, Zheng Date: Tue Jan 3 11:11:12 2017 +0800 mds: propagate error encountered during opening inode by number Fixes: http://tracker.ceph.com/issues/18179 Signed-off-by: Yan, Zheng (cherry picked from commit 2213cc2dcc0e8fb01bcae3863d0d8a4a1fd8873f) commit a51e9b5af9fbdb5b81ec20e30dfda2222d6ac418 Author: Yan, Zheng Date: Mon Jan 9 20:47:37 2017 +0800 qa/tasks/cephfs: fix kernel force umount Fixes: http://tracker.ceph.com/issues/18396 Signed-off-by: Yan, Zheng (cherry picked from commit 4cdeeaac105d7824452a8b578632b7371275b45c) commit ba7b6ced0a69991bf4f095172575dc536736d40d Author: Yan, Zheng Date: Tue Jan 10 17:16:40 2017 +0800 client: fix Client::handle_cap_flushsnap_ack() crash Struct CapSnap holds a reference to its parent inode. So erasing struct CapSnap from Inode::cap_snaps may drop inode's last reference. The inode gets freed in the middle of erasing struct CapSnap Fixes: http://tracker.ceph.com/issues/18460 Signed-off-by: Yan, Zheng (cherry picked from commit 525c52fd491ed1ced385c8047872e3f557f8423f) commit 551ce273a132d4e7b59b818377568d40cf597f85 Author: Jason Dillaman Date: Fri Mar 10 10:56:38 2017 -0500 rbd: prevent adding multiple mirror peers to a single pool The rbd-mirror daemon does not currently support replication from multiple peers. Until that is supported, add a temporary restriction to prevent confusion. Fixes: http://tracker.ceph.com/issues/19256 Signed-off-by: Jason Dillaman (cherry picked from commit c0c9d1014d57b3d5b95e7513fcc38d04b9ea5165) commit 3112ab57269847233c1e006ca05ba02071d579b0 Author: Jason Dillaman Date: Tue Feb 21 15:33:01 2017 -0500 rbd-mirror: retry object copy after -ENOENT error Fixes: http://tracker.ceph.com/issues/18990 Signed-off-by: Jason Dillaman (cherry picked from commit b4f36d5dc3f4f3cbb23f61cbb945b222248a50df) commit 46ba56025c283a70b0f384ef38f47542f0885c5b Author: Jason Dillaman Date: Tue Feb 21 13:09:39 2017 -0500 rbd-mirror: object copy should always reference valid snapshots If a remote snapshot is deleted while an image sync is in-progress, associate the read request against the most recent, valid remote snapshot for a given snapshot object clone. Signed-off-by: Jason Dillaman (cherry picked from commit 9a91efc3047963364944f8be91cee8e8f6afc49a) commit d1ac70a4d06ed07a66e7886c00c712a26fd72e9d Author: Jason Dillaman Date: Tue Feb 21 11:52:00 2017 -0500 rbd-mirror: replace complex object op tuple with struct Signed-off-by: Jason Dillaman (cherry picked from commit 0c181527c0e151784a0f7c466aaa70b0772f91b1) commit bafa2b0c83d32b97a6dd85e681f2344384c539ac Author: Jason Dillaman Date: Fri Jan 20 14:26:43 2017 -0500 journal: don't hold future lock during assignment It's possible that the future raced with its owner and reaches an empty reference count. This was resulting in the future being destructed while its lock was still held. Fixes: http://tracker.ceph.com/issues/18618 Signed-off-by: Jason Dillaman (cherry picked from commit 0f21ceef8336e35ca16148a9d58f511037911418) commit e3a230cb19d62bdcd18f3685d384d76e83b42ec5 Author: tang.jin Date: Mon Jan 16 22:28:23 2017 +0800 rbd: add error prompt when input command 'snap set limit' is incomplete Signed-off-by: Tang Jin (cherry picked from commit 8860028f508a9be5a08f512022cfb042021fd19f) commit b9258ccfd836d49d64c8de3cb614a4f0ce9806aa Author: Gaurav Kumar Garg Date: Tue Jan 10 15:25:13 2017 +0100 rbd: bench-write should return error if io-size >= 4G Currently if user perform bench-write with io-size > 4G then its crashing because currently during memory allocation bufferptr taking size of buffer as a unsigned and io-size > 4G will overflow with unsigned. so during memset operation it will try to set io_size size of memory area pointed by bufferptr, (bufferptr area is: (4G - io_size)), so it will cause segmentation fault. Fix is to return error if io-size >= 4G Fixes: http://tracker.ceph.com/issues/18422 Reported-by: Jason Dillaman Signed-off-by: Gaurav Kumar Garg (cherry picked from commit 6ab73e5f420e89e19b52e39dab28fa4c94e00197) commit d826600ee2c1b50675a0c8642c70ad803bca02a9 Author: Mykola Golub Date: Mon Jan 9 18:31:21 2017 +0100 librbd: metadata_set API operation should not change global config setting Fixes: http://tracker.ceph.com/issues/18465 Signed-off-by: Mykola Golub (cherry picked from commit 27465b5916b55ac3c2846c74b89f4362ad17ff1e) commit a0ad2d11cea867eb19d40a121907ec43f37e39a3 Author: Jason Dillaman Date: Tue Dec 6 15:51:51 2016 -0500 librbd: delay mirror registration when creating clones Fixes: http://tracker.ceph.com/issues/17993 Signed-off-by: Jason Dillaman (cherry picked from commit 778e1126a0343e2221446b8e13b48df5ccac263c) commit e91de89b0d62ee4869992254ff291c6a23bcafcc Author: Jason Dillaman Date: Thu Jan 5 12:12:57 2017 -0500 librbd: possible deadlock with flush if refresh in-progress Fixes: http://tracker.ceph.com/issues/18419 Signed-off-by: Jason Dillaman (cherry picked from commit b95f92a5572d3035c20eba07e76d2c825a9853f7) commit c2232643d0ee38c8148f13e69065a90c003f46e3 Author: Jason Dillaman Date: Fri Jan 6 15:59:22 2017 -0500 rbd-mirror: avoid processing new events after stop requested Fixes: http://tracker.ceph.com/issues/18441 Signed-off-by: Jason Dillaman (cherry picked from commit d30873b269441815b5fc7de14c7d9a1077a17d8d) commit faf80bc1dd332f0ccaaa1caf2679c4b2be02288d Author: Orit Wasserman Date: Wed Apr 5 13:31:08 2017 +0300 radosgw-admin: use zone id when creating a zone Fixes: http://tracker.ceph.com/issues/19498 Signed-off-by: Orit Wasserman (cherry picked from commit 3fea36d635fcba8ca584a1c0ec9f07840009402c) commit a5fe8124831daaaa67eb89555058d87600d3fff0 Author: Casey Bodley Date: Tue Mar 14 15:43:13 2017 -0400 qa: rgw task uses period instead of region-map Signed-off-by: Casey Bodley (cherry picked from commit e3e3a71d1f1fb43bb4172ce2dfac9a28ca89df0f) commit 0e964e477ac1e1a2f6f8c28b8fd0da8e9225360c Author: Casey Bodley Date: Tue Mar 14 14:18:15 2017 -0400 rgw-admin: remove deprecated regionmap commands Fixes: http://tracker.ceph.com/issues/18725 Signed-off-by: Casey Bodley (cherry picked from commit 5830c1849a0c0110d17c37784808e456e6dcb7b3) commit 03e5be4498d31c738f08d534bf8f27381fb8c12b Author: liuchang0812 Date: Fri Feb 10 18:02:03 2017 +0800 rgw: don't return skew time in pre-signed url Fixes: http://tracker.ceph.com/issues/18828 Signed-off-by: liuchang0812 (cherry picked from commit dd8b348f4aad0124e8a4457117bf3f5f76af7bdb) commit 595c31f390fda700e6e346c208849d23c74bdf5f Author: Jing Wenjun Date: Fri Feb 24 04:45:04 2017 +0800 rgw: add the remove-x-delete feature to cancel swift object expiration In openstack swift, it also support the feature to cancel the object expiration, which could be found at last point in https://docs.openstack.org/user-guide/cli-swift-set-object-expiration.html. we can remove the object expiration by set 'X-Remove-Delete-At:'. This patch also could fix the bug that when we set the object expiration and then upload the same object to the container again. The previous object expiration also works, which is not compatible with the openstack swift. Fixes: http://tracker.ceph.com/issues/19074 Signed-off-by: Jing Wenjun (cherry picked from commit 230429ebc4ac9b5840bb93c7e0eeb5edbb949106) commit 3bbd3c5dc316ebd0fffc1a4cf22f44bd01330f6e Author: hrchu Date: Wed Mar 29 02:17:04 2017 +0000 doc: rgw: correct the quota section Add the missing option and fix typo. Fixes: http://tracker.ceph.com/issues/19397 Signed-off-by: Chu, Hua-Rong (cherry picked from commit 51a88267f0d7f51aeb62092949b66b9f6c062e15) commit c3d8444890c394077177c555908ed78dc824a587 Author: Jing Wenjun Date: Wed Feb 8 15:07:43 2017 +0800 rgw: fix swift cannot disable object versioning we should be able to disable object verioning by removing its X-Versions-Location metadata header by sending an empty key value. this description can be found at No.8 in http://docs.openstack.org/user-guide/cli-swift-set-object-versions.html. Fixes: http://tracker.ceph.com/issues/18852 Signed-off-by: Jing Wenjun (cherry picked from commit 17c5a0edd2227703cec867f0f588d4eae36dfe1c) commit eb84959bac369dbb86674f9e4225ba1c836c104c Merge: 85b7bc82e2 dc8ef3508b Author: Nathan Cutler Date: Thu Apr 13 18:20:05 2017 +0200 Merge pull request #13609 from smithfarm/wip-19064-kraken kraken: AttributeError: Thrasher instance has no attribute 'ceph_objectstore_tool' Reviewed-by: Kefu Chai commit e9a10eaccd1d36e74969127fea7a595a21645b6a Author: Sage Weil Date: Wed Mar 1 13:18:44 2017 -0600 PendingReleaseNotes: warning about 'osd rm ...' and #19119 See http://tracker.ceph.com/issues/19119 Signed-off-by: Sage Weil (cherry picked from commit be96003c464481d8e84825178d600234a0d64d22) Conflicts: PendingReleaseNotes - drop "Calculation of recovery priorities has been updated" because that was included in 11.2.0 release notes - do not backport >=12.0.0 release notes - change heading to 11.2.1 commit 85b7bc82e28c6f1da08c6f6829555724d43255d2 Merge: dd44cfa736 46548c5125 Author: Nathan Cutler Date: Thu Apr 13 16:43:50 2017 +0200 Merge pull request #13790 from shinobu-x/wip-18298-kraken kraken: mon: force_create_pg could leave pg stuck in creating state Reviewed-by: Josh Durgin commit dd44cfa7368b193997bc59114e187c7f51e66cb1 Merge: d477944262 177141ff17 Author: Nathan Cutler Date: Thu Apr 13 16:41:16 2017 +0200 Merge pull request #13500 from shinobu-x/wip-18587-kraken kraken: mon: 'osd crush move ...' doesnt work on osds Reviewed-by: Josh Durgin Reviewed-by: Nathan Cutler commit d4779442620a8f6da5484be6188eeb9f636319c0 Merge: 37bb087113 33d9a50dec Author: Nathan Cutler Date: Thu Apr 13 16:39:51 2017 +0200 Merge pull request #13490 from shinobu-x/wip-18820-kraken kraken: osd: --flush-journal: sporadic segfaults on exit Reviewed-by: Josh Durgin Reviewed-by: Nathan Cutler commit 37bb087113e3c8f8f7a79ee256ae7651a5165bc2 Merge: 8a6e631b29 d8c0141a75 Author: Nathan Cutler Date: Thu Apr 13 16:35:45 2017 +0200 Merge pull request #13295 from asheplyakov/kraken-bp-18497 kraken: osd: publish PG stats when backfill-related states change Reviewed-by: Josh Durgin Reviewed-by: Sage Weil commit 8a6e631b292c56b4a577fd58cd604d31da8ba0bf Merge: b7877d1bb5 af2af7015a Author: Nathan Cutler Date: Thu Apr 13 16:30:00 2017 +0200 Merge pull request #13253 from liewegas/wip-enxio-kraken kraken: osd: do not send ENXIO on misdirected op by default Reviewed-by: Josh Durgin commit b7877d1bb5e5a38ad2cae4b0833cbefeed2547cc Merge: e1f548e38e b49a71daf5 Author: Nathan Cutler Date: Thu Apr 13 16:28:08 2017 +0200 Merge pull request #14322 from shinobu-x/wip-18619-kraken kraken: osd: ceph degraded and misplaced status output inaccurate Reviewed-by: Josh Durgin commit 7e30b630e2806c73ea503871599f958b58df7934 Author: Jason Dillaman Date: Thu Mar 16 12:28:41 2017 -0400 librbd: is_exclusive_lock_owner API should ping OSD This is required to detect if a peer has been silently blacklisted and is therefore no longer the lock owner. Fixes: http://tracker.ceph.com/issues/19287 Signed-off-by: Jason Dillaman (cherry picked from commit e15db05960a284bdf3701256722299d553cfd5aa) Conflicts: src/librbd/ManagedLock.[h|cc]: logic moved to ExclusiveLock commit e1f548e38ef38ad7dc9631ba61e35b52781dfc2f Merge: 086a4c8de1 baa42b6d7c Author: Nathan Cutler Date: Thu Apr 13 11:11:44 2017 +0200 Merge pull request #13216 from ovh/bp-osd-updateable-throttles-kraken kraken: osd: allow client throttler to be adjusted on-fly, without restart Reviewed-by: Josh Durgin commit 086a4c8de1243c2324d18cbe2d67704218065f83 Merge: 95d4b4cefd 6d719558eb Author: Nathan Cutler Date: Thu Apr 13 10:45:10 2017 +0200 Merge pull request #14321 from shinobu-x/wip-kraken-15025 kraken: osd: New added OSD always down when full flag is set Reviewed-by: Josh Durgin commit 95d4b4cefd0aff88e9c8fe25e44a4b33dfe66451 Merge: c4e1e706dc 2b7083f8dd Author: Nathan Cutler Date: Thu Apr 13 10:42:31 2017 +0200 Merge pull request #13091 from dzafman/wip-18624-kraken kraken: osd: Revert "PrimaryLogPG::failed_push: update missing as well" Reviewed-by: Josh Durgin commit c4e1e706dc8b7efd723bffd9b770b8b09c9a2300 Merge: 4ea954d75e 06093bcb91 Author: Nathan Cutler Date: Wed Apr 12 23:52:06 2017 +0200 Merge pull request #12790 from SUSE/wip-18418-kraken kraken: osd: leveldb corruption leads to Operation not permitted not handled and assert Reviewed-by: Josh Durgin commit 4ea954d75e0ff4969a5a35c77a229d43c66b9b28 Merge: a4b0d2eeb5 478d0315cb Author: Nathan Cutler Date: Wed Apr 12 23:03:40 2017 +0200 Merge pull request #13172 from smithfarm/wip-18713-kraken kraken: rgw: radosgw-admin period update reverts deleted zonegroup Reviewed-by: Casey Bodley commit a4b0d2eeb579b5a347d5a09f97eff8ca5a428101 Merge: ce6423ac2c 29f55d7dd3 Author: Nathan Cutler Date: Wed Apr 12 23:02:35 2017 +0200 Merge pull request #13176 from smithfarm/wip-18709-kraken kraken: rgw: multisite: sync status reports master is on a different period Reviewed-by: Casey Bodley commit ce6423ac2c3d92011ab5e2434a3daec82195e085 Merge: 577e3b4f23 5e15025ffb Author: Nathan Cutler Date: Wed Apr 12 23:01:42 2017 +0200 Merge pull request #13864 from smithfarm/wip-19156-kraken kraken: rgw: typo in rgw_admin.cc Reviewed-by: Casey Bodley commit 577e3b4f23cca525d49cc02b6873a69ddf7f98e3 Merge: fca689ec2f c56a3134eb Author: Nathan Cutler Date: Wed Apr 12 23:00:37 2017 +0200 Merge pull request #14137 from smithfarm/wip-19331-kraken kraken: rgw: upgrade to multisite v2 fails if there is a zone without zone info Reviewed-by: Casey Bodley commit fca689ec2f98ea40e3bc2ef7d338e73cb551934b Merge: 16f9611e05 84447a42e8 Author: Nathan Cutler Date: Wed Apr 12 22:59:18 2017 +0200 Merge pull request #14141 from smithfarm/wip-19342-kraken kraken: rgw: "period update" does not remove short_zone_ids of deleted zones Reviewed-by: Casey Bodley commit 16f9611e054e24f0e9324079b07ab2571a699caa Merge: 2fec93964c 76ce6841f7 Author: Nathan Cutler Date: Wed Apr 12 22:57:16 2017 +0200 Merge pull request #14142 from smithfarm/wip-19354-kraken kraken: rgw: multisite: some 'radosgw-admin data sync' commands hang Reviewed-by: Casey Bodley commit 2fec93964cccd43cedda95f724392f3f22bd47cb Merge: 41770bef0d 51e8ba0a06 Author: Nathan Cutler Date: Wed Apr 12 22:56:22 2017 +0200 Merge pull request #14144 from smithfarm/wip-19356-kraken kraken: rgw: when converting region_map we need to use rgw_zone_root_pool Reviewed-by: Casey Bodley commit 41770bef0d640c5cee48ec117e2c90bd3e13f442 Merge: 278ad230f3 f9e3e9f9ff Author: Yuri Weinstein Date: Wed Apr 12 10:08:40 2017 -0700 Merge pull request #14485 from zmc/wip-upgrade-openstack suites/upgrade/hammer-jewel-x: Add volumes Reviewed-by: Yuri Weinstein commit b8d988f7f19112c3cc2a8379a4ed20418414abba Author: Kefu Chai Date: Sat Apr 1 23:04:22 2017 +0800 qa/tasks/workunit.py: use "overrides" as the default settings of workunit otherwise the settings in "workunit" tasks are always overridden by the settings in template config. so we'd better follow the way of how "install" task updates itself with the "overrides" settings: it uses the "overrides" as the *defaults*. Fixes: http://tracker.ceph.com/issues/19429 Signed-off-by: Kefu Chai (cherry picked from commit 47080150a17d238f38d9da824d227393ad767aad) commit 9212c1ec702be1b927a5a10dc1514803a13d85d5 Author: Kefu Chai Date: Thu Mar 30 12:37:01 2017 +0800 tasks/workunit.py: specify the branch name when cloning a branch c1309fb failed to specify a branch when cloning using --depth=1, which by default clones the HEAD. and we can not "git checkout" a specific sha1 if it is not HEAD, after cloning using '--depth=1', so in this change, we dispatch "tag", "branch", "HEAD" using three Refspec classes. Signed-off-by: Kefu Chai Signed-off-by: Dan Mick (cherry picked from commit 9ca7ccf5f1739f731da8bf31260594aea3a2932d) commit 665f03a467c1b59eb80782e4ba3bbeb48b5d5efc Author: Dan Mick Date: Tue Mar 28 20:08:13 2017 -0700 tasks/workunit.py: when cloning, use --depth=1 Help avoid killing git.ceph.com. A depth 1 clone takes about 7 seconds, whereas a full one takes about 3:40 (much of it waiting for the server to create a huge compressed pack) Signed-off-by: Dan Mick (cherry picked from commit c1309fbef300a062138ac40eb5d3e5081b833072) commit 63153e957862ac0b46b8a205ed81d319b66ea6f5 Author: Kefu Chai Date: Mon Feb 27 16:29:40 2017 +0800 qa/tasks/workunit: use ceph.git as an alternative of ceph-ci.git for workunit repo if we run upgrade test, where, for example, "jewel" is not in ceph-ci.git repo, we should check ceph.git to clone the workunits. Signed-off-by: Kefu Chai (cherry picked from commit edceabbd47697cb4e7d21798e57a58969a92ce79) commit 471303c79a552bbd3765a6c47507fc2f2926a29d Author: Kefu Chai Date: Fri Feb 24 16:43:07 2017 +0800 qa/tasks/workunit: use the suite repo for cloning workunit as "workunits" reside in ceph/qa/workunits, it's more intuitive to respect suite-repo option when cloning workunits. Signed-off-by: Kefu Chai (cherry picked from commit 4cf28de4c9a60a4618d101f555f5a2e59b7ab109) commit 278ad230f35c7742b6b066e693335dceaf291f3d Merge: baeac21515 c4f8114c6b Author: Nathan Cutler Date: Wed Apr 12 16:42:37 2017 +0200 Merge pull request #14098 from shinobu-x/wip-19192-kraken kraken: tools: ceph-brag fails to count "in" mds Reviewed-by: Kefu Chai commit baeac21515b9f472330f72fab6762e4e2535e6ba Merge: 29eb1ec09a d91697237a Author: Nathan Cutler Date: Wed Apr 12 16:11:30 2017 +0200 Merge pull request #14155 from smithfarm/wip-18947-kraken kraken: rbd-mirror: additional test stability improvements Reviewed-by: Jason Dillaman commit 29eb1ec09ab66cf8d1cee411809ab44a5a5d13b6 Merge: c2e63ada80 6269f15223 Author: Nathan Cutler Date: Wed Apr 12 16:10:47 2017 +0200 Merge pull request #14153 from smithfarm/wip-18892-kraken kraken: librbd: Incomplete declaration for ContextWQ in librbd/Journal.h Reviewed-by: Jason Dillaman commit c2e63ada80e3baecbaa8f567695ab860257e5cc5 Merge: 14caf24f1f 3b0a7a8d2a Author: Nathan Cutler Date: Wed Apr 12 16:09:55 2017 +0200 Merge pull request #14151 from smithfarm/wip-18822-kraken kraken: tests: run-rbd-unit-tests.sh assert in lockdep_will_lock, TestLibRBD.ObjectMapConsistentSnap Reviewed-by: Jason Dillaman commit 14caf24f1f3bc94d1a2a081957376474e8c3d66f Merge: 041b553a14 85b5817567 Author: Nathan Cutler Date: Wed Apr 12 16:06:58 2017 +0200 Merge pull request #14149 from smithfarm/wip-18777-kraken kraken: rbd: rbd --pool=x rename y z does not work Reviewed-by: Jason Dillaman commit 041b553a144cf798679caebcdb16d2ce2104c7ab Merge: 1d4617e1f2 24ecaed808 Author: Nathan Cutler Date: Wed Apr 12 15:50:51 2017 +0200 Merge pull request #14315 from smithfarm/wip-19460-kraken kraken: build/ops: spec file mentions non-existent ceph-create-keys systemd unit file, causing ceph-mon units to not be enabled via preset Reviewed-by: Ken Dreyer commit 1d4617e1f25498a6213cf94c3ff0701b61283ed2 Merge: c860645bfa fcb86696eb Author: Nathan Cutler Date: Wed Apr 12 15:42:27 2017 +0200 Merge pull request #13875 from smithfarm/wip-19172-kraken kraken: rgw: S3 create bucket should not do response in json Reviewed-by: Abhishek Lekshmanan commit c860645bfaa89ae48c4343208be8613fdc2538ce Merge: 2b7c762223 86a9450380 Author: Nathan Cutler Date: Wed Apr 12 15:34:55 2017 +0200 Merge pull request #13873 from smithfarm/wip-19164-kraken kraken: rgw: radosgw-admin: add the 'object stat' command to usage Reviewed-by: Casey Bodley commit 2b7c762223ee628a8e29e216fee4417b5b570edc Merge: 2869d35262 d96ae9ead2 Author: Nathan Cutler Date: Wed Apr 12 15:32:54 2017 +0200 Merge pull request #13245 from smithfarm/wip-18776-kraken kraken: rbd: qemu crash triggered by network issues Reviewed-by: Jason Dillaman commit 2869d35262bc261783b3418649458d2d6e263f18 Merge: 9b17d88cf2 85bda6e263 Author: Nathan Cutler Date: Wed Apr 12 15:32:17 2017 +0200 Merge pull request #13247 from smithfarm/wip-18456-kraken kraken: rbd: attempting to remove an image with incompatible features results in partial removal Reviewed-by: Jason Dillaman commit 9b17d88cf2ef4df40d26622ffa627d2fb70a557f Merge: 4a3f60f04f deb6d2c64c Author: Nathan Cutler Date: Wed Apr 12 15:31:24 2017 +0200 Merge pull request #13132 from rjfd/wip-18609-kraken kraken: librbd: allow to open an image without opening parent image Reviewed-by: Jason Dillaman commit 4a3f60f04fde783fdfbf7ce1ea592ee348be34d3 Merge: cfed98ae29 16b2fd00ec Author: Nathan Cutler Date: Wed Apr 12 15:17:09 2017 +0200 Merge pull request #13185 from smithfarm/wip-18721-kraken kraken: build/ops: systemd restarts Ceph Mon to quickly after failing to start Reviewed-by: Boris Ranto Reviewed-by: Wido den Hollander commit 57f3656231b7780ff11f95a9feaaa28c3d573b35 Author: Jason Dillaman Date: Thu Mar 16 12:27:08 2017 -0400 pybind: fix incorrect exception format strings Signed-off-by: Jason Dillaman (cherry picked from commit 68617455f534a612ade1331f43b032ab524704ae) commit cfed98ae291f12c119083ba08fe813b9f0bc9eb9 Merge: 5abdffe950 e40f48dbfc Author: Nathan Cutler Date: Wed Apr 12 13:55:16 2017 +0200 Merge pull request #13845 from smithfarm/wip-19147-kraken kraken: rgw: DUMPABLE flag is cleared by setuid preventing coredumps Reviewed-by: Brad Hubbard commit 6e5ac14ab6f53424831e4cf08a3349a12b1ca947 Author: Radoslaw Zarzynski Date: Fri Feb 17 00:56:34 2017 +0100 rgw: make sending Content-Length in 204 and 304 controllable This commit introduces a new configurable "rgw print prohibited content length" to let operator decide whether RadosGW complies to RFC 7230 (a part of the HTTP specification) or violates it but follows the Swift's behavior. Fixes: http://tracker.ceph.com/issues/16602 Signed-off-by: Radoslaw Zarzynski (cherry picked from commit d8e3e64ec97a3c222a56bb6f510e5e23d7858615) commit 5abdffe950b51d5aff3500fd8dd7454a279da2fc Merge: 0517890f58 6f06cf80a7 Author: Nathan Cutler Date: Wed Apr 12 13:43:09 2017 +0200 Merge pull request #13555 from smithfarm/wip-18707-kraken kraken: mds: failed filelock.can_read(-1) assertion in Server::_dir_is_nonempty Reviewed-by: Yan, Zheng Reviewed-by: John Spray commit 0517890f58106eed3c0348a0a18b24bde7b340ab Merge: a48a897710 b464d1a571 Author: Nathan Cutler Date: Wed Apr 12 13:37:49 2017 +0200 Merge pull request #13488 from shinobu-x/wip-18955-kraken kraken: ceph-disk: Fix getting wrong group name when --setgroup in bluestore Reviewed-by: Loic Dachary commit a48a8977100838fc4140d9d59511a6cc21ec0075 Merge: 59c5511cb1 29c8c3fb86 Author: Nathan Cutler Date: Wed Apr 12 12:46:00 2017 +0200 Merge pull request #13112 from jcsp/wip-18678 kraken: mds: finish clientreplay requests before requesting active state Reviewed-by: Yan, Zheng Reviewed-by: John Spray commit 59c5511cb18286db1b032db8f3459763cc8dbb7c Merge: 8ded39b989 dab9fe761e Author: Nathan Cutler Date: Wed Apr 12 10:52:42 2017 +0200 Merge pull request #14048 from asheplyakov/reindex-on-pg-split-kraken kraken: osd: pg log split does not rebuild index for parent or child Reviewed-by: Kefu Chai commit 8ded39b9899ea461756650c17e1446682690c6f0 Merge: 2a2bcc6051 35b13c6687 Author: Nathan Cutler Date: Wed Apr 12 10:47:12 2017 +0200 Merge pull request #13116 from smithfarm/wip-18403-kraken kraken: mon: cache tiering: base pool last_force_resend not respected (racing read got wrong version) Reviewed-by: Kefu Chai commit 2a2bcc6051ad0d4badfdbabb81bd3d9cfb449234 Merge: 005e803abe 7475e44214 Author: Nathan Cutler Date: Wed Apr 12 10:41:21 2017 +0200 Merge pull request #13487 from shinobu-x/wip-18819-kraken kraken: common: possible lockdep false alarm for ThreadPool lock Reviewed-by: Jason Dillaman Reviewed-by: Nathan Cutler commit 005e803abe5dbe652bbee1c3e353fb38db446800 Merge: 42e93ae1b5 a379430d19 Author: Nathan Cutler Date: Wed Apr 12 10:36:12 2017 +0200 Merge pull request #14147 from smithfarm/wip-18769-kraken kraken: rbd: [ FAILED ] TestJournalTrimmer.RemoveObjectsWithOtherClient Reviewed-by: Jason Dillaman commit 42e93ae1b5e1e65a8abb3f37469030ed4145fbda Merge: fb29415450 ecb459f083 Author: Nathan Cutler Date: Wed Apr 12 02:53:06 2017 +0200 Merge pull request #13026 from SUSE/wip-18606-kraken kraken: tools: ceph-disk prepare writes osd log 0 with root owner Reviewed-by: Loic Dachary commit f9e3e9f9ff22901d90e783a28efdfafa12be650f Author: Zack Cerza Date: Tue Apr 11 09:25:50 2017 -0600 suites/upgrade-hammer-jewel-x: Add volumes Signed-off-by: Zack Cerza commit 543e4b90603b5992ea5d1c66d89e077375faf14e Author: Nathan Cutler Date: Thu Mar 2 12:41:07 2017 +0100 build/ops: add psmisc dependency to ceph-base (deb and rpm) Fixes: http://tracker.ceph.com/issues/19129 Signed-off-by: Nathan Cutler (cherry picked from commit 769b695465162bc8424abf8e2f259e6765b5bbff) commit fb29415450eda1ec480600132a937efbe988a1f4 Merge: 10b441aac7 6b3655fe4b Author: Loic Dachary Date: Sun Apr 9 22:40:21 2017 +0200 Merge pull request #13497 from shinobu-x/wip-17821-kraken kraken: ceph-disk: does not support cluster names different than 'ceph' Reviewed-by: Loic Dachary commit 910b9899702cfc08b74df6d988e1981efdb33f88 Author: Brad Hubbard Date: Mon Apr 3 13:37:17 2017 +1000 ceph-disk: Populate mount options when running "list" Also tidy up by moving duplicated code into a function Fixes: http://tracker.ceph.com/issues/17331 Signed-off-by: Brad Hubbard (cherry picked from commit 7943ab2e01e24f2dfc5b6f1d3ffdc8a49e01af45) commit 5c7a07677aa142fadbab36d0c1b24aea32ce9ad5 Author: Ilya Dryomov Date: Tue Mar 28 11:49:08 2017 +0200 osdc/Objecter: respect epoch barrier in _op_submit() Epoch barrier instructs us to avoid sending (i.e. pause) any OSD ops until we see a barrier epoch. The only thing epoch_barrier check in target_should_be_paused() does is keep already paused ops paused. We need to actually pause incoming OSD ops in _op_submit(). Fixes: http://tracker.ceph.com/issues/19396 Signed-off-by: Ilya Dryomov (cherry picked from commit f8e8efc0a53d7bd807cc0c2178aef7c4bed62ab7) commit c2cc96e6c19c77d0ea13e8a015d0dfa66ba1b304 Author: Fabian Grünbichler Date: Wed Mar 22 16:13:50 2017 +0100 common: fix segfault in public IPv6 addr picking sockaddr is only 16 bytes big, so declaring net as sockaddr and then casting to sockaddr_in6 in case of IPv6 cannot work. using sockaddr_storage works for both IPv4 and IPv6, and is used in other code parts as well. note that the tests did not find this issue as they declared the bigger structs and casted the references to (sockaddr *) Fixes: http://tracker.ceph.com/issues/19371 Signed-off-by: Fabian Grünbichler (cherry picked from commit ae2ee3d3835fe25b35eeb1a841ee5234cd69eb65) commit b49a71daf51bf2fdd892cbfc033c0760cbce0464 Author: David Zafman Date: Wed Jan 18 08:33:40 2017 -0800 osd: Calculate degraded and misplaced more accurately Calculate num_object_copies based on the larger of pool size, up set size and acting set size. Calculate num_objects_degraded as the difference between num_object_copies and all copies found on acting set and backfilling up set OSDs. Calculate num_objects_misplaced as all copies on acting set OSDs not in up set less copies that have been backfilled to up set OSDs. Fixes: http://tracker.ceph.com/issues/18619 Signed-off-by: David Zafman (cherry picked from commit 8423bc40759cca137f61e7b755411719a84369d4) commit 6d719558eb8135b3104132e7e70c163169e93876 Author: Mingxin Liu Date: Mon Mar 13 23:41:58 2017 +0800 osd: don't share osdmap with objecter when preboot Signed-off-by: Mingxin Liu (cherry picked from commit a5a3644eecc49b4eea890c6999fe87536495dcbe) commit 24ecaed8081f0bfec3324f86423f80e5d29c99b3 Author: Sébastien Han Date: Thu Mar 30 09:32:31 2017 +0200 systemd: remove all occurence of ceph-create-keys ceph-create-keys unit file does not exist anymore so there is no need to call it. Signed-off-by: Sébastien Han (cherry picked from commit 205cff12b3fe577a02db119ed3084d5c46ba76ce) commit b77d780c91f96ce9bb63fa1c09d507d02e555178 Author: Sébastien Han Date: Wed Mar 29 15:47:56 2017 +0200 systemd: remove ceph-create-keys from presets ceph-create-keys unit file was removed here: * https://github.com/ceph/ceph/commit/8bcb4646b6b9846bb965cdec3ca2a21eb3b26bab * https://github.com/ceph/ceph/commit/dc5fe8d415858358bd0baf5d8dce0a753f5e0cea As a consequence the systemctl preset command now fails to run since the unit does not exist anymore. Due to the redirection in /dev/null we don't know what's happening. Ultimately the mon unit doesn't get enabled and the mon service won't start after reboot. Removing the old/non-existent unit makes the command succeed now. Signed-off-by: Sébastien Han (cherry picked from commit 151c0ce213186b2196531ce74478ed7671eb9819) commit 10b441aac71ad2d7a7ee6181c958a8bb0f710213 Merge: 55444ce3ae df59d6d5f7 Author: Jason Dillaman Date: Tue Mar 28 21:00:31 2017 -0400 Merge pull request #14163 from trociny/wip-19368 kraken: librbd: possible race in ExclusiveLock handle_peer_notification Reviewed-by: Jason Dillaman commit 55444ce3aea473f5cd7c8f6eeb770b819318a6a4 Merge: 98a87fa97c 2d668468f6 Author: David Zafman Date: Tue Mar 28 15:25:39 2017 -0700 Merge pull request #14200 from dzafman/wip-18533-again Fixes: http://tracker.ceph.com/issues/19391 Reviewed-by: Josh Durgin commit 2d668468f60dd55a5591aa74bda491545368eee0 Author: David Zafman Date: Mon Mar 20 17:28:45 2017 -0700 filestore, tools: Fix logging of DBObjectMap check() repairs Signed-off-by: David Zafman (cherry picked from commit 1704f62c0831e6b07138f7dd14a89fef3c9ed2c1) commit f4945116b70b2ca457565ed313d2e0b52cf13383 Author: David Zafman Date: Fri Mar 3 15:04:02 2017 -0800 osd: Simplify DBObjectMap by no longer creating complete tables Bump the version for new maps to 3 Make clone less efficient but simpler Add rename operation (use instead of clone/unlink) For now keep code that understands version 2 maps Signed-off-by: David Zafman (cherry picked from commit 738156a99ed1caf61e5a8230eb8048360056c08e) No ghobject_t::operator>() so use Kraken cmp_bitwise() instead Need to use MIN_GHOBJ/MAX_GHOBJ instead of std::min/std::max commit 06c27d7ec0fa31000232b8f45a0fbeb6702f0180 Author: David Zafman Date: Wed Feb 15 16:17:32 2017 -0800 ceph-osdomap-tool: Fix seg fault with large amount of check error output Signed-off-by: David Zafman (cherry picked from commit 1dda0411f4fbb14ce1e0062da9f14ec3af505d39) commit 4810c544b10972e6b276517a2bb4ba2da0b06697 Author: David Zafman Date: Wed Feb 15 15:02:33 2017 -0800 osd: Add automatic repair for DBObjectMap bug Add repair command to ceph-osdomap-tool too Under some situations the previous rm_keys() code would generated a corrupt complete table. There is no way to figure out what the table should look like now. By removing the entries we fix the corruption and aren't much worse off because the corruption caused some deleted keys to re-appear. This doesn't breaking the parent/child relationship during repair because some of the keys may still be contained in the parent. Signed-off-by: David Zafman (cherry picked from commit 4cd3c74c928a32e065ed9543d6c91d8718a6ae3d) Conflicts: src/os/filestore/DBObjectMap.h (trivial) commit 61dad6c15dc7ac94c4bea1fb57823d78ae2db5a1 Author: David Zafman Date: Wed Feb 15 14:59:40 2017 -0800 ceph-osdomap-tool: Fix tool exit status Signed-off-by: David Zafman (cherry picked from commit 666f14ed90655a2d1bedde8561949625db7a9e6c) commit a72f923186ac490fe780f7168a6a34c7d933cf96 Author: Samuel Just Date: Fri Feb 10 15:51:42 2017 -0800 DBObjectMap: rewrite rm_keys and merge_new_complete Leverage the updated in_complete_region and needs_parent to simplify these methods. Signed-off-by: Samuel Just (cherry picked from commit c4dffb68eaafe724f7fdae93a4285a7f8003ea29) commit 960431a6b8e79f6832a85e462889dd1f26f12624 Author: Samuel Just Date: Fri Feb 10 15:50:57 2017 -0800 DBObjectMap: strengthen in_complete_region post condition Previously, in_complete_region didn't guarantee anything about where it left complete_iter pointing. It will be handy for complete_iter to be pointing at the lowest interval which ends after to_test. Make it so. Signed-off-by: Samuel Just (cherry picked from commit 97b35f4d7d4862da4b6f50ecaef0d292a671fd04) commit a3c3e3ef4e3d20baf28000f11b4ea5bb45e161f9 Author: Samuel Just Date: Fri Feb 10 15:48:57 2017 -0800 DBObjectMap: fix next_parent() The previous implementation assumed that lower_bound(parent_iter->key()) always leaves the iterator on_parent(). There isn't any guarantee, however, that that key isn't present on the child as well. Signed-off-by: Samuel Just (cherry picked from commit 74a7631d0938d7b44894f022224eab10a90d5cec) commit 86881dd361e21e3e4553b416331ca393e88b7a3e Author: Samuel Just Date: Thu Feb 9 10:47:59 2017 -0800 test_object_map: add tests to trigger some bugs related to 18533 Signed-off-by: Samuel Just (cherry picked from commit f131dbcf5bb17107c029f942a57e9bf4432a26ee) commit 4c7fbea9e4c171ab21f3dcd6ca0b17abb3420979 Author: David Zafman Date: Tue Feb 14 12:40:33 2017 -0800 test: Add ceph_test_object_map to make check tests Signed-off-by: David Zafman (cherry picked from commit 0e97a01bd7291458881ee53cece2d887f6333669) commit ef5aeab9b22292c366489668ee21ce4b9d2d8572 Author: David Zafman Date: Wed Feb 8 18:56:27 2017 -0800 ceph-osdomap-tool: Add --debug and only show internal logging if enabled Signed-off-by: David Zafman (cherry picked from commit 5fb2b2d13953979e5da9f571ab8c4b0b510b8368) commit c5d462f3524f0e5eff1594d7ddf97c3c9c0882a8 Author: David Zafman Date: Wed Feb 8 18:55:48 2017 -0800 osd: DBOjectMap::check: Dump complete mapping when inconsistency found Signed-off-by: David Zafman (cherry picked from commit fcf1e17c645e8fad5216c3e59627c817e5c858c7) commit b37b0298ca9c002033b3ae8ebcaeb7a24690447d Author: David Zafman Date: Wed Feb 8 15:38:51 2017 -0800 test_object_map: Use ASSERT_EQ() for check() so failure doesn't stop testing Signed-off-by: David Zafman (cherry picked from commit 053a273cbc02d6902a4bb1f11db1ea946498df3a) commit 8f297204a428b48d2119df0e1a8da41e4f15e4fb Author: David Zafman Date: Wed Feb 8 10:02:40 2017 -0800 tools: Check for overlaps in internal "complete" table for DBObjectMap Changed check to return an error count and fix tool error message Signed-off-by: David Zafman (cherry picked from commit e5e8eb962db6187ea19b96ba29ac83469c90b4ea) Conflicts: src/os/filestore/DBObjectMap.h (trivial) commit bc2228a903bbb89a11d7863f9307d6c3b5d4928c Author: David Zafman Date: Wed Feb 8 09:40:49 2017 -0800 tools: Add dump-headers command to ceph-osdomap-tool Signed-off-by: David Zafman (cherry picked from commit f4101591ad701a62fe027c4744ca8ea505f44bdc) Conflicts: src/os/filestore/DBObjectMap.h (trivial) commit 97186735783a9758a1695a9938b22c2b4b46f530 Author: David Zafman Date: Mon Feb 6 21:09:42 2017 -0800 tools: Add --oid option to ceph-osdomap-tool Signed-off-by: David Zafman (cherry picked from commit 2d94889e9ee3359017b1efd560f3557ce03ccee6) commit c208c91c7f845896b4fe0befd62cc66bee69018f Author: David Zafman Date: Mon Feb 6 21:31:18 2017 -0800 osd: Remove unnecessary assert and assignment in DBObjectMap Fix and add comment(s) Signed-off-by: David Zafman (cherry picked from commit 937e6a03ea4692cc44d53faa0615f8e808c9eb03) commit df59d6d5f7deb586cf14a6ef6984e6847db08852 Author: Mykola Golub Date: Tue Mar 28 15:55:11 2017 +0200 kraken: librbd: possible race in ExclusiveLock handle_peer_notification This is a direct commit to kraken -- the master diverged after ManagedLock refactoring and is not affected. Fix: http://tracker.ceph.com/issues/19368 Signed-off-by: Mykola Golub commit d91697237ad77eaf60098536a2cd3538ea9aeabf Author: Jason Dillaman Date: Mon Nov 21 15:31:43 2016 -0500 qa/workunits/rbd: resolve potential rbd-mirror race conditions Fixes: http://tracker.ceph.com/issues/18935 Signed-off-by: Jason Dillaman (cherry picked from commit 63eae97afc1a92412525468263fb8696a243ebac) commit 6269f152238b77b0d02e4ad865164a531541d6db Author: Boris Ranto Date: Wed Feb 8 23:47:57 2017 +0100 librbd: Include WorkQueue.h since we use it We use m_work_queue of type ContextWQ in handle_update function but we do not include common/WorkQueue.h that defines ContextWQ. This results in dereference of an incomplete type and causes build error in latest Fedora rawhide (future 26). Fixes: http://tracker.ceph.com/issues/18862 Signed-off-by: Boris Ranto (cherry picked from commit 480f82847ad1fc7959f1fe5a90761a5a24550993) commit 3b0a7a8d2aa0dbc7cfce317bd327f88afce2469e Author: Jason Dillaman Date: Wed Jan 18 20:54:22 2017 -0500 librbd: avoid possible recursive lock when racing acquire lock Fixes: http://tracker.ceph.com/issues/17447 Signed-off-by: Jason Dillaman (cherry picked from commit 5e46e8eb664f573bd70ae7c96a6d9a98b0deb09e) commit 85b5817567c22acf63719a65bd65e77709ffce0c Author: Gaurav Kumar Garg Date: Mon Jan 30 13:03:20 2017 +0100 rbd: destination pool should be source pool if it is not specified Currently if user perform image rename operation and user give pool name as a optional parameter (--pool=) then currently its taking this optional pool name for source pool and making destination pool name default pool name. With this fix if user provide pool name as a optional pool name parameter then it will consider both soruce and destination pool name as optional parameter pool name. Fixes: http://tracker.ceph.com/issues/18326 Reported-by: МАРК КОРЕНБЕРГ Signed-off-by: Gaurav Kumar Garg (cherry picked from commit 01f23aa99fb694da326ab408e75b33c640ce660b) commit a379430d1921a1e5a330b4bfe0189a9a21dc5f81 Author: Jason Dillaman Date: Mon Jan 30 17:41:51 2017 -0500 journal: stop processing removal after error Fixed: http://tracker.ceph.com/issues/18738 Signed-off-by: Jason Dillaman (cherry picked from commit 040004f53f7d334fc9452e3f5fd80169cad5f65e) commit 51e8ba0a06f2b5574f82f0ca63a9e9b73c291b51 Author: Orit Wasserman Date: Sun Mar 12 12:11:28 2017 +0200 rgw: use rgw_zone_root_pool for region_map like is done in hammer Fixes: http://tracker.ceph.com/issues/19195 Signed-off-by: Orit Wasserman (cherry picked from commit c91dd6d9efd148e0fe0f027dde537e977de9aa26) commit 76ce6841f7066b4902665dcf752568d0b5417652 Author: lu.shasha Date: Mon Feb 27 15:52:43 2017 +0800 rgw: use separate http_manager for read_sync_status concurrent users of read_sync_status() use different cr managers, when get_resource must use http_manager related to the cr manager. Fixes: http://tracker.ceph.com/issues/19236 Signed-off-by: Shasha Lu (cherry picked from commit c412024889f8995d98096ac863bafee71624bd70) commit 84447a42e82a21e08f71ea6113216d9a2cb2ab09 Author: Casey Bodley Date: Thu Mar 9 15:24:08 2017 -0500 rgw: clear old zone short ids on period update the short ids of old, removed zones were being kept in the period to guard against hash collisions with new zones but for a hash collision to cause a wrong object to sync, that object would have to be uploaded simultaneously to two different zones that had the same short id to avoid this, we just have to prevent the period from containing two colliding zones at the same time - we don't have to remember old zone short ids forever Fixes: http://tracker.ceph.com/issues/15618 Signed-off-by: Casey Bodley (cherry picked from commit 9c45633c836c966ab1f75ea2b1ad3fa0a4886600) commit c56a3134ebe1b531af09b19214a16b37e0e8a6d8 Author: Orit Wasserman Date: Thu Mar 9 13:03:24 2017 +0200 rgw: skip conversion of zones without any zoneparams Fixes: http://tracker.ceph.com/issues/19231 Signed-off-by: Orit Wasserman (cherry picked from commit 36cf5a5c8179c6313346b2e29286c537c6fefce8) commit d0d70359811c306116921cf8f3768a97da16a813 Author: Orit Wasserman Date: Thu Mar 9 11:16:26 2017 +0200 rgw: better debug information for upgrade Signed-off-by: Orit Wasserman (cherry picked from commit e9f3bf8eab1dd46a92f54b0f7afe1f4c0e4204db) commit 01f5d8899b8d16e419e1bfdd4e8bbd80a058df93 Author: Danny Al-Gaaf Date: Tue Jan 31 18:01:32 2017 +0100 rgw/rgw_rados.cc: prefer ++operator for non-primitive iterators Signed-off-by: Danny Al-Gaaf (cherry picked from commit 7086cf9a73f2ec1eb96c0e752beb1b74fca18570) commit c4f8114c6b86bc1e0afabecf9424dadd30b37006 Author: Kefu Chai Date: Mon Mar 6 11:33:27 2017 +0800 brag: count the number of mds in fsmap not in mdsmap this change was introduced in 4e9b953 Fixes: http://tracker.ceph.com/issues/19192 Signed-off-by: Peng Zhang (cherry picked from commit 2d25a9c0c760664d3de33ecca0e0272c1031cd46) commit 81f3d860324da49f71b757e6cc8f32966272cf5c Author: Brad Hubbard Date: Tue Mar 21 12:22:20 2017 +1000 tools/rados: Check return value of connect Fail gracefully if Rados::connect returns an error. Fixes: http://tracker.ceph.com/issues/19319 Signed-off-by: Brad Hubbard (cherry picked from commit c119091ef0844e4a1ddd790a8bfef8f06bb57d58) commit 076e4b7991e3bd734a41604fa7cb00c7b03d7749 Author: Casey Bodley Date: Thu Oct 20 15:01:01 2016 -0400 rgw: fix break inside of yield in RGWFetchAllMetaCR the yield macro is implemented with for/switch, so the breaks in RGWFetchAllMetaCR weren't being applied to the for loop as expected - so any of these breaks send RGWFetchAllMetaCR into an infinite loop removed the yield {} block, so that breaks will apply to the for loop as intended, then added a single yield; statement to allow the entries_index consumer to run one per iteration Fixes: http://tracker.ceph.com/issues/17655 Signed-off-by: Casey Bodley (cherry picked from commit 190bd385a7be52867d65740c410884f5c8cbc21f) commit 5dcf60702cd1733dbe7f2d1d7dc68a096684bf77 Author: Casey Bodley Date: Fri Mar 3 12:10:40 2017 -0500 rgw: delete_system_obj() fails on empty object name Signed-off-by: Casey Bodley (cherry picked from commit 67401193f871db95a6045915fa59dce8c5dd1012) commit dea53b3bf267709d2a8daacd961997892ad2d0f4 Author: Casey Bodley Date: Fri Mar 3 11:42:45 2017 -0500 rgw: if user.email is empty, dont try to delete Fixes: http://tracker.ceph.com/issues/18980 Signed-off-by: Casey Bodley (cherry picked from commit 022ecf0fcc8e44912c8758ee1d9a452dc23cbbce) commit dab9fe761ec7f46b7da035f710f824516363119c Author: Sage Weil Date: Fri Feb 17 12:46:38 2017 -0500 osd/PGLog: avoid return by value on ginormous log Signed-off-by: Sage Weil (cherry picked from commit a6ead998771753d95382abd082f451e6f67744e4) commit eac5610896e59bc8e711ed48446aa6967ebb61c8 Author: Sage Weil Date: Fri Feb 17 14:50:38 2017 -0500 osd/PGLog: reindex properly on pg log split When pg_log_t::split_out_child() runs it builds the list, which means the old indexes are wrong (the point to bad memory), but index() will not rebuild them because ever since b858e869e78927dccebaa350d246bd74af7f1de9 we won't rebuild them if they are already built. Fix that by calling unindex() before the split. Further, the new child log also needs to be indexed. Fix that too. Fixes: http://tracker.ceph.com/issues/18975 Signed-off-by: Sage Weil (cherry picked from commit 643ae42cf27f16dd6ed4e1402acc0483bb9fca74) commit 0227920438d98a8c1d74f0ec7a95bc63fa066acf Author: Ilya Dryomov Date: Wed Mar 1 17:19:04 2017 +0100 osd/OSDMap: don't set weight to IN when OSD is destroyed Since commit 4e28f9e63644 ("osd/OSDMap: clear osd_info, osd_xinfo on osd deletion"), weight is set to IN when OSD is deleted. This changes the result of applying an incremental for clients, not just OSDs. Because CRUSH computations are obviously affected, pre-4e28f9e63644 servers disagree with post-4e28f9e63644 clients on object placement, resulting in misdirected requests. Fixes: http://tracker.ceph.com/issues/19119 Signed-off-by: Ilya Dryomov (cherry picked from commit a6009d1039a55e2c77f431662b3d6cc5a8e8e63f) commit d4a43e1fd56cfce1e1b64827a3d98c14ec69e193 Author: Jing Wenjun Date: Fri Feb 3 19:06:31 2017 +0800 rgw: swift: fix anonymous user's error code of getting object The openstack swift will return 401 rather than 403 when the anon user has no permission to get objects. Fixes: http://tracker.ceph.com/issues/18806 Signed-off-by: Jing Wenjun (cherry picked from commit 318de28e1011914dc7f3701549eb3ef227abbc3c) commit fcb86696eb4999689c7da71d565e1262beba9dec Author: Abhishek Lekshmanan Date: Thu Feb 16 17:40:50 2017 +0100 doc: rgw: make a note abt system users vs normal users Mention that system users don't behave like normal users in context of normal rest operations Fixes: http://tracker.ceph.com/issues/18889 Signed-off-by: Abhishek Lekshmanan (cherry picked from commit a47bcf70c9f51a6601b809cba219f5615b204d34) commit 86a945038059d976e2f7227d9ee327ec2899d15c Author: root Date: Tue Feb 7 14:37:36 2017 +0530 rgw: Let the object stat command be shown in the usage Fixes: http://tracker.ceph.com/issues/19013 Signed-off-by: Pavan Rallabhandi (cherry picked from commit 0fe76f83d19be098ef54fb0492a376fef3aa9e23) commit 1f86be6e19dc3e26f62bf57c55d662414247b480 Author: Casey Bodley Date: Mon Feb 20 16:00:01 2017 -0500 rgw: RGWMetaSyncShardControlCR retries with backoff on all error codes RGWBackoffControlCR only treats EBUSY and EAGAIN as 'temporary' error codes, with all other errors being fatal when exit_on_error is set to RGWMetaSyncShardControlCR, a 'fatal' error means that no further sync is possible on that shard until the gateway restarts this changes RGWMetaSyncShardControlCR to set exit_on_error to false, so that it will continue to retry with backoff no matter what error code it gets Fixes: http://tracker.ceph.com/issues/19019 Signed-off-by: Casey Bodley (cherry picked from commit 3e4059557fd6cad5d31014327f60832b36d04a6c) commit b39bef09611653723a29c989a6012064f17bd586 Author: root Date: Tue Feb 21 16:33:29 2017 +0530 rgw: Correct the return codes for the health check feature Fixes: http://tracker.ceph.com/issues/19025 Signed-off-by: Pavan Rallabhandi (cherry picked from commit 4da2bf310f6d43423554c32e43ebf90ad2c3f3a9) commit 5e15025ffbf80f65cde891ad47fe1f1b913465cd Author: Ronak Jain Date: Wed Feb 22 12:03:46 2017 +0530 rgw: Fixes typo in rgw_admin.cc Issue: http://tracker.ceph.com/issues/19026 Signed-off-by: Ronak Jain (cherry picked from commit 58837ef6ce8cbcfc2cac29d5f833b2cf62d8737a) commit e40f48dbfce23cbc2164f8789db4688fbec9c3bb Author: Brad Hubbard Date: Mon Feb 27 13:06:59 2017 +1000 rgw: set dumpable flag after setuid post ff0e521 ff0e521 resolved the issue for the other daemons but not for rgw since it calls setuid (via civetweb) after the new code sets PR_SET_DUMPABLE. Add another prctl call before wait_shutdown. Fixes: http://tracker.ceph.com/issues/19089 Signed-off-by: Brad Hubbard (cherry picked from commit bc458d39630b599e0e1ca9fe25ad7455fcffdd10) commit 47cf8f6699ec0690e2a296b3abe289ce5d212e1c Author: Yehuda Sadeh Date: Mon Feb 27 10:35:01 2017 -0800 rgw: don't init rgw_obj from rgw_obj_key when it's incorrect to do so Fixes: http://tracker.ceph.com/issues/19096 rgw_obj_key currently deals with the bucket index key, and not representing a (name, instance, ns) tupple. Need to initialize it in two steps. Signed-off-by: Yehuda Sadeh (cherry picked from commit 392c5d9dae6ba699014ffe6e1e67818fa62d7e41) commit 03b1ab4b13780a53c7b9017f3a6207dab2ffff5b Author: Jing Wenjun Date: Thu Jan 19 21:00:26 2017 +0800 rgw: swift: the http referer acl in swift API should be shown The container acl about http referer set should be shown in container metadata dump. Fixes: http://tracker.ceph.com/issues/18665 Signed-off-by: Jing Wenjun (cherry picked from commit 3860ddad084c5b8d9d5ec83aeb3220207cd0e5ad) commit 98a87fa97c9b23e21a05130c72730f5034691310 Merge: 7e16ea218d e2074cef71 Author: Sage Weil Date: Sat Mar 4 11:22:44 2017 -0600 Merge pull request #13501 from shinobu-x/wip-18371-kraken kraken: ceph-disk: error on _bytes2str commit 46548c51255518e422d544f0d0776068051fdacc Author: Adam C. Emerson Date: Mon Nov 14 19:33:56 2016 -0500 common: Unskew clock In preparation to deglobalizing CephContext, remove the CephContext* parameter to ceph_clock_now() and ceph::real_clock::now() that carries a configurable offset. Signed-off-by: Adam C. Emerson (cherry picked from commit 750ad8340c827d2f8896e1251e45f921dddb9f30) Conflicts: src/mon/PGMonitor.cc commit 1120cd74d88b163f7cd1c81ae5670bf6d6153715 Author: Sage Weil Date: Tue Jan 31 12:31:42 2017 -0500 mon/PGMonitor: fix force_create_pg We weren't carefully registering the creating PG. In particular, the current osd mappings (acting, up, etc.) weren't getting filled in, which meant the PG wasn't (necessarily) mapped to an OSD until the OSDMap updated and we recalculated mappings. Fix by sending us through the common code path so that all fields get updated properly. Fixes: http://tracker.ceph.com/issues/18298 Signed-off-by: Sage Weil (cherry picked from commit 9a41a0b7289fa59f4b747a63e152e88af6e8abd5) Conflicts: src/mon/PGMonitor.cc commit 396f3325507cf899433da0a5f4964b81e0c2caa8 Author: Sage Weil Date: Tue Jan 31 12:31:12 2017 -0500 mon/PGMonitor: clean up some send_pg_creates cruft ff1fd4b4b69b52ef5ae734115b1d733aad03b03e already removed most of this; remove the now useless stub and clean up the unneede con check. Signed-off-by: Sage Weil (cherry picked from commit b79a85e80a1a660c5a5746270edb86d3f09cd983) Conflicts: src/mon/PGMonitor.cc commit 6196b6edd5ea87c9fabe598a309516e1c0ba73e9 Author: Sage Weil Date: Fri Dec 16 17:42:26 2016 -0500 mon/MonCommands: remove send_pg_creates from cmd table Forgot this in 6cbdd6750cf330047d52817b9ee9af31a7d318ae Signed-off-by: Sage Weil (cherry picked from commit 15f17bcca0bde2ed15bb45d67659f4ebbd4cb516) commit c8d70ee0129ce2cc459a2be386531aeee5c6c422 Author: Jing Wenjun Date: Thu Jan 19 21:26:17 2017 +0800 rgw: swift: The http referer should be parsed to compare in swift API The http referer should be parsed to compare with the url set on the container read acl. If we set .r:www.example.com on container read acl, we should parse the hostname 'www.example.com' of the http referer like 'http://www.example.com' from the http request. Fixes: http://tracker.ceph.com/issues/18685 Signed-off-by: Jing Wenjun (cherry picked from commit 941dfad67174ae3ec517e76bf4028c50fb46fe82) commit 7e16ea218d38703f1903dca002905f040a821590 Merge: ccc1982472 6adf39b983 Author: Kefu Chai Date: Fri Mar 3 11:14:48 2017 +0800 Merge pull request #13757 from liewegas/wip-pgp-kraken qa/tasks: set pgp = pg num on thrashing finish Reviewed-by: Kefu Chai commit 6adf39b98305b6af8d0192b35ec22c5007293d32 Author: Kefu Chai Date: Sun Feb 19 13:10:44 2017 +0800 test: Thrasher: do not update pools_to_fix_pgp_num if nothing happens we should not update pools_to_fix_pgp_num if the pool is not expanded or the pg_num is not increased due to pgs being created. this prevent us from fixing the pgp_num after done with thrashing if we actually did nothing when fixing the pgp_num when thrashing, but we removed the pool from pools_to_fix_pgp_num after set_pool_pgpnum() returns. Signed-off-by: Kefu Chai (cherry picked from commit c0f0cde399a80cea617c115417e5390a63ec674e) commit ab5538fd7e3b727616272edb565e3782d004632a Author: Kefu Chai Date: Sun Feb 12 13:16:57 2017 +0800 test: Thrasher: update pgp_num of all expanded pools if not yet otherwise wait_until_healthy will fail after timeout as seeing warning like: HEALTH_WARN pool cephfs_data pg_num 182 > pgp_num 172 Signed-off-by: Kefu Chai (cherry picked from commit 136483a8f940710cbe40804afa7af2eac6728d50) commit ccc1982472898efe2fcbd6b1cd3cad7afbf2c015 Merge: ae8eff5f2d 2c9889bb76 Author: Zack Cerza Date: Wed Mar 1 15:16:17 2017 -0700 Merge pull request #13707 from zmc/wip-openstack-volumes-kraken qa/suites/ceph-deploy: Drop OpenStack volume count commit 2c9889bb768198fe292de0c3edb076707e8a50e0 Author: Zack Cerza Date: Tue Feb 28 13:07:15 2017 -0700 qa/suites/ceph-deploy: Drop OpenStack volume count Looks like we only need two per node, since there is only one OSD per node, and ceph-deploy wants two disks per OSD to account for the journal. Signed-off-by: Zack Cerza (cherry picked from commit 87072e277c9ef259c9ee2ae1f761e252aa216713) commit ae8eff5f2dddcc064485ab69221c6d716e63925f Merge: 9fa9248e92 f9a282bc9d Author: Zack Cerza Date: Mon Feb 27 12:23:10 2017 -0700 Merge pull request #13673 from zmc/wip-openstack-volumes-kraken qa/suites/{ceph-ansible,rest}: OpenStack volumes commit f9a282bc9dda98a103db2c7ebe63450dad23cc5b Author: Zack Cerza Date: Mon Feb 27 09:14:41 2017 -0700 qa/suites/rest: Openstack volumes Signed-off-by: Zack Cerza (cherry picked from commit 99d942145f4206c00aca30c0bb74f0edc4bac798) commit e0332944c7647cc269d6b18ca867896d5bea2fa7 Author: Zack Cerza Date: Mon Feb 27 09:06:26 2017 -0700 qa/suites/ceph-ansible: Openstack volumes Signed-off-by: Zack Cerza (cherry picked from commit 964b983bdbd412311bce56184d12d1b6d43c7f28) commit 9fa9248e92f34127ca8b0d3b021b5ac0da0f6609 Merge: f159208832 b2e3692347 Author: Brad Hubbard Date: Sun Feb 26 09:23:43 2017 +1000 Merge pull request #13485 from shinobu-x/wip-18644-kraken kraken: osd: condition object_info_t encoding on required (not up) features Reviewed-by: Nathan Cutler commit f1592088328480c6545435a7031d8f06277a9d0c Merge: c59d13d162 2ab8239b09 Author: Zack Cerza Date: Fri Feb 24 17:22:11 2017 -0700 Merge pull request #13641 from zmc/wip-fs-openstack-kraken qa/suites/fs: Add openstack volume configuration commit 2ab8239b09f5d1215ad828eae7c64fb0f199d663 Author: Zack Cerza Date: Fri Feb 24 15:25:19 2017 -0700 qa/suites/fs: Add openstack volume configuration Signed-off-by: Zack Cerza (cherry picked from commit b076d89a3f1fbad7d477913812b2e17529abeacf) commit c59d13d162a2b2d51fe7c65a56a2a17b7d94aa8f Merge: 8f1f2018fb 596cfbbd0f Author: Zack Cerza Date: Fri Feb 24 14:44:04 2017 -0700 Merge pull request #13638 from zmc/wip-openstack-volumes-kraken qa/suites/{knfs,hadoop,samba}: OpenStack volume configuration commit 8f1f2018fb06bd56b63cb661a852ba6b44c5e749 Merge: 1e71356ee4 cec66c4dac Author: Zack Cerza Date: Fri Feb 24 14:43:34 2017 -0700 Merge pull request #13635 from zmc/wip-kcephfs-openstack-kraken qa/suites/kcephfs: Openstack volume configuration commit 6b3655fe4bd3c493e6b10dd597d791a5b572df2b Author: Loic Dachary Date: Wed Feb 22 01:49:12 2017 +0100 ceph-disk: dmcrypt activate must use the same cluster as prepare When dmcrypt is used, the fsid cannot be retrieved from the data partition because it is encrypted. Store the fsid in the lockbox to enable dmcrypt activation using the same logic as regular activation. The fsid is used to retrive the cluster name that was used during prepare, reason why activation does not and must not have a --cluster argument. Fixes: http://tracker.ceph.com/issues/17821 Signed-off-by: Loic Dachary (cherry picked from commit 7f66672b675abbc0262769d32a38112c781fefac) Conflicts: src/ceph-disk/ceph_disk/main.py commit 596cfbbd0fc521ef66e7e11a7098ea78adeb2565 Author: Zack Cerza Date: Fri Feb 24 13:46:54 2017 -0700 qa/suites/samba: Openstack volume configuration Signed-off-by: Zack Cerza (cherry picked from commit e0296d706422ea4dc01d84f8786f6f7104c3d996) commit 3359050ce87426f3b41afe85c117de2d47ece395 Author: Zack Cerza Date: Fri Feb 24 13:45:18 2017 -0700 qa/suites/hadoop: Openstack volume configuration Signed-off-by: Zack Cerza (cherry picked from commit 3fef0a49da2ccfdceba7b98e9096be8305da1111) commit a2ff395cb66fc20fd9a18e74a6cba5510d144a61 Author: Zack Cerza Date: Fri Feb 24 13:44:33 2017 -0700 qa/suites/knfs: Add openstack volume configuration Signed-off-by: Zack Cerza (cherry picked from commit 62c6fd3371adf0f420c12d9c7e2b3a2a0c69256b) commit cec66c4dacb39600f74927c9f8890805d3f7acdc Author: Zack Cerza Date: Fri Feb 24 13:37:23 2017 -0700 qa/suites/kcephfs: Openstack volume configuration Signed-off-by: Zack Cerza (cherry picked from commit ec6fb28eaf8e2db327e4afc115879a40c7664e07) commit 1e71356ee44828a0624d77f77b6a22580d1df28b Merge: 90c61976e8 7439d4cff2 Author: Zack Cerza Date: Fri Feb 24 12:16:02 2017 -0700 Merge pull request #13633 from zmc/wip-krbd-openstack-kraken qa/suites/krbd: Add openstack volume configuration commit 90c61976e82c5e23d4d3cbae00eddb47e649bb65 Merge: 91a2ab03e2 4c55a78a52 Author: Zack Cerza Date: Fri Feb 24 12:15:29 2017 -0700 Merge pull request #13613 from ceph/wip-rgw-openstack-kraken qa/suites/rgw: Add openstack volume configuration commit 91a2ab03e268227164ffec37b693d370485cc004 Merge: 6c6b185bab 19ca0db392 Author: Yuri Weinstein Date: Fri Feb 24 10:37:12 2017 -0800 Merge pull request #13618 from tmuthamizhan/wip-rm-trusty-kraken qa: drop ubuntu trusty support Reviewed-by: Yuri Weinstein commit 7439d4cff231e31dbf7c19591560bc0aa778f68a Author: Zack Cerza Date: Fri Feb 24 11:17:45 2017 -0700 qa/suites/krbd: Add openstack volume configuration Signed-off-by: Zack Cerza (cherry picked from commit 201b4d0d1e92bf95ac6a8f2951b664763030f12d) commit 4c55a78a52f9e089b97ab8a2feb9876ebdba0516 Author: Zack Cerza Date: Thu Feb 23 10:14:05 2017 -0700 qa/suites/rgw: Add openstack volume configuration Without this, OSDs will fail to create on instances whose root fs isn't xfs. (cherry picked from commit 8af4c35f9577ef5a88307ea5cbbe2561a473926c) Signed-off-by: Zack Cerza commit b2e369234705ae78774261c62081729e6a50e8e5 Author: Ilya Dryomov Date: Mon Feb 6 12:51:05 2017 +0100 osd/OSDMap: require OSD features only of OSDs Fix up commit 1a5cc32f0a3b ("osd/OSDMap: reflect REQUIRE_*_OSDS flag in required features") -- require_*_osds shouldn't affect older clients. Fixes: http://tracker.ceph.com/issues/18831 Signed-off-by: Ilya Dryomov (cherry picked from commit ab558bbf2956157cbde59c155a5180a62ae73d07) Conflicts: src/osd/OSDMap.cc commit 19ca0db392b7bdf9758aae4fb6bb6cb927411568 Author: Tamil Muthamizhan Date: Fri Feb 17 21:06:43 2017 +0000 qa: drop ubuntu trusty support ceph-ansible dropped support for OS that doesnt support systemd Signed-off-by: Tamil Muthamizhan (cherry picked from commit 4d4b38eca81f7b57e3d3b31e1c13e7ab0ba5b30f) Signed-off-by: Tamil Muthamizhan commit 6c6b185bab1e0b7d7446b97d5d314b4dd60360ff Merge: 7d9ef63d65 8ea14ce9ce Author: Yuri Weinstein Date: Thu Feb 23 14:55:56 2017 -0800 Merge pull request #13283 from zmc/wip-smoke-openstack-kraken qa/suites/smoke: add openstack requirements Reviewed-by: Yuri Weinstein commit dc8ef3508b0b3f902772eed947e5861720036dab Author: Nathan Cutler Date: Thu Feb 2 23:23:54 2017 +0100 tests: Thrasher: eliminate a race between kill_osd and __init__ If Thrasher.__init__() spawns the do_thrash thread before initializing the ceph_objectstore_tool property, do_thrash races with the rest of Thrasher.__init__() and in some cases do_thrash can call kill_osd() before Trasher.__init__() progresses much further. This can lead to an exception ("AttributeError: Thrasher instance has no attribute 'ceph_objectstore_tool'") being thrown in kill_osd(). This commit eliminates the race by making sure the ceph_objectstore_tool attribute is initialized before the do_thrash thread is spawned. Fixes: http://tracker.ceph.com/issues/18799 Signed-off-by: Nathan Cutler (cherry picked from commit b519d38fb1967628ad8a1c46fcfb3f984de58790) commit 7d9ef63d65aad40ce1f2f9f324d68c5fa9d77369 Merge: 033fdb1406 ab8558e05e Author: Mykola Golub Date: Tue Feb 21 16:16:26 2017 +0200 Merge pull request #13201 from dillaman/wip-18703-kraken kraken: librbd: prevent self-blacklisting during break lock Reviewed-by: Mykola Golub commit 033fdb14064d2a5ae9078944ab5ae8938a583d81 Merge: f65754e312 cc04659798 Author: Mykola Golub Date: Tue Feb 21 16:15:35 2017 +0200 Merge pull request #13102 from dillaman/wip-18668 kraken: test: use librados API to retrieve config params Reviewed-by: Mykola Golub commit 6f06cf80a7ff7407af40f46a0aefb4b88f926390 Author: Yan, Zheng Date: Wed Jan 18 16:23:49 2017 +0800 mds: fix incorrect assertion in Server::_dir_is_nonempty() when filelock is in XLOCKDONE state. client of xlocker can rdlock the filelock. In that case, only client of xlocker can read the lock. Fixes: http://tracker.ceph.com/issues/18578 Signed-off-by: Yan, Zheng (cherry picked from commit fe4ab52b30079a785be053a9fd0197d6990737fe) commit 914d8a6109f7c2125080104bd4bf463c45833c3c Author: craigchi Date: Thu Feb 16 19:21:48 2017 +0800 ceph-disk: Fix getting wrong group name when --setgroup in bluestore ceph-disk prepare --setgroup will be wrong when using with bluestore Signed-off-by: craigchi (cherry picked from commit a8c0870e7370a0948e8e7fd53d3376b85bf9c649) commit 1bc9cfff42237b572e90e976697b1ac5faafad4a Author: Sage Weil Date: Tue Feb 14 15:00:09 2017 -0500 osd/PG: restrict want_acting to up+acting on recovery completion On recovery completion we recalculate want_acting to see if we should add recently backfilled osds into acting. However, at this point we may have gotten infos from others OSDs outside of up/acting that could be used for want_acting. We currently assert that only up/acting osds are used in PG::RecoveryState::Active::react(const AdvMap&), so we must restrict want_acting to up/acting here. We could remove this restriction, but it would mean 1) checking on every map change that want_acting hasn't been invalidated, and if so, recalculating want_acting and requesting a new pg_temp. Also, presumably 2) on each new info, checking whether we can construct a better want_acting, and if so, doing it. That would be a good thing, but is a more complicated change. In reality this case comes up very rarely, so simply make our post-recovery want_acting calculation limit itself to up+acting. See 1db67c443d84dc5d1ff53cc820fdfd4a2128b680 for the assertion. Signed-off-by: Sage Weil (cherry picked from commit 0f2dee9aa48a00a7f2f809cd4d20e98df771da81) commit e2074cef713a973f7e8fe23055e6fa27cae50e42 Author: Kefu Chai Date: Tue Jan 3 20:40:00 2017 +0800 ceph-disk: convert none str to str before printing it Error('somethings goes wrong', e) is thrown if exception `e` is caught in ceph-disk, where e is not a string. so we can not just concat it in Error's __str__(). so cast it to str before doing so. introduced by d0e29c7 Fixes: http://tracker.ceph.com/issues/18371 Signed-off-by: Kefu Chai (cherry picked from commit 5e0dd1e7df43a3be589d17878714756a22052d8e) commit 177141ff1745e93996875b7d8d72af6f00307731 Author: Sage Weil Date: Wed Jan 18 17:02:54 2017 -0600 mon/OSDMonitor: make 'osd crush move ...' work on osds Currently it only allows you to move buckets, which is annoying and much less useful. To move an OSD you need to use create-or-move, which is harder to use. Fixes: http://tracker.ceph.com/issues/18587 Signed-off-by: Sage Weil (cherry picked from commit 47956475dea8bb8e07331dd76344a60b776b5158) commit 8c2f590338dec75dabfb1e9ae0d2c516c91b2636 Author: Boris Ranto Date: Wed Jan 25 12:39:40 2017 +0100 systemd: Start OSDs after MONs Currently, we start/stop OSDs and MONs simultaneously. This may cause problems especially when we are shutting down the system. Once the mon goes down it causes a re-election and the MONs can miss the message from the OSD that is going down. Resolves: http://tracker.ceph.com/issues/18516 Signed-off-by: Boris Ranto (cherry picked from commit 7f4acf45dd0d86e7d9992a8c30e5876fb57b1914) commit 33d9a50dec40346511f8480061dadab2a30174cc Author: Alexey Sheplyakov Date: Tue Feb 7 16:47:45 2017 +0400 ceph-osd: --flush-journal: sporadic segfaults on exit FileStore holds a number of recources like op thread pool and work queue, key/value DB threads, etc. These should be properly stopped (released) before exiting to avoid segfaults on exit. Note: more code paths (mkfs, dump_journal, etc) need similar fixes, these will be submitted as separate patches. Fixes: http://tracker.ceph.com/issues/18820 Signed-off-by: Alexey Sheplyakov (cherry picked from commit 00184814c156f6194a6ba4b696073ca1c18a3f8f) commit b464d1a5719ef36410a38cb31e8c4aab802732a6 Author: craigchi Date: Thu Feb 16 19:21:48 2017 +0800 ceph-disk: Fix getting wrong group name when --setgroup in bluestore ceph-disk prepare --setgroup will be wrong when using with bluestore Signed-off-by: craigchi (cherry picked from commit a8c0870e7370a0948e8e7fd53d3376b85bf9c649) commit 7475e442143de0f0b1919ec8dab8b6d2446eb12b Author: Mykola Golub Date: Sat Feb 4 15:27:11 2017 +0100 common: possible lockdep false alarm for ThreadPool lock Fixes: http://tracker.ceph.com/issues/18819 Signed-off-by: Mykola Golub (cherry picked from commit 8677dea4cd462d2141da28623a82b208cbc926f6) commit f65754e312f11823c5bb7e9fe24655f3fd68bcd2 Merge: 21b2aa49b3 476b535f85 Author: Loic Dachary Date: Mon Feb 13 08:19:36 2017 +0100 Merge pull request #13330 from smithfarm/wip-18870-kraken kraken: tests: SUSE yaml facets in qa/distros/all are out of date Reviewed-by: Abhishek Lekshmanan Reviewed-by: Loic Dachary commit 21b2aa49b35bbbbd87ad3b3f27920e79c569f17a Merge: 958a4c9cc1 d819ec7198 Author: Yuri Weinstein Date: Fri Feb 10 15:41:06 2017 -0800 Merge pull request #12984 from ceph/wip-cherry-pick-4tamil qa: Wip cherry pick 4tamil ceph-ansible Reviewed-by: Yuri Weinstein commit 958a4c9cc135c34f18aa83a62b726bcefbb511b7 Merge: b2955f4cc5 c968ed34b1 Author: Kefu Chai Date: Thu Feb 9 22:48:01 2017 +0800 Merge pull request #13239 from smithfarm/wip-18805-kraken kraken: tests: ignore bogus ceph-objectstore-tool error in ceph_manager Reviewed-by: Kefu Chai commit 476b535f85c5b9a8bcd294e1b1629849efe9e391 Author: Nathan Cutler Date: Wed Feb 8 21:23:54 2017 +0100 tests: drop buildpackages.py The buildpackages suite has been moved to teuthology. This cleans up a file that was left behind by https://github.com/ceph/ceph/pull/13297 Fixes: http://tracker.ceph.com/issues/18846 Signed-off-by: Nathan Cutler (cherry picked from commit 6b7443fb50c117ee7f20d53bbc7530bb0eb7ebd5) commit ad456bfa2bab14032593c76b5a7f7434381bd1e8 Author: Nathan Cutler Date: Wed Feb 8 15:27:00 2017 +0100 tests: update SUSE yaml facets in qa/distros/all Fixes: http://tracker.ceph.com/issues/18856 Signed-off-by: Nathan Cutler (cherry picked from commit 0bd56e871a8549d4b0b1211f09dad2d1120fb606) commit b2955f4cc5506224d75e64c202392ee9a39fb7bf Merge: ce8edcfed6 d24ddec58e Author: Loic Dachary Date: Wed Feb 8 18:50:52 2017 +0100 Merge pull request #13298 from dachary/wip-18849-kraken kraken: tests: remove qa/suites/buildpackages Reviewed-by: Nathan Cutler commit d24ddec58e0086ce6a5f08feb6a96c4022f13a4a Author: Loic Dachary Date: Tue Feb 7 18:33:29 2017 +0100 buildpackages: remove because it does not belong It should live in teuthology, not in Ceph. And it is currently broken: there is no need to keep it around. Fixes: http://tracker.ceph.com/issues/18846 Signed-off-by: Loic Dachary (cherry picked from commit 5a43f8d57925da227c95480501ceec10a29395d8) commit c968ed34b100baeb3a6a3245ad39c2a2d6fab04e Author: Nathan Cutler Date: Mon Feb 6 18:43:49 2017 +0100 tests: fix regression in qa/tasks/ceph_master.py https://github.com/ceph/ceph/pull/13194 introduced a regression: 2017-02-06T16:14:23.162 INFO:tasks.thrashosds.thrasher:Traceback (most recent call last): File "/home/teuthworker/src/github.com_ceph_ceph_master/qa/tasks/ceph_manager.py", line 722, in wrapper return func(self) File "/home/teuthworker/src/github.com_ceph_ceph_master/qa/tasks/ceph_manager.py", line 839, in do_thrash self.choose_action()() File "/home/teuthworker/src/github.com_ceph_ceph_master/qa/tasks/ceph_manager.py", line 305, in kill_osd output = proc.stderr.getvalue() AttributeError: 'NoneType' object has no attribute 'getvalue' This is because the original patch failed to pass "stderr=StringIO()" to run(). Fixes: http://tracker.ceph.com/issues/16263 Signed-off-by: Nathan Cutler Signed-off-by: Kefu Chai (cherry picked from commit db2582e25e390fcaf75952eb59a73dcff643f49c) commit d8c0141a7506cd38d385270f5a1cf315d808974e Author: Sage Weil Date: Fri Dec 30 17:28:59 2016 -0500 osd/PG: publish PG stats when backfill-related states change These frequently get flushed because other updates happen, but we should explicitly ensure that the mon sees these state changes. Fixes: http://tracker.ceph.com/issues/18369 Signed-off-by: Sage Weil (cherry picked from commit d4adeb7500a113097fdd717ada0231f68badafbb) commit 1d25327b00f01e987dec022412edb95040d69a42 Author: Sage Weil Date: Thu Jan 26 14:22:53 2017 -0500 os/bluestore: fix statfs to not include DB partition in free space If we report the DB space as vailable, ceph thinks the OSD can store more data and will not mark the cluster as full as easily. And in reality, we can't actually store data in this space--only metadata. Avoid the problem by not reporting it as available. Fixes: http://tracker.ceph.com/issues/18599 Signed-off-by: Sage Weil (cherry picked from commit c66d5babb1e283869ba0f1f59029bead5ca5f37d) commit 8ea14ce9ce185204f6906b3919f7fac1419a26a9 Author: Zack Cerza Date: Thu Jan 12 14:55:26 2017 -0700 Add openstack requirements to smoke suite Signed-off-by: Zack Cerza (cherry picked from commit fe9b7552d116b150f178c4cc778fa27cd8d956d1) commit af2af7015adb0e6617da619ffe89e70e0904fcec Author: Sage Weil Date: Fri Feb 3 17:38:05 2017 -0500 osd: do not send ENXIO on misdirected op by default In practice this tends to get bubbled up the stack as an error on the caller, and they usually do not handle it properly. For example, with librbd, this turns into EIO and break the VM. Instead, this will manifest as a hung op on the client. That is also not ideal, but given that the root cause here is generally a bug, it's not clear what else would be better. We already log an error in the cluster log, so teuthology runs will continue to fail. Signed-off-by: Sage Weil (cherry picked from commit 923e7f5ce5ed437af15e178299a61029ff48e4a2) # Conflicts: # PendingReleaseNotes commit 85bda6e263400118830aa40a889bdb695d321b58 Author: Dongsheng Yang Date: Thu Dec 22 21:00:41 2016 -0500 librbd: don't remove an image w/ incompatible features Fixes: http://tracker.ceph.com/issues/18315 Signed-off-by: Dongsheng Yang (cherry picked from commit f76127b5e617923d14adb62bfb836a635c14f209) commit d96ae9ead2d2f58180e533a63c640098d5134047 Author: Jason Dillaman Date: Tue Jan 17 11:55:00 2017 -0500 osdc: cache should ignore error bhs during trim A read error (such as injecting a timeout into an OSD op) might result in a bh in an error state. These should be trimable by the cache. Fixes: http://tracker.ceph.com/issues/18436 Signed-off-by: Jason Dillaman (cherry picked from commit 5910ed9de9856b5821488a1836487bbbd3d6460e) commit 9620088542f80efb97960b4d9d51876ff1784aa9 Author: Nathan Cutler Date: Tue Jan 31 00:46:22 2017 +0100 tests: ignore bogus ceph-objectstore-tool error in ceph_manager Fixes: http://tracker.ceph.com/issues/16263 Signed-off-by: Nathan Cutler (cherry picked from commit 046e873026c59e733f1844b28ffdc030cbe57b36) commit baa42b6d7c616e4d3518416fcb86fef5c12a82db Author: Piotr Dałek Date: Tue Jan 31 16:07:18 2017 +0100 OSD: allow client throttler to be adjusted on-fly, without restart This patch allows the osd_client_message_cap and osd_client_message_size_cap to be adjusted on-fly, using admin socket functionality. Fixes: http://tracker.ceph.com/issues/18791 Signed-off-by: Piotr Dałek (cherry picked from commit 64c309d7e18a975931b526e6f5d6f610c3a0d632) commit ab8558e05e765683691799acc4cfb2280cd537b2 Author: Jason Dillaman Date: Wed Jan 25 14:45:56 2017 -0500 librbd: prevent self-blacklisting during break lock Fixes: http://tracker.ceph.com/issues/18666 Signed-off-by: Jason Dillaman (cherry picked from commit 5c590acaec4dd66a9a8c3aa0ec8ab904dd350216) Conflicts: src/librbd/exclusive_lock/BreakRequest.cc: managed lock refactor src/test/librbd/exclusive_lock/test_mock_BreakRequest.cc: managed lock refactor commit 16b2fd00ecf7d7fcc7590f0ca030ff33a32f2196 Author: Wido den Hollander Date: Mon Jan 23 08:18:27 2017 +0100 systemd: Restart Mon after 10s in case of failure In some situations the IP address the Monitor wants to bind to might not be available yet. This might for example be a IPv6 Address which is still performing DAD or waiting for a Router Advertisement to be send by the Router(s). Have systemd wait for 10s before starting the Mon and increase the amount of times it does so to 5. This allows the system to bring up IP Addresses in the mean time while systemd waits with restarting the Mon. Fixes: #18635 Signed-off-by: Wido den Hollander (cherry picked from commit e73eb8cc1e0d45af1f0b7852c551f2ddfb82a520) commit 29f55d7dd3f35423294fa63cd1e3ccb18e6f7449 Author: Abhishek Lekshmanan Date: Fri Jan 13 16:32:55 2017 +0100 rgw_admin: read master log shards from master's current period Also make the sync output look similar to the output of data sync Signed-off-by: Abhishek Lekshmanan (cherry picked from commit cc306c506ca6607223cb89cd388f8e18673c4fe2) commit 0aebe210079e456c23e857e9c684495813151726 Author: Abhishek Lekshmanan Date: Fri Jan 13 16:29:47 2017 +0100 rgw: allow getting master log shards info on specified period This is needed for rgw admin's sync status or else we end up always publishing that we're behind since we are always checking against master's first period to sync from Signed-off-by: Abhishek Lekshmanan (cherry picked from commit 063c949d4409a18a22b64791d497e20f7473bc01) commit 17d68c22da7c3eac336f5ea8ecac13b5e8d75568 Author: Abhishek Lekshmanan Date: Thu Jan 12 22:09:01 2017 +0100 rgw_admin: get master's period from store's current period info This ensures that we get the current period in contrast to the admin log which gets the master's earliest period. Fixes: http://tracker.ceph.com/issues/18064 Signed-off-by: Abhishek Lekshmanan (cherry picked from commit 4ca18df7198a9f0ded8b0100a70b5db7187c3de4) commit 478d0315cb8457ecfd5cb6f96c82d12e066c7c2f Author: Orit Wasserman Date: Sun Dec 25 12:36:34 2016 +0200 rgw: clear master_zonegroup when reseting RGWPeriodMap Fixes: http://tracker.ceph.com/issues/17239 Signed-off-by: Orit Wasserman (cherry picked from commit d8f42fe6be659c1d48bf04b30aa54ad616936145) commit bfc058d8f951d7694104433f709fc73940511b0a Author: Michal Koutný Date: Wed Jan 18 20:15:29 2017 +0100 rgw: Use decoded URI when verifying TempURL Instead of calliing url_decode directly, we reuse s->decoded_uri that is initialized in RGWREST::preprocess(). Fixes: http://tracker.ceph.com/issues/18590 Signed-off-by: Michal Koutný (cherry picked from commit 4e1318f4dcbfd64c3ec94f4addf6e38ddd6c013a) commit deb6d2c64c432869caf8747a75ed2e555acc772b Author: Ricardo Dias Date: Tue Jan 10 15:11:19 2017 +0000 librbd: allow to open an image without opening parent image Fixes: http://tracker.ceph.com/issues/18325 Signed-off-by: Ricardo Dias (cherry picked from commit 61af1c25015de087a2423811548d975dd7d430b4) Conflicts: src/librbd/exclusive_lock/PostAcquireRequest.cc - deleted, does not exist in kraken src/librbd/librbd.cc - removed rbd_group_* functions that don't exist in kraken commit 35b13c6687026f08c100a28f2775d6e772a0e734 Author: Sage Weil Date: Thu Dec 29 12:08:28 2016 -0500 mon/OSDMonitor: set last_force_op_resend on overlay pool too We currently set the last_force_op_resend field on the base pool when we set or clear the overlay. Set it on the cache/overlay pool too. The Objecter should resend even with a change only to the base pool, but the OSD needs to see the change on the overlay pool to correctly discard the op. Fixes: http://tracker.ceph.com/issues/18366 Signed-off-by: Sage Weil (cherry picked from commit 08c3618771b852840aa88cff1ca98d980d802941) commit ce8edcfed6cd908779efd229202eab1232d16f1c Merge: 16fc6d8a2a 7db1bf762c Author: John Spray Date: Thu Jan 26 00:33:46 2017 +0100 Merge pull request #13024 from SUSE/wip-18604-kraken kraken: cephfs test failures (ceph.com/qa is broken, should be download.ceph.com/qa) commit 16fc6d8a2af89bb8a620298729d3951ef32945f7 Merge: f36efa76d8 ed82de11f5 Author: John Spray Date: Thu Jan 26 00:31:53 2017 +0100 Merge pull request #12813 from SUSE/wip-18439-kraken kraken: fuse: TestVolumeClient.test_evict_client failure creating pidfile commit f36efa76d83205783ac5931ea678357ebddc62f6 Merge: 08b560c51b 61ca8fc351 Author: John Spray Date: Thu Jan 26 00:29:55 2017 +0100 Merge pull request #12951 from jcsp/wip-18361-kraken kraken: client: populate metadata during mount commit 08b560c51b4218697b922cb208da2708f8c50021 Merge: d50af9e948 8d0eb6e287 Author: John Spray Date: Thu Jan 26 00:29:00 2017 +0100 Merge pull request #13030 from SUSE/wip-18612-kraken kraken: client: segfault on ceph_rmdir path / commit d50af9e9488501778d1a82d32ba4042ecb5fca54 Merge: 0fbf923290 8b79964696 Author: John Spray Date: Thu Jan 26 00:27:57 2017 +0100 Merge pull request #13028 from SUSE/wip-18531-kraken kraken: speed up readdir by skipping unwanted dn commit 0fbf9232903b8e7d7d93c91ea0dafcb4ff8f3c15 Merge: e4d348b3d8 1ac9886896 Author: John Spray Date: Thu Jan 26 00:25:34 2017 +0100 Merge pull request #12835 from SUSE/wip-18463-kraken kraken: Decode errors on backtrace will crash MDS commit 29c8c3fb861a7153a13e3f56aaa7772c9116c751 Author: Yan, Zheng Date: Tue Jan 10 11:16:47 2017 +0800 mds: finish clientreplay requests before requesting active state All clientreplay requests' finish contexts should be executed before MDCache::export_remaining_imported_caps(). Otherwise MDCache::try_reconnect_cap() may fail to reconnect client caps. Fixes: http://tracker.ceph.com/issues/18461 Signed-off-by: Yan, Zheng (cherry picked from commit 80dae314ee90e79d60e2cfee301e43a435c10801) commit 2b7083f8dd4c8f281f6f8ccbd13f06f08de6bcc1 Author: David Zafman Date: Tue Jan 24 16:35:23 2017 -0800 Revert "PrimaryLogPG::failed_push: update missing as well" This reverts commit dd48b972afde2dfa9ab1a6942c7961750222986d. Fixes: http://tracker.ceph.com/issues/18659 Signed-off-by: David Zafman commit cc046597983bd491cc66081cc33d9046264fe24b Author: Jason Dillaman Date: Tue Jan 24 09:24:52 2017 -0500 librbd: improve debug logging for lock / watch state machines Signed-off-by: Jason Dillaman commit c5976ac36751e15fd81c67945a2c6d049dc7b316 Author: Jason Dillaman Date: Mon Jan 23 21:24:41 2017 -0500 test: use librados API to retrieve config params The CephContext object is not ABI-stable, so it is necessary to use the "conf_get" librados methods to safely retrieve a setting. Fixes: http://tracker.ceph.com/issues/18617 Signed-off-by: Jason Dillaman (cherry picked from commit 8ad40645ac3948f8341e9a80ce8aff8ac5b9ad11) Conflicts: src/test/librbd/test_librbd.cc: trivial resolution commit e4d348b3d850465ae25d7b3cbc8f21fac20ae478 Merge: 03458fae64 f583485a4e Author: Radoslaw Zarzynski Date: Mon Jan 23 20:46:55 2017 +0100 Merge pull request #13006 from rzarzynski/wip-rgw-18476-kraken kraken: rgw: fix handling RGWUserInfo::system in RGWHandler_REST_SWIFT. Reviewed-by: Yehuda Sadeh commit f583485a4eacdf489ce00f93cd49dc147bfdb5f9 Author: Radoslaw Zarzynski Date: Tue Jan 10 12:09:50 2017 +0100 rgw: fix handling RGWUserInfo::system in RGWHandler_REST_SWIFT. Before this patch the flag was wrongly handled in the Swift API implementation. In rare conditions this might result in setting req_state::system_request. This may happen only if both of those conditions are fulfilled: * RadosGW is running in a multi-site configuration (at least one user with the system flag turned on is present), * the "rgw_swift_account_in_url" configurable has been switched to true. The value is false by default and our documentation doesn't actually mention about the option. The issue doesn't affect Jewel nor any previous release. Fixes: http://tracker.ceph.com/issues/18476 Signed-off-by: Radoslaw Zarzynski (cherry picked from commit 8dac93392b6679c3ad9bb28ea66f6bb8c7be511d) commit 03458fae64df24d58426e5b3274dd1981ef42fc6 Merge: 61b1beef1d dca2265c41 Author: Nathan Cutler Date: Mon Jan 23 19:29:57 2017 +0100 Merge pull request #13044 from SUSE/wip-18571-kraken kraken: Python Swift client commands in Quick Developer Guide don't match configuration in vstart.sh Reviewed-by: Casey Bodley commit 7db1bf762c5503933bbbb266e14fc1bd69120dd8 Author: John Spray Date: Tue Jan 17 17:12:46 2017 +0100 qa: update remaining ceph.com to download.ceph.com Fixes: http://tracker.ceph.com/issues/18574 Signed-off-by: John Spray (cherry picked from commit 549d993d3fd8ffffa280ed4a64aca41d1c6f2da1) commit 0e0d149895198ee74cff85353eabf19aa4677258 Author: Mingxin Liu Date: Mon Jan 2 13:20:10 2017 +0800 mon: do not send duplicated osdmap msg to not sync'ed osd prior to this change: a peon may forward the pgstats to leader, and record it locally, but leader will check if osd has the latest map before process, if not, will use a route op to indicate peon to send it, then poen will delete routed op when fininaly send out which make peon cannot send pgstatack when leader has processed the pgstat update. so osd will always track it util reach a threshold block pgstats sending, at worst, reopen mon session. also, both leader and peon will send out the osdmap message to the osd. after this change: only the peon will send out the osdmap message. and the pgstatack message will be routed to the osd as expected. so the osd will not keep track of the "acked" pg stats in its queue forever before times out. Fixes: http://tracker.ceph.com/issues/18458 Signed-off-by: Mingxin Liu (cherry picked from commit 57274488c072ec6912b700288ce5b1ea8372d162) commit dca2265c41bf035855b30d3279e1ec5726d74ffc Author: Ronak Jain Date: Fri Jan 13 16:57:45 2017 +0530 Doc: Fixes Python Swift client commands Fixes: http://tracker.ceph.com/issues/17746 Signed-off-by: Ronak Jain (cherry picked from commit 8c79959557d60f619adf1a3ed1b5bd1112ceaabb) commit 8d0eb6e2870593b94cb71f29bcac7a1f422cd101 Author: Michal Jarzabek Date: Thu Jan 12 21:22:20 2017 +0000 client/Client.cc: prevent segfaulting The segfaulting in the rmdir function is caused by calling filepath::last_dentry() function. last_dentry() function assumes that the bits vector has always at least one element, which is not the case for the the filepath object created with "/" input. This commit also fixes other functions affected by this bug: link, unlink, rename, mkdir, mknod and symlink. Fixes: http://tracker.ceph.com/issues/9935 Signed-off-by: Michal Jarzabek (cherry picked from commit 6ed7f2364ae5507bab14c60b582929aa7b0ba400) commit 61b1beef1dc4802c32367fc71968101a09042c15 Merge: f223e27eeb 6206e1998a Author: Sage Weil Date: Fri Jan 20 11:14:24 2017 -0600 Merge pull request #13011 from liewegas/wip-18595-kraken os/bluestore: fix Allocator::allocate() int truncation Reviewed-by: Igor Fedotov commit 8b7996469652d01143806e1e815894afde3f426b Author: Xiaoxi Chen Date: Tue Jan 10 19:11:08 2017 -0700 mds/server: skip unwanted dn in handle_client_readdir We can skip unwanted dn which < (offset_key, snap) via map.lower_bound, rather than iterate across them. Previously we iterate and skip dn which < (offset_key, dn->last), as dn->last >= snap means (offset_key, dn->last) >= (offset_key, snap), and such iterate_and_skip logic still keep, so this commit doesnt change code logic but an optimization. Signed-off-by: Xiaoxi Chen (cherry picked from commit 52fe52baf920c672ac7f63a3087dcd31137891b6) commit ecb459f0837dd05d2802320e7fac3246025e6c13 Author: Samuel Matzek Date: Mon Jan 16 11:11:31 2017 -0600 Ceph-disk to use correct user in check_journal_req The ceph-disk tool calls ceph-osd to check the journal requirements using OSD id 0. This creates a log file for osd-0 on the system using the current user/group for file permissions. When ceph-disk is run as root this makes the file owned by root which makes the osd daemon for osd.0 unable to write to its own log file. This commit changes the journal reqs calls of ceph-osd to pass the ceph user and group so ceph-osd creates the log file with the appropriate permissions. Fixes: http://tracker.ceph.com/issues/18538 Signed-off-by: Samuel Matzek (cherry picked from commit bcf7514bf53693ec61e482341787c80494589faf) commit 6206e1998a1636f6339ef2f1bd2d67a69cd3abf0 Author: Sage Weil Date: Thu Jan 19 19:51:45 2017 -0600 os/bluestore: fix Allocator::allocate() int truncation An allocation of size 0x80000000 gets truncated to 0. Larger values can break things in similar ways. Introduced in 5ab034345d7320fbc86a2133c0c29ec1aca4b71a. Fixes: http://tracker.ceph.com/issues/18595 Signed-off-by: Sage Weil (cherry picked from commit e19aa8484632ac7d83aa5dc868a1fe4dc167d9b9)