blob: 257301a3d730f09d15b88fb7e3097b51e9cbe394 (
plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
|
Using the original execution date of: 2017-05-03 13:33:24Z
Current cluster status:
* Node List:
* Node controller-1: UNCLEAN (offline)
* Online: [ controller-0 controller-2 ]
* RemoteOnline: [ galera-0 galera-1 galera-2 messaging-0 messaging-1 messaging-2 ]
* Full List of Resources:
* messaging-0 (ocf:pacemaker:remote): Started controller-0
* messaging-1 (ocf:pacemaker:remote): Started controller-1 (UNCLEAN)
* messaging-2 (ocf:pacemaker:remote): Started controller-0
* galera-0 (ocf:pacemaker:remote): Started controller-1 (UNCLEAN)
* galera-1 (ocf:pacemaker:remote): Started controller-0
* galera-2 (ocf:pacemaker:remote): Started controller-1 (UNCLEAN)
* Clone Set: rabbitmq-clone [rabbitmq]:
* Started: [ messaging-0 messaging-1 messaging-2 ]
* Stopped: [ controller-0 controller-1 controller-2 galera-0 galera-1 galera-2 ]
* Clone Set: galera-master [galera] (promotable):
* Promoted: [ galera-0 galera-1 galera-2 ]
* Stopped: [ controller-0 controller-1 controller-2 messaging-0 messaging-1 messaging-2 ]
* Clone Set: redis-master [redis] (promotable):
* redis (ocf:heartbeat:redis): Unpromoted controller-1 (UNCLEAN)
* Promoted: [ controller-0 ]
* Unpromoted: [ controller-2 ]
* Stopped: [ galera-0 galera-1 galera-2 messaging-0 messaging-1 messaging-2 ]
* ip-192.168.24.6 (ocf:heartbeat:IPaddr2): Started controller-0
* ip-10.0.0.102 (ocf:heartbeat:IPaddr2): Started controller-0
* ip-172.17.1.14 (ocf:heartbeat:IPaddr2): Started controller-1 (UNCLEAN)
* ip-172.17.1.17 (ocf:heartbeat:IPaddr2): Started controller-1 (UNCLEAN)
* ip-172.17.3.15 (ocf:heartbeat:IPaddr2): Started controller-0
* ip-172.17.4.11 (ocf:heartbeat:IPaddr2): Started controller-1 (UNCLEAN)
* Clone Set: haproxy-clone [haproxy]:
* haproxy (systemd:haproxy): Started controller-1 (UNCLEAN)
* Started: [ controller-0 controller-2 ]
* Stopped: [ galera-0 galera-1 galera-2 messaging-0 messaging-1 messaging-2 ]
* openstack-cinder-volume (systemd:openstack-cinder-volume): Started controller-0
* stonith-fence_ipmilan-525400bbf613 (stonith:fence_ipmilan): Started controller-0
* stonith-fence_ipmilan-525400b4f6bd (stonith:fence_ipmilan): Started controller-0
* stonith-fence_ipmilan-5254005bdbb5 (stonith:fence_ipmilan): Started controller-1 (UNCLEAN)
Transition Summary:
* Fence (reboot) messaging-1 'resources are active but connection is unrecoverable'
* Fence (reboot) galera-2 'resources are active but connection is unrecoverable'
* Fence (reboot) controller-1 'peer is no longer part of the cluster'
* Stop messaging-1 ( controller-1 ) due to node availability
* Move galera-0 ( controller-1 -> controller-2 )
* Stop galera-2 ( controller-1 ) due to node availability
* Stop rabbitmq:2 ( messaging-1 ) due to node availability
* Stop galera:1 ( Promoted galera-2 ) due to node availability
* Stop redis:0 ( Unpromoted controller-1 ) due to node availability
* Move ip-172.17.1.14 ( controller-1 -> controller-2 )
* Move ip-172.17.1.17 ( controller-1 -> controller-2 )
* Move ip-172.17.4.11 ( controller-1 -> controller-2 )
* Stop haproxy:0 ( controller-1 ) due to node availability
* Move stonith-fence_ipmilan-5254005bdbb5 ( controller-1 -> controller-2 )
Executing Cluster Transition:
* Pseudo action: messaging-1_stop_0
* Pseudo action: galera-0_stop_0
* Pseudo action: galera-2_stop_0
* Pseudo action: rabbitmq-clone_pre_notify_stop_0
* Pseudo action: galera-master_demote_0
* Pseudo action: redis-master_pre_notify_stop_0
* Pseudo action: stonith-fence_ipmilan-5254005bdbb5_stop_0
* Fencing controller-1 (reboot)
* Resource action: rabbitmq notify on messaging-2
* Resource action: rabbitmq notify on messaging-0
* Pseudo action: rabbitmq-clone_confirmed-pre_notify_stop_0
* Pseudo action: redis_post_notify_stop_0
* Resource action: redis notify on controller-0
* Resource action: redis notify on controller-2
* Pseudo action: redis-master_confirmed-pre_notify_stop_0
* Pseudo action: redis-master_stop_0
* Pseudo action: haproxy-clone_stop_0
* Fencing galera-2 (reboot)
* Pseudo action: galera_demote_0
* Pseudo action: galera-master_demoted_0
* Pseudo action: galera-master_stop_0
* Pseudo action: redis_stop_0
* Pseudo action: redis-master_stopped_0
* Pseudo action: haproxy_stop_0
* Pseudo action: haproxy-clone_stopped_0
* Fencing messaging-1 (reboot)
* Resource action: galera-0 start on controller-2
* Pseudo action: rabbitmq_post_notify_stop_0
* Pseudo action: rabbitmq-clone_stop_0
* Pseudo action: galera_stop_0
* Resource action: galera monitor=10000 on galera-0
* Pseudo action: galera-master_stopped_0
* Pseudo action: redis-master_post_notify_stopped_0
* Pseudo action: ip-172.17.1.14_stop_0
* Pseudo action: ip-172.17.1.17_stop_0
* Pseudo action: ip-172.17.4.11_stop_0
* Resource action: stonith-fence_ipmilan-5254005bdbb5 start on controller-2
* Resource action: galera-0 monitor=20000 on controller-2
* Pseudo action: rabbitmq_stop_0
* Pseudo action: rabbitmq-clone_stopped_0
* Resource action: redis notify on controller-0
* Resource action: redis notify on controller-2
* Pseudo action: redis-master_confirmed-post_notify_stopped_0
* Resource action: ip-172.17.1.14 start on controller-2
* Resource action: ip-172.17.1.17 start on controller-2
* Resource action: ip-172.17.4.11 start on controller-2
* Resource action: stonith-fence_ipmilan-5254005bdbb5 monitor=60000 on controller-2
* Pseudo action: rabbitmq-clone_post_notify_stopped_0
* Pseudo action: redis_notified_0
* Resource action: ip-172.17.1.14 monitor=10000 on controller-2
* Resource action: ip-172.17.1.17 monitor=10000 on controller-2
* Resource action: ip-172.17.4.11 monitor=10000 on controller-2
* Resource action: rabbitmq notify on messaging-2
* Resource action: rabbitmq notify on messaging-0
* Pseudo action: rabbitmq_notified_0
* Pseudo action: rabbitmq-clone_confirmed-post_notify_stopped_0
Using the original execution date of: 2017-05-03 13:33:24Z
Revised Cluster Status:
* Node List:
* Online: [ controller-0 controller-2 ]
* OFFLINE: [ controller-1 ]
* RemoteOnline: [ galera-0 galera-1 messaging-0 messaging-2 ]
* RemoteOFFLINE: [ galera-2 messaging-1 ]
* Full List of Resources:
* messaging-0 (ocf:pacemaker:remote): Started controller-0
* messaging-1 (ocf:pacemaker:remote): Stopped
* messaging-2 (ocf:pacemaker:remote): Started controller-0
* galera-0 (ocf:pacemaker:remote): Started controller-2
* galera-1 (ocf:pacemaker:remote): Started controller-0
* galera-2 (ocf:pacemaker:remote): Stopped
* Clone Set: rabbitmq-clone [rabbitmq]:
* Started: [ messaging-0 messaging-2 ]
* Stopped: [ controller-0 controller-1 controller-2 galera-0 galera-1 galera-2 messaging-1 ]
* Clone Set: galera-master [galera] (promotable):
* Promoted: [ galera-0 galera-1 ]
* Stopped: [ controller-0 controller-1 controller-2 galera-2 messaging-0 messaging-1 messaging-2 ]
* Clone Set: redis-master [redis] (promotable):
* Promoted: [ controller-0 ]
* Unpromoted: [ controller-2 ]
* Stopped: [ controller-1 galera-0 galera-1 galera-2 messaging-0 messaging-1 messaging-2 ]
* ip-192.168.24.6 (ocf:heartbeat:IPaddr2): Started controller-0
* ip-10.0.0.102 (ocf:heartbeat:IPaddr2): Started controller-0
* ip-172.17.1.14 (ocf:heartbeat:IPaddr2): Started controller-2
* ip-172.17.1.17 (ocf:heartbeat:IPaddr2): Started controller-2
* ip-172.17.3.15 (ocf:heartbeat:IPaddr2): Started controller-0
* ip-172.17.4.11 (ocf:heartbeat:IPaddr2): Started controller-2
* Clone Set: haproxy-clone [haproxy]:
* Started: [ controller-0 controller-2 ]
* Stopped: [ controller-1 galera-0 galera-1 galera-2 messaging-0 messaging-1 messaging-2 ]
* openstack-cinder-volume (systemd:openstack-cinder-volume): Started controller-0
* stonith-fence_ipmilan-525400bbf613 (stonith:fence_ipmilan): Started controller-0
* stonith-fence_ipmilan-525400b4f6bd (stonith:fence_ipmilan): Started controller-0
* stonith-fence_ipmilan-5254005bdbb5 (stonith:fence_ipmilan): Started controller-2
|
