blob: 612e7015da2b1f841df55720ee513b1ff4741592 (
plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
|
Current cluster status:
* Node List:
* RemoteNode database-0: UNCLEAN (offline)
* RemoteNode database-2: UNCLEAN (offline)
* Online: [ controller-0 controller-1 controller-2 ]
* RemoteOnline: [ database-1 messaging-0 messaging-1 messaging-2 ]
* GuestOnline: [ galera-bundle-1 rabbitmq-bundle-0 rabbitmq-bundle-1 rabbitmq-bundle-2 redis-bundle-0 redis-bundle-2 ]
* Full List of Resources:
* database-0 (ocf:pacemaker:remote): Stopped
* database-1 (ocf:pacemaker:remote): Started controller-2
* database-2 (ocf:pacemaker:remote): Stopped
* messaging-0 (ocf:pacemaker:remote): Started controller-2
* messaging-1 (ocf:pacemaker:remote): Started controller-2
* messaging-2 (ocf:pacemaker:remote): Started controller-2
* Container bundle set: rabbitmq-bundle [192.168.24.1:8787/rhosp12/openstack-rabbitmq-docker:pcmklatest]:
* rabbitmq-bundle-0 (ocf:heartbeat:rabbitmq-cluster): Started messaging-0
* rabbitmq-bundle-1 (ocf:heartbeat:rabbitmq-cluster): Started messaging-1
* rabbitmq-bundle-2 (ocf:heartbeat:rabbitmq-cluster): Started messaging-2
* Container bundle set: galera-bundle [192.168.24.1:8787/rhosp12/openstack-mariadb-docker:pcmklatest]:
* galera-bundle-0 (ocf:heartbeat:galera): FAILED Promoted database-0 (UNCLEAN)
* galera-bundle-1 (ocf:heartbeat:galera): Promoted database-1
* galera-bundle-2 (ocf:heartbeat:galera): FAILED Promoted database-2 (UNCLEAN)
* Container bundle set: redis-bundle [192.168.24.1:8787/rhosp12/openstack-redis-docker:pcmklatest]:
* redis-bundle-0 (ocf:heartbeat:redis): Unpromoted controller-0
* redis-bundle-1 (ocf:heartbeat:redis): Stopped
* redis-bundle-2 (ocf:heartbeat:redis): Unpromoted controller-2
* ip-192.168.24.11 (ocf:heartbeat:IPaddr2): Stopped
* ip-10.0.0.104 (ocf:heartbeat:IPaddr2): Stopped
* ip-172.17.1.19 (ocf:heartbeat:IPaddr2): Started controller-2
* ip-172.17.1.11 (ocf:heartbeat:IPaddr2): Stopped
* ip-172.17.3.13 (ocf:heartbeat:IPaddr2): Stopped
* ip-172.17.4.19 (ocf:heartbeat:IPaddr2): Started controller-2
* Container bundle set: haproxy-bundle [192.168.24.1:8787/rhosp12/openstack-haproxy-docker:pcmklatest]:
* haproxy-bundle-docker-0 (ocf:heartbeat:docker): Started controller-0
* haproxy-bundle-docker-1 (ocf:heartbeat:docker): Stopped
* haproxy-bundle-docker-2 (ocf:heartbeat:docker): Started controller-2
* openstack-cinder-volume (systemd:openstack-cinder-volume): Stopped
* stonith-fence_ipmilan-525400244e09 (stonith:fence_ipmilan): Started controller-2
* stonith-fence_ipmilan-525400cdec10 (stonith:fence_ipmilan): Started controller-2
* stonith-fence_ipmilan-525400c709f7 (stonith:fence_ipmilan): Stopped
* stonith-fence_ipmilan-525400a7f9e0 (stonith:fence_ipmilan): Started controller-0
* stonith-fence_ipmilan-525400a25787 (stonith:fence_ipmilan): Started controller-2
* stonith-fence_ipmilan-5254005ea387 (stonith:fence_ipmilan): Stopped
* stonith-fence_ipmilan-525400542c06 (stonith:fence_ipmilan): Stopped
* stonith-fence_ipmilan-525400aac413 (stonith:fence_ipmilan): Started controller-2
* stonith-fence_ipmilan-525400498d34 (stonith:fence_ipmilan): Stopped
Transition Summary:
* Fence (reboot) galera-bundle-2 (resource: galera-bundle-docker-2) 'guest is unclean'
* Fence (reboot) galera-bundle-0 (resource: galera-bundle-docker-0) 'guest is unclean'
* Start database-0 ( controller-0 )
* Start database-2 ( controller-1 )
* Recover galera-bundle-docker-0 ( database-0 )
* Start galera-bundle-0 ( controller-0 )
* Recover galera:0 ( Promoted galera-bundle-0 )
* Recover galera-bundle-docker-2 ( database-2 )
* Start galera-bundle-2 ( controller-1 )
* Recover galera:2 ( Promoted galera-bundle-2 )
* Promote redis:0 ( Unpromoted -> Promoted redis-bundle-0 )
* Start redis-bundle-docker-1 ( controller-1 )
* Start redis-bundle-1 ( controller-1 )
* Start redis:1 ( redis-bundle-1 )
* Start ip-192.168.24.11 ( controller-0 )
* Start ip-10.0.0.104 ( controller-1 )
* Start ip-172.17.1.11 ( controller-0 )
* Start ip-172.17.3.13 ( controller-1 )
* Start haproxy-bundle-docker-1 ( controller-1 )
* Start openstack-cinder-volume ( controller-0 )
* Start stonith-fence_ipmilan-525400c709f7 ( controller-1 )
* Start stonith-fence_ipmilan-5254005ea387 ( controller-1 )
* Start stonith-fence_ipmilan-525400542c06 ( controller-0 )
* Start stonith-fence_ipmilan-525400498d34 ( controller-1 )
Executing Cluster Transition:
* Resource action: database-0 start on controller-0
* Resource action: database-2 start on controller-1
* Pseudo action: redis-bundle-master_pre_notify_start_0
* Resource action: stonith-fence_ipmilan-525400c709f7 start on controller-1
* Resource action: stonith-fence_ipmilan-5254005ea387 start on controller-1
* Resource action: stonith-fence_ipmilan-525400542c06 start on controller-0
* Resource action: stonith-fence_ipmilan-525400498d34 start on controller-1
* Pseudo action: redis-bundle_start_0
* Pseudo action: galera-bundle_demote_0
* Resource action: database-0 monitor=20000 on controller-0
* Resource action: database-2 monitor=20000 on controller-1
* Pseudo action: galera-bundle-master_demote_0
* Resource action: redis notify on redis-bundle-0
* Resource action: redis notify on redis-bundle-2
* Pseudo action: redis-bundle-master_confirmed-pre_notify_start_0
* Pseudo action: redis-bundle-master_start_0
* Resource action: stonith-fence_ipmilan-525400c709f7 monitor=60000 on controller-1
* Resource action: stonith-fence_ipmilan-5254005ea387 monitor=60000 on controller-1
* Resource action: stonith-fence_ipmilan-525400542c06 monitor=60000 on controller-0
* Resource action: stonith-fence_ipmilan-525400498d34 monitor=60000 on controller-1
* Pseudo action: galera_demote_0
* Pseudo action: galera_demote_0
* Pseudo action: galera-bundle-master_demoted_0
* Pseudo action: galera-bundle_demoted_0
* Pseudo action: galera-bundle_stop_0
* Resource action: galera-bundle-docker-0 stop on database-0
* Resource action: galera-bundle-docker-2 stop on database-2
* Pseudo action: stonith-galera-bundle-2-reboot on galera-bundle-2
* Pseudo action: stonith-galera-bundle-0-reboot on galera-bundle-0
* Pseudo action: galera-bundle-master_stop_0
* Resource action: redis-bundle-docker-1 start on controller-1
* Resource action: redis-bundle-1 monitor on controller-1
* Resource action: ip-192.168.24.11 start on controller-0
* Resource action: ip-10.0.0.104 start on controller-1
* Resource action: ip-172.17.1.11 start on controller-0
* Resource action: ip-172.17.3.13 start on controller-1
* Resource action: openstack-cinder-volume start on controller-0
* Pseudo action: haproxy-bundle_start_0
* Pseudo action: galera_stop_0
* Resource action: redis-bundle-docker-1 monitor=60000 on controller-1
* Resource action: redis-bundle-1 start on controller-1
* Resource action: ip-192.168.24.11 monitor=10000 on controller-0
* Resource action: ip-10.0.0.104 monitor=10000 on controller-1
* Resource action: ip-172.17.1.11 monitor=10000 on controller-0
* Resource action: ip-172.17.3.13 monitor=10000 on controller-1
* Resource action: haproxy-bundle-docker-1 start on controller-1
* Resource action: openstack-cinder-volume monitor=60000 on controller-0
* Pseudo action: haproxy-bundle_running_0
* Pseudo action: galera_stop_0
* Pseudo action: galera-bundle-master_stopped_0
* Resource action: redis start on redis-bundle-1
* Pseudo action: redis-bundle-master_running_0
* Resource action: redis-bundle-1 monitor=30000 on controller-1
* Resource action: haproxy-bundle-docker-1 monitor=60000 on controller-1
* Pseudo action: galera-bundle_stopped_0
* Pseudo action: galera-bundle_start_0
* Pseudo action: galera-bundle-master_start_0
* Resource action: galera-bundle-docker-0 start on database-0
* Resource action: galera-bundle-0 monitor on controller-1
* Resource action: galera-bundle-docker-2 start on database-2
* Resource action: galera-bundle-2 monitor on controller-1
* Pseudo action: redis-bundle-master_post_notify_running_0
* Resource action: galera-bundle-docker-0 monitor=60000 on database-0
* Resource action: galera-bundle-0 start on controller-0
* Resource action: galera-bundle-docker-2 monitor=60000 on database-2
* Resource action: galera-bundle-2 start on controller-1
* Resource action: redis notify on redis-bundle-0
* Resource action: redis notify on redis-bundle-2
* Resource action: redis notify on redis-bundle-1
* Pseudo action: redis-bundle-master_confirmed-post_notify_running_0
* Pseudo action: redis-bundle_running_0
* Resource action: galera start on galera-bundle-0
* Resource action: galera start on galera-bundle-2
* Pseudo action: galera-bundle-master_running_0
* Resource action: galera-bundle-0 monitor=30000 on controller-0
* Resource action: galera-bundle-2 monitor=30000 on controller-1
* Pseudo action: redis-bundle-master_pre_notify_promote_0
* Pseudo action: redis-bundle_promote_0
* Pseudo action: galera-bundle_running_0
* Resource action: redis notify on redis-bundle-0
* Resource action: redis notify on redis-bundle-2
* Resource action: redis notify on redis-bundle-1
* Pseudo action: redis-bundle-master_confirmed-pre_notify_promote_0
* Pseudo action: redis-bundle-master_promote_0
* Pseudo action: galera-bundle_promote_0
* Pseudo action: galera-bundle-master_promote_0
* Resource action: redis promote on redis-bundle-0
* Pseudo action: redis-bundle-master_promoted_0
* Resource action: galera promote on galera-bundle-0
* Resource action: galera promote on galera-bundle-2
* Pseudo action: galera-bundle-master_promoted_0
* Pseudo action: redis-bundle-master_post_notify_promoted_0
* Pseudo action: galera-bundle_promoted_0
* Resource action: galera monitor=10000 on galera-bundle-0
* Resource action: galera monitor=10000 on galera-bundle-2
* Resource action: redis notify on redis-bundle-0
* Resource action: redis notify on redis-bundle-2
* Resource action: redis notify on redis-bundle-1
* Pseudo action: redis-bundle-master_confirmed-post_notify_promoted_0
* Pseudo action: redis-bundle_promoted_0
* Resource action: redis monitor=20000 on redis-bundle-0
* Resource action: redis monitor=60000 on redis-bundle-1
* Resource action: redis monitor=45000 on redis-bundle-1
Revised Cluster Status:
* Node List:
* Online: [ controller-0 controller-1 controller-2 ]
* RemoteOnline: [ database-0 database-1 database-2 messaging-0 messaging-1 messaging-2 ]
* GuestOnline: [ galera-bundle-0 galera-bundle-1 galera-bundle-2 rabbitmq-bundle-0 rabbitmq-bundle-1 rabbitmq-bundle-2 redis-bundle-0 redis-bundle-1 redis-bundle-2 ]
* Full List of Resources:
* database-0 (ocf:pacemaker:remote): Started controller-0
* database-1 (ocf:pacemaker:remote): Started controller-2
* database-2 (ocf:pacemaker:remote): Started controller-1
* messaging-0 (ocf:pacemaker:remote): Started controller-2
* messaging-1 (ocf:pacemaker:remote): Started controller-2
* messaging-2 (ocf:pacemaker:remote): Started controller-2
* Container bundle set: rabbitmq-bundle [192.168.24.1:8787/rhosp12/openstack-rabbitmq-docker:pcmklatest]:
* rabbitmq-bundle-0 (ocf:heartbeat:rabbitmq-cluster): Started messaging-0
* rabbitmq-bundle-1 (ocf:heartbeat:rabbitmq-cluster): Started messaging-1
* rabbitmq-bundle-2 (ocf:heartbeat:rabbitmq-cluster): Started messaging-2
* Container bundle set: galera-bundle [192.168.24.1:8787/rhosp12/openstack-mariadb-docker:pcmklatest]:
* galera-bundle-0 (ocf:heartbeat:galera): Promoted database-0
* galera-bundle-1 (ocf:heartbeat:galera): Promoted database-1
* galera-bundle-2 (ocf:heartbeat:galera): Promoted database-2
* Container bundle set: redis-bundle [192.168.24.1:8787/rhosp12/openstack-redis-docker:pcmklatest]:
* redis-bundle-0 (ocf:heartbeat:redis): Promoted controller-0
* redis-bundle-1 (ocf:heartbeat:redis): Unpromoted controller-1
* redis-bundle-2 (ocf:heartbeat:redis): Unpromoted controller-2
* ip-192.168.24.11 (ocf:heartbeat:IPaddr2): Started controller-0
* ip-10.0.0.104 (ocf:heartbeat:IPaddr2): Started controller-1
* ip-172.17.1.19 (ocf:heartbeat:IPaddr2): Started controller-2
* ip-172.17.1.11 (ocf:heartbeat:IPaddr2): Started controller-0
* ip-172.17.3.13 (ocf:heartbeat:IPaddr2): Started controller-1
* ip-172.17.4.19 (ocf:heartbeat:IPaddr2): Started controller-2
* Container bundle set: haproxy-bundle [192.168.24.1:8787/rhosp12/openstack-haproxy-docker:pcmklatest]:
* haproxy-bundle-docker-0 (ocf:heartbeat:docker): Started controller-0
* haproxy-bundle-docker-1 (ocf:heartbeat:docker): Started controller-1
* haproxy-bundle-docker-2 (ocf:heartbeat:docker): Started controller-2
* openstack-cinder-volume (systemd:openstack-cinder-volume): Started controller-0
* stonith-fence_ipmilan-525400244e09 (stonith:fence_ipmilan): Started controller-2
* stonith-fence_ipmilan-525400cdec10 (stonith:fence_ipmilan): Started controller-2
* stonith-fence_ipmilan-525400c709f7 (stonith:fence_ipmilan): Started controller-1
* stonith-fence_ipmilan-525400a7f9e0 (stonith:fence_ipmilan): Started controller-0
* stonith-fence_ipmilan-525400a25787 (stonith:fence_ipmilan): Started controller-2
* stonith-fence_ipmilan-5254005ea387 (stonith:fence_ipmilan): Started controller-1
* stonith-fence_ipmilan-525400542c06 (stonith:fence_ipmilan): Started controller-0
* stonith-fence_ipmilan-525400aac413 (stonith:fence_ipmilan): Started controller-2
* stonith-fence_ipmilan-525400498d34 (stonith:fence_ipmilan): Started controller-1
|
