blob: edaed2246d7ca93bbee463c617db91e9b5946640 (
plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
|
Using the original execution date of: 2038-02-17 06:13:20Z
Current cluster status:
* Node List:
* RemoteNode overcloud-novacompute-1: UNCLEAN (offline)
* Online: [ controller-0 controller-1 controller-2 ]
* RemoteOnline: [ overcloud-novacompute-0 ]
* GuestOnline: [ galera-bundle-0 galera-bundle-1 galera-bundle-2 rabbitmq-bundle-0 rabbitmq-bundle-1 rabbitmq-bundle-2 redis-bundle-0 redis-bundle-1 redis-bundle-2 ]
* Full List of Resources:
* overcloud-novacompute-0 (ocf:pacemaker:remote): Started controller-0
* overcloud-novacompute-1 (ocf:pacemaker:remote): FAILED controller-1
* Container bundle set: rabbitmq-bundle [192.168.24.1:8787/rhosp13/openstack-rabbitmq:pcmklatest]:
* rabbitmq-bundle-0 (ocf:heartbeat:rabbitmq-cluster): Started controller-2
* rabbitmq-bundle-1 (ocf:heartbeat:rabbitmq-cluster): Started controller-0
* rabbitmq-bundle-2 (ocf:heartbeat:rabbitmq-cluster): Started controller-1
* Container bundle set: galera-bundle [192.168.24.1:8787/rhosp13/openstack-mariadb:pcmklatest]:
* galera-bundle-0 (ocf:heartbeat:galera): Promoted controller-2
* galera-bundle-1 (ocf:heartbeat:galera): Promoted controller-0
* galera-bundle-2 (ocf:heartbeat:galera): Promoted controller-1
* Container bundle set: redis-bundle [192.168.24.1:8787/rhosp13/openstack-redis:pcmklatest]:
* redis-bundle-0 (ocf:heartbeat:redis): Promoted controller-2
* redis-bundle-1 (ocf:heartbeat:redis): Unpromoted controller-0
* redis-bundle-2 (ocf:heartbeat:redis): Unpromoted controller-1
* ip-192.168.24.11 (ocf:heartbeat:IPaddr2): Started controller-2
* ip-10.0.0.110 (ocf:heartbeat:IPaddr2): Stopped
* ip-172.17.1.14 (ocf:heartbeat:IPaddr2): Started controller-1
* ip-172.17.1.17 (ocf:heartbeat:IPaddr2): Started controller-2
* ip-172.17.3.11 (ocf:heartbeat:IPaddr2): Started controller-0
* ip-172.17.4.17 (ocf:heartbeat:IPaddr2): Started controller-1
* Container bundle set: haproxy-bundle [192.168.24.1:8787/rhosp13/openstack-haproxy:pcmklatest]:
* haproxy-bundle-docker-0 (ocf:heartbeat:docker): Started controller-2
* haproxy-bundle-docker-1 (ocf:heartbeat:docker): Started controller-0
* haproxy-bundle-docker-2 (ocf:heartbeat:docker): Started controller-1
* stonith-fence_compute-fence-nova (stonith:fence_compute): FAILED controller-2
* Clone Set: compute-unfence-trigger-clone [compute-unfence-trigger]:
* compute-unfence-trigger (ocf:pacemaker:Dummy): Started overcloud-novacompute-1 (UNCLEAN)
* Started: [ overcloud-novacompute-0 ]
* Stopped: [ controller-0 controller-1 controller-2 ]
* nova-evacuate (ocf:openstack:NovaEvacuate): Started controller-0
* stonith-fence_ipmilan-5254008be2cc (stonith:fence_ipmilan): Started controller-1
* stonith-fence_ipmilan-525400803f9e (stonith:fence_ipmilan): Started controller-0
* stonith-fence_ipmilan-525400fca120 (stonith:fence_ipmilan): Started controller-2
* stonith-fence_ipmilan-525400953d48 (stonith:fence_ipmilan): Started controller-2
* stonith-fence_ipmilan-525400b02b86 (stonith:fence_ipmilan): Started controller-1
* Container bundle: openstack-cinder-volume [192.168.24.1:8787/rhosp13/openstack-cinder-volume:pcmklatest]:
* openstack-cinder-volume-docker-0 (ocf:heartbeat:docker): Started controller-0
Transition Summary:
* Fence (reboot) overcloud-novacompute-1 'remote connection is unrecoverable'
* Stop overcloud-novacompute-1 ( controller-1 ) due to node availability
* Start ip-10.0.0.110 ( controller-1 )
* Recover stonith-fence_compute-fence-nova ( controller-2 )
* Stop compute-unfence-trigger:1 ( overcloud-novacompute-1 ) due to node availability
Executing Cluster Transition:
* Resource action: overcloud-novacompute-1 stop on controller-1
* Resource action: stonith-fence_compute-fence-nova stop on controller-2
* Fencing overcloud-novacompute-1 (reboot)
* Cluster action: clear_failcount for overcloud-novacompute-1 on controller-1
* Resource action: ip-10.0.0.110 start on controller-1
* Resource action: stonith-fence_compute-fence-nova start on controller-2
* Resource action: stonith-fence_compute-fence-nova monitor=60000 on controller-2
* Pseudo action: compute-unfence-trigger-clone_stop_0
* Resource action: ip-10.0.0.110 monitor=10000 on controller-1
* Pseudo action: compute-unfence-trigger_stop_0
* Pseudo action: compute-unfence-trigger-clone_stopped_0
Using the original execution date of: 2038-02-17 06:13:20Z
Revised Cluster Status:
* Node List:
* RemoteNode overcloud-novacompute-1: UNCLEAN (offline)
* Online: [ controller-0 controller-1 controller-2 ]
* RemoteOnline: [ overcloud-novacompute-0 ]
* GuestOnline: [ galera-bundle-0 galera-bundle-1 galera-bundle-2 rabbitmq-bundle-0 rabbitmq-bundle-1 rabbitmq-bundle-2 redis-bundle-0 redis-bundle-1 redis-bundle-2 ]
* Full List of Resources:
* overcloud-novacompute-0 (ocf:pacemaker:remote): Started controller-0
* overcloud-novacompute-1 (ocf:pacemaker:remote): FAILED
* Container bundle set: rabbitmq-bundle [192.168.24.1:8787/rhosp13/openstack-rabbitmq:pcmklatest]:
* rabbitmq-bundle-0 (ocf:heartbeat:rabbitmq-cluster): Started controller-2
* rabbitmq-bundle-1 (ocf:heartbeat:rabbitmq-cluster): Started controller-0
* rabbitmq-bundle-2 (ocf:heartbeat:rabbitmq-cluster): Started controller-1
* Container bundle set: galera-bundle [192.168.24.1:8787/rhosp13/openstack-mariadb:pcmklatest]:
* galera-bundle-0 (ocf:heartbeat:galera): Promoted controller-2
* galera-bundle-1 (ocf:heartbeat:galera): Promoted controller-0
* galera-bundle-2 (ocf:heartbeat:galera): Promoted controller-1
* Container bundle set: redis-bundle [192.168.24.1:8787/rhosp13/openstack-redis:pcmklatest]:
* redis-bundle-0 (ocf:heartbeat:redis): Promoted controller-2
* redis-bundle-1 (ocf:heartbeat:redis): Unpromoted controller-0
* redis-bundle-2 (ocf:heartbeat:redis): Unpromoted controller-1
* ip-192.168.24.11 (ocf:heartbeat:IPaddr2): Started controller-2
* ip-10.0.0.110 (ocf:heartbeat:IPaddr2): Started controller-1
* ip-172.17.1.14 (ocf:heartbeat:IPaddr2): Started controller-1
* ip-172.17.1.17 (ocf:heartbeat:IPaddr2): Started controller-2
* ip-172.17.3.11 (ocf:heartbeat:IPaddr2): Started controller-0
* ip-172.17.4.17 (ocf:heartbeat:IPaddr2): Started controller-1
* Container bundle set: haproxy-bundle [192.168.24.1:8787/rhosp13/openstack-haproxy:pcmklatest]:
* haproxy-bundle-docker-0 (ocf:heartbeat:docker): Started controller-2
* haproxy-bundle-docker-1 (ocf:heartbeat:docker): Started controller-0
* haproxy-bundle-docker-2 (ocf:heartbeat:docker): Started controller-1
* stonith-fence_compute-fence-nova (stonith:fence_compute): Started controller-2
* Clone Set: compute-unfence-trigger-clone [compute-unfence-trigger]:
* Started: [ overcloud-novacompute-0 ]
* Stopped: [ controller-0 controller-1 controller-2 overcloud-novacompute-1 ]
* nova-evacuate (ocf:openstack:NovaEvacuate): Started controller-0
* stonith-fence_ipmilan-5254008be2cc (stonith:fence_ipmilan): Started controller-1
* stonith-fence_ipmilan-525400803f9e (stonith:fence_ipmilan): Started controller-0
* stonith-fence_ipmilan-525400fca120 (stonith:fence_ipmilan): Started controller-2
* stonith-fence_ipmilan-525400953d48 (stonith:fence_ipmilan): Started controller-2
* stonith-fence_ipmilan-525400b02b86 (stonith:fence_ipmilan): Started controller-1
* Container bundle: openstack-cinder-volume [192.168.24.1:8787/rhosp13/openstack-cinder-volume:pcmklatest]:
* openstack-cinder-volume-docker-0 (ocf:heartbeat:docker): Started controller-0
|
