1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
|
We need a fencing agent which can work in a variety of guest cluster
configurations and host configurations.
Requirements
1. Nonrequirement of guest to host networking. Virtual machines
may be configured to run using a nework unknown to the host
operating system. Therefore, the ability to run without network
communication between the guest and the hsot is required.
2. Ease of configuration. The absolute minimum possible configuration
must be available.
3. Nonrequirement of host clustering software. Multiple layers of
configuration sucks. While I fundamentally disagree with the general
idea that running CMAN on the host constitutes a "heavyweight
cluster", perception is important.
4. Ability to support RHEV-M, oVirt server, and other virtual machine
management technologies. This is beneficial from a security standpoint
since it is assumed the management server will be aware of what VMs
are allowed to fence what other VMs.
5. Upgrade compatibility with fence_xvm from a configuration standpoint.
This may be provided by a symlink over fence_xvm. If this feature
can not be provided as a matter of design, a method to convert an
existing fence_xvm/fence_xvmd configuration to fence_virt must be
present.
Guest to Host Interaction
-------------------------
The proposal is to use various communications media plugins in order
to facilitate flexibility with respect to how virtual machine
environments are configured.
There are at least 3 simple plugins for guest/client to host/server
communications:
* Direct serial. The guest sends fencing requests out via /dev/ttySX
in the guest. The host is listening on a Unix domain socket[1],
and forwards fencing requests accordingly.
This satisifies most of the requirements, but adds a conundrum
when configuring guest clusters, as /dev/ttySX may be /dev/ttySY
on another guest. So, either we must account for this per-guest
configuration discrepancy or we must make it an administrative
requirement to provide the same serial device on each host
* Multicast. This violates the networking requirement, but this is
okay since this method of operation is optional. This operational
mode provides for one of the simpler configurations: all that is
needed is the guest's name or UUID. The guest to host
communications operates in the same manner as fence_xvm/fence_xvmd,
except that there is an implied requirement on restricting the
multicast packets accepted to be from the local guests.
* VM Channel over Serial. This works like direct serial, but
instead of owning the whole device, the device may be shared between
multiple applications. The server subscribes to a channel and
listens for fencing requests on the channel; the client in the
guest OS connects to the channel and issues fencing requests across
it. One interesting thing is that it may be possible to provide
unprivileged users the ability to fence using this method (I
do not claim to know if this is useful or not).
Host to Hypervisor interaction
------------------------------
Similar to the way we have plugins for guest to host interaction,
we also have plugins which actually do the real work. These plugins
are responsible for all of the actual real work performed, including
tracking VMs if required, forwarding requests to the appropriate hosts
or management services, and handling the responses.
We propose at 5 plugins in this case:
* Libvirt (local-only). There is no intracommunication and no
migration support is provided
* Cluster CPG (+ libvirt). This the way fence_xvmd
operates today. This setup has the most requirements on the
infrastructure, as it requires guest to host networking _and_
host-to-host clustering in order to keep track of virtual
machines. The benefit is that it is self-contained and requires
no external management nodes. VM states are stored so that other
CPG group members know the locations of other VMs and can make
some decisions about whether a VM is dead based on whether a host
is dead (i.e. if fencing is in use or can be performed on the
host).
* Libvirt-QMF ... ??? Subscription to the appropriate cluster
specific AMQP channel is required on the host side, but this
handles routing the message very easily. The fencing request
is forwarded to the other listeners on the channel, the VM owner
takes the action requested and returns a value. When new VMs
are created, the event is broadcast out via the AMQP channel so
other hosts know the locations of other VMs and can make some
decisions about whether a VM is dead based on whether a host
is dead (i.e. if fencing is in use or can be performed on the
host).
* oVirt Manager. The request is forwarded to the oVirt Manager
and the oVirt manager is responsible for taking the appropriate
action and responding to the request.
* RHEV-M. The request is forwarded to the RHEV-M node, which is
responsible for taking the appropriate action and responding to
the request.
These plugins have no requirements on which guest to host communication
plugin is used (you could, if you wanted, use 'direct serial' with
'cluster cpg', or 'multicast' with 'RHEV-H' for example).
These plugins must also be able to discover where appropriate. For
example, the cpg plugin can only be used if corosync/openais
is running. A defined plugin preference order should be specified/documented
so that the host daemon behaves in a predictable manner in absence of
host-side configuration data (about which plugin to use).
[1] TCP was also explored, however, the security is much better
using a Unix domain socket, despite the additional complexity
of listening for VM creation events.
|
