STONITH

STONITH (Shoot The Other Node in the Head) is a fencing technique for remotely powering down a node in a cluster. LifeKeeper can provide STONITH capabilities by using external power switch controls, IPMI-enabled motherboard controls and hypervisor-provided power capabilities to power off the other nodes in a cluster.

Using IPMI with STONITH

IPMI (Intelligent Platform Management Interface) defines a set of common interfaces to a computer system which can be used to monitor system health and manage the system. Used with STONITH, it allows the cluster software to instruct the switch via a serial or network connection to power off or reboot a cluster node that appears to have died thus ensuring that the unhealthy node cannot access or corrupt any shared data.

Package Requirements

• IPMI tools package (e.g. ipmitool-1.8.11-6.el6.x86_64.rpm)

STONITH in VMware vSphere Environments

vCLI (vSphere Command-Line Interface) is a command-line interface supported by VMware for managing your virtual infrastructure including the ESXi hosts and virtual machines. You can choose the vCLI command best suited for your needs and apply it for your LifeKeeper STONITH usage between VMware virtual machines.

Package Requirements

STONITH Server

• • VMware vSphere SDK Package or VMware vSphere CLI. (vSphere CLI is included in the same installation package as the vSphere SDK).
    

Monitored Virtual Machine

• • VMware Tools
    

Installation and Configuration

After installing LifeKeeper and configuring communication paths for each node in the cluster, install and configure STONITH.

1. Install the LifeKeeper STONITH script by running the following command:

   
   /opt/LifeKeeper/samples/STONITH/stonith-install

2. (*For IPMI usage only) Using BIOS or the ipmitool command, set the following BMC (Baseboard Management Controller) variables:

• Use Static IP

• IP address

• Sub netmask

• User name

• Password

• Add Administrator privilege level to the user

• Enable network access to the user

Example using ipmitool command

(For detailed information, see the ipmitool man page.)

3. Edit the configuration file.

Update the configuration file to enable STONITH and add the power off command line.

*Note: Power off is recommended over reboot to avoid fence loops (i.e. two machines have lost communication but can still STONITH each other, taking turns powering each other off and rebooting).

/opt/LifeKeeper/config/stonith.conf

<vm_id>

vSphere CLI commands run on top of vSphere SDK for Perl. is used as an identifier of the VM. This variable should point to the VM’s configuration file for the VM being configured.

To find the configuration file path:

1. Type the following command:
   

   vmware-cmd -H <vmware host> -l

2. This will return a list of VMware hosts.
   

   Example output from vmware-cmd -l with three vms listed:
   

   /vmfs/volumes/4e08c1b9-d741c09c-1d3e-0019b9cb28be/lampserver/lampserver.vmx
   

   /vmfs/volumes/4e1e1386-0b862fae-a859-0019b9cb28bc/oracle10/oracle.vmx
   

   /vmfs/volumes/4e08c1b9-d741c09c-1d3e-0019b9cb28be/lampserver02/lampserver02.vmx

Find the VM being configured in the resulting list.

3. Paste the path name into the variable. The example above would then become:
   

   vmware-cmd -H 10.0.0.1 -U root -P secret /vmfs/volumes/4e08c1b9-d741c09c-1d3e-0019b9cb28be/lampserver/lampserver.vmx stop hard

*Note: For further information on VMware commands, use vmware-cmd with no arguments to display a help page about all options.

Expected Behaviors

When LifeKeeper detects a communication failure with a node, that node will be powered off and a failover will occur. Once the issue is repaired, the node will have to be manually powered on.