You are here: User Guide > Using LifeKeeper Linux > Technical Notes

Technical Notes

We strongly recommend that you read the following technical notes concerning configuration and operational issues related to your LifeKeeper environment.

LifeKeeper Features

Item Description

Licensing

LifeKeeper requires unique runtime license keys for each server.  This applies to both physical and virtual servers.  A license key is required for the LifeKeeper core software, as well as for each separately packaged LifeKeeper recovery kit.  The installation support script installs a Licensing Utilities package that obtains and displays the Host ID of your server. The Host IDs, along with the Activation ID(s) provided with your software, are used to obtain license keys from the SIOS Technology Corp. website.

Large Cluster Support

LifeKeeper supports large cluster configurations, up to 32 servers. There are many factors other than LifeKeeper, however, that can affect the number of servers supported in a cluster.  This includes items such as the storage interconnect and operating system or storage software limitations.  Refer to the vendor-specific hardware and software configuration information to determine the maximum supported cluster size.

International-ization and localization

LifeKeeper for Linux v5.2 and later does support wide/multi-byte characters in resource and tag names but does not include native language message support. The LifeKeeper GUI can be localized by creating locale-specific versions of the Java property files, although currently only the English version is fully localized. However, many of the messages displayed by the LifeKeeper GUI come from the LifeKeeper core, so localization of the GUI will provide only a partial solution for users until the core software is fully localized.

See also lklin00002100 under Restrictions or Known Issues for additional information.

LifeKeeper MIB File

LifeKeeper can be configured to issue SNMP traps describing the events that are occurring within the LifeKeeper cluster.  See the lk_configsnmp(8) man page for more information about configuring this capability.  The MIB file describing the LifeKeeper traps can be found at /opt/LifeKeeper/include/LifeKeeper-MIB.txt.

Watchdog

LifeKeeper supports the watchdog feature.  The feature was tested by SIOS Technology Corp. on Red Hat EL 5.5 64-bit, Red Hat EL 5.6 32-bit and Red Hat EL 6 + softdog.

STONITH

LifeKeeper supports the STONITH feature.  This feature was tested by SIOS Technology Corp. on SLES 11 on IBM x3550 x86_64 architecture and  RHEL5.5 64-bit. 

XFS File System

The XFS file system does not use the fsck utility to check and fix a file system but instead relies on mount to replay the log. If there is a concern that there may be a consistency problem, the system administrator should unmount the file system by taking it out of service and run xfs_check(8) and xfs_repair(8) to resolve any issues.

IPv6

SIOS has migrated to the use of the ip command and away from the ifconfig command (for more information, see the IPv6 Known Issue).

Tuning

Item Description

IPC Semaphores and IPC Shared Memory

LifeKeeper requires Inter-Process Communication (IPC) semaphores and IPC shared memory.  The default Red Hat values for the following Linux kernel options are located in/usr/src/linux/include/linux/sem.h and should be sufficient to support most LifeKeeper configurations.

Option       Required    Default Red Hat 6.2
SEMOPM        14                     32
SEMUME        20                     32
SEMMNU        60                     32000
SEMMAP        25                     32000
SEMMNI         25                     128

System File Table

LifeKeeper requires that system resources be available in order to failover successfully to a backup system.  For example, if the system file table is full, LifeKeeper may be unable to start new processes and perform a recovery.  In kernels with enterprise patches, including those supported by LifeKeeper, file-max, the maximum number of open files in the system, is configured by default to 1/10 of the system memory size, which should be sufficient to support most LifeKeeper configurations.  Configuring the file-max value lower than the default could result in unexpected LifeKeeper failures.

The value of file-max may be obtained using the following command:

 cat /proc/sys/fs/file-nr

This will return three numbers.  The first represents the high watermark of file table entries (i.e. the maximum the system has seen so far); the second represents the current number of file table entries, and the third is the file-max value.

To adjust file-max, add (or alter) the “fs,file-max” value in /etc/sysctl.conf (see sysctl.conf(5) for the format) and then run

sysctl –p

to update the system.  The value in /etc/sysctl.conf will persist across reboots.

LifeKeeper Operations

Item Description

Kernel Debugger (kdb)

init s

Before using the Kernel Debugger (kdb) or moving to init s on a LifeKeeper protected server, you should first either shut down LifeKeeper on that server or switch any LifeKeeper protected resources over to the backup server.  Use of kdb with the LifeKeeper SCSI Reservation Daemons (lkscsid and lkccissd) enabled (they are enabled by default) can also lead to unexpected panics.

System Panic on Locked Shared Devices

LifeKeeper uses a lock to protect shared data from being accessed by other servers on a shared SCSI Bus.  If LifeKeeper cannot access a device as a result of another server taking the lock on a device, then a critical error has occurred and quick action should be taken or data can be corrupted.  When this condition is detected, LifeKeeper enables a feature that will cause the system to panic. 

If LifeKeeper is stopped by some means other than ‘lkstop,’ with shared devices still reserved (this could be caused by executing kdb or init s), then the LifeKeeper locking mechanism may trigger a kernel panic when the other server recovers the resource(s).  All resources must be placed out-of-service before stopping LifeKeeper in this manner.

Recovering Out-of-Service Hierarchies

As part of the recovery following the failure of a LifeKeeper server, resource hierarchies that were configured on the failed server but which were not in-service anywhere at the time of the server failure are recovered on the highest priority alive server at the time of the failure. This is the case no matter where the out-of-service hierarchy was last in service, including the failed server, the recovering server, or some other server in the cluster.

Coexistence with Linux Firewalls and SELinux

The firewall and SELinux are enabled upon installation. After installation is complete, SELinux should be disabled and the firewall should be modified.  

LifeKeeper will not install or function if the SELinux mode is "enabled" or "permissive." To disable SELinux in RedHat, run the system-config-securitylevel-tui tool from the console of the host system. While SELinux for SLES 11 SP1 is available, it too must be disabled (http://www.novell.com/linux/releasen.../SUSE-SLES/11/).

AppArmor (for distributions that use this security model) may be enabled.

LifeKeeper will function if a host firewall is enabled. However, unless absolutely necessary, it is recommended that the firewall be disabled and that the LifeKeeper protected resources reside behind another shielding firewall.

If LifeKeeper must coexist on firewall enabled hosts, note that LifeKeeper uses specific ports for communication paths, GUI, IP and Data Replication. When using the Linux firewall feature, the specific ports that LifeKeeper is using need to be opened.

To disable or modify the RedHat firewall, run the system-config-securitylevel-tui tool from the console of the host system. To disable or modify the SUSE firewall, run yast2 and choose Security and Users, then Firewall. Refer to Running LifeKeeper with a Firewall for details. 

Suid Mount Option

The suid mount option is the default when mounting as root and is not written to the /etc/mtab by the mount command. The suid mount option is not needed in LifeKeeper environments.

Server Configuration

Item Description

BIOS Updates

The latest available BIOS should always be installed on all LifeKeeper servers.

Package Dependencies List for LifeKeeper 7.5 and Later

The following is a list of dependencies that may be necessary for the required packages in LifeKeeper 7.5 depending upon your OS distribution.

IMPORTANT: The 32-bit versions of these packages are required.

Please note that there may be additional packages that must be installed to satisfy the dependencies of this list.

bzip2 OR libbz2 OR bzip2-lib
glibc
iproute OR iproute2
iptables
iputils
libstdc++ OR libstdc++43
mktemp
nfs-utils OR nfs-kernel-server (if protecting NFS shares)
pam
zlib

Note: The ORs are the Linux OS distribution variances.

Please note that this list is not all inclusive. Depending on the base packages and Linux OS distribution, additional package dependencies may be required. Also, if the configure script detects that certain optional software components are installed, additional package dependencies may be necessary. 

You may want to consider using a repository-based package manager such as yum or zypper that is designed to automatically resolve dependencies by searching in predefined software repositories, thereby easing the installation of these required packages. 

Confirm Failover and Block Resource Failover Settings

Make sure you review and understand the following descriptions, examples and considerations before setting the Confirm Failover or Block Resource Failover in your LifeKeeper environment. These settings are available from the command line or on the Properties panel in the LifeKeeper GUI.

Confirm Failover On:

Definition – Enables manual failover confirmation from System A to System B (where System A is the server whose properties are being displayed in the Properties Panel and System B is the system to the left of the checkbox). If this option is set on a system, it will require a manual confirmation by a system administrator before allowing LifeKeeper to perform a failover recovery of a system that it detects as failed.  

Use the lk_confirmso command to confirm the failover. By default, the administrator has 10 minutes to run this command. This time can be changed by modifying the CONFIRMSOTO setting in /etc/default/LifeKeeper. If the administrator does not run the lk_confirmso command within the time allowed, the failover will either proceed or be blocked. By default, the failover will proceed. This behavior can be changed by modifying the COMFIRMSODEF setting in /etc/default/LifeKeeper.

Example: If you wish to block automatic failovers completely, then you should set the Confirm Failover On option in the Properties panel and also set CONFIRMSODEF to 1 (block failover) and CONFIRMSOTO to 0 (do not wait to decide on the failover action).

When to select this setting:

This setting is used in most Disaster Recovery and other WAN configurations where the configuration does not include redundant heartbeat communications paths.

In a regular site (non multi-site cluster), open the Properties page from one server and then select the server that you want the Confirm Failover flag to be set on.

For a Multi-site WAN configuration: Enable manual failover confirmation

For a Multi-site LAN configuration: Do not enable manual failover confirmation

In a multi-site cluster environment – from the non-disaster system, select the DR system and check the set confirm failover flag. You will need to open the Properties panel and select this setting for each non-disaster server in the cluster.

Set Block Resource Failover On:

Definition - By default, all resource failures will result in a recover event that will attempt to recover the failed resource on the local system. If local recovery fails or is not enabled, then LifeKeeper transfers the resource hierarchy to the next highest priority system for which the resource is defined. However, if this setting is selected on a designated system(s), all resource transfers due to a resource failure will be blocked from the given system.

When the setting is enabled, the following message is logged:

Local recovery failure, failover blocked, MANUAL INTERVENTION REQUIRED

Conditions/Considerations:

In a Multi-site configuration, do not select Block Failover for any server in the configuration.

Remember: This setting will not affect failover behavior if there is a complete system failure. It will only block failovers due to resource failures.

NFS Client Options

When setting up a LifeKeeper protected NFS server, how the NFS clients connect to that server can make a significant impact on the speed of reconnection on failover. 

NFS Client Mounting Considerations

An NFS Server provides a network-based storage system to client computers. To utilize this resource, the client systems must “mount” the file systems that have been NFS exported by the NFS server. There are several options that system administrators must consider on how NFS clients are connected to the LifeKeeper protected NFS resources.

UDP or TCP?

The NFS Protocol can utilize either the User Datagram Protocol (UDP) or the Transmission Control Protocol (TCP). NFS has historically used the UDP protocol for client-server communication. One reason for this is that it is easier for NFS to work in a stateless fashion using the UDP protocol. This “statelessness” is valuable in a high availability clustering environment, as it permits easy reconnection of clients if the protected NFS server resource is switched between cluster hosts. In general, when working with a LifeKeeper protected NFS resource, the UDP protocol tends to work better than TCP.

Sync Option in /etc/exports

Specifying “sync” as an export option is recommended for LifeKeeper protected NFS resources. The “sync” option tells NFS to commit writes to the disk before sending an acknowledgment back to the NFS client. The contrasting “async” option is also available, but using this option can lead to data corruption, as the NFS server will acknowledge NFS writes to the client before committing them to disk. NFS clients can also specify “sync” as an option when they mount the NFS file system.

Red Hat EL6 (and Fedora 14) Clients with Red Hat EL6 NFS Server

Due to what appears to be a bug in the NFS server for Red Hat EL6, NFS clients running Red Hat EL6 (and Fedora 14) cannot specify both an NFS version (nfsvers) and UDP in the mount command. This same behavior has been observed on an Ubuntu10.10 client as well. This behavior is not seen with Red Hat EL5 clients when using a Red Hat EL6 NFS server, and it is also not seen with any clients using a Red Hat EL5 NFS server. The best combination of NFS mount directives to use with Red Hat EL6 (Fedora 14) clients and a Red Hat EL 6 NFS server is:

mount <protected-IP>:<export> <mount point>  
-o nfsvers=2,sync,hard,intr,timeo=1

  • This combination produces the fastest re-connection times for the client in case of a switchover or failover of the LifeKeeper protected NFS server.

Red Hat EL5 NFS Clients with a Red Hat EL6 NFS Server

The best combination of options when using NFS clients running Red Hat EL5 with a Red Hat EL6 NFS server for fast reconnection times is:

mount <protected-IP>:<export> <mount point>  
-o nfsvers=3,sync,hard,intr,timeo=1,udp
 

© 2012 SIOS Technology Corp., the industry's leading provider of business continuity solutions, data replication for continuous data protection.