- Cloud Environment -

How to properly use AWS instance storage (ephemeral storage/disk)

  • If SIOS AMIs are not being deployed on an AWS server, these disk initialization steps must be considered. This was introduced with Windows 2016 and continues in later releases.
    • AWS ephemeral storage (Instance Store) must be initialized every time a node starts up from a “cold” shutdown (i.e. having been shut down in the AWS Console or from a CLI “shutdown /s” from Windows – not a reboot).
    • Windows 2012 R2 performs this step automatically (Ec2Config service). AWS includes scripts (EC2Launch) to perform this initialization but the script has to be manually scheduled by the user so that it runs at boot time.
    • The SIOS AMIs on AWS have this task set up automatically. If you launch your own Windows image in AWS and fail to set the script up to run at boot, and you are using ephemeral storage for bitmaps (as recommended in Best Practices), the bitmap file/volume will be missing and DataKeeper mirrors will not recreate.

SOLUTION
  • Launch Disk Management and make sure the ephemeral storage is on-line, healthy, and is assigned the volume letter you selected for the ephemeral storage.
  • Ensure that the drive/drive letter assigned to the ephemeral storage is listed in the registry located at HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\ExtMirr\Parameters . . . BitmapBaseDir

Note: Apply this workaround to all nodes in the cluster.
LONG TERM SOLUTION
Schedule the EC2Launch script to run at boot time on each system in the cluster to ensure that the ephemeral storage is always available.

- Administrative Processes -

How to change the hostname of an existing license

  • From the License Support menu select List License

  • Select the checkbox for the license you want to change (you can only do one license at a time)

  • Click the Action dropdown and select Repair

  • Enter the new hostname

  • Select Repair then select Complete
How to determine what GPOs have been applied to a server

ISSUE:

Group Policy Objects are sometimes deployed or redeployed after a server is rebooted impacting the way the LifeKeeper (GUI) or DataKeeper may or may not perform.



Solution:



There are two methods:



From Start\Run enter rsop.msc
  • The console will provide a list of policies that have been applied to a server
Or from an elevated command prompt, enter gpresult /Scope Computer /v
  • This output will display a list of policies that have been applied to a server

    Note: To determine what GPOs are being enforced per a user, enter gpresult /Scope user /v


- LifeKeeper GUI -

How to deploy OpenJDK on LifeKeeper

  • Download OpenJDK, the Windows x64 Installer, from https://openjdk.java.net/

  • Deploy the software on the Target node first

  • In LifeKeeper, perform a switchover/In-Service

  • Repeat the steps above on other Target(s) after the switchovers are completed



Note: This is a Highly Available deployment as you are never installing the software on the Source/Active node.

From a Command Prompt

type javac -version

Your output should be as follows:

“C:\Windows\system32>javac -version javac 14.0.1”

If not, you may need to add the following to your Systems Variable PATH/environment:

  • Control Panel\System and Security\System\System Properties\
  • Select the Advance Tab
  • Select Startup and Recovery
  • Select Environment Variables…
  • Select Path and then Edit

    Add C:\Program Files\Java\jdk-14.0.1\bin to your string to reflect the following in the example below:

    C:\Program Files (x86)\Common
    Files\Oracle\Java\javapath;SystemRoot%\system32;%SystemRoot;%SystemRoot%\System32\Wbem;%SYSTEMROOT%\System32\WindowsPowerShell\v1.0\;C:\Program
    Files\Microsoft SQL Server\Client SDK\ODBC\110\Tools\Binn\;C:\Program Files (x86)\Microsoft SQL Server\120\Tools\Binn\;C:\Program Files\Microsoft SQL Server\120\Tools\Binn\;C:\Program Files\Microsoft SQL Server\120\DTS\Binn\;C:\Program Files (x86)\Microsoft SQL Server\120\Tools\Binn\ManagementStudio\;C:\Program
    Files (x86)\Microsoft SQL Server\120\DTS\Binn\;C:\Program Files\Java\jdk-14.0.1\bin

    Perform the aforementioned on all nodes in your cluster(s).

- Mirror Solutions -

How to recreate a mirror for an existing clustered DataKeeper Resource

ISSUE:

A customer may have experienced some issues where a DataKeeper Resource is listed as Offline via Windows Server Failover Cluster.

Note: An existing job in the DataKeeper GUI will most likely have a RED X status



Solution:


  • Remove all of the old mirror remnants from the Source and Target nodes
  • Manually recreate the mirror via the emcmd . createmirror command



To remove the mirror configuration from both nodes:

  • cd %extmirrbase%/support (This is a shortcut to the DataKeeper\Support directory)
  • run “cleanupmirror (drive letter)Note: Run this command on the Source first. This command performs the following:
    • deletes the local mirror only (NO DATA)
    • resets the switchover flag
    • notifies the driver and service that no mirror is present


To verify that the mirror has been properly removed, execute the following:

  • cd %extmirrbase% (This is a shortcut to the DataKeeper directory)
  • run “emcmd . getmirrorvolinfo (drive letter)
    The output should reflect no mirror e.g. 0 servername 0



Now you are in a position to manually recreate the mirror.

The syntax is as follows:


  • emcmd (Source IP) createmirror (drive letter) (Target IP) A or S (Async mirror/Sync Mirror)
    Upon completion, the DataKeeper UI will now reflect a Resyncing status. This can also can be verified from a command line by executing:
  • emcmd . getmirrorvolinfo e
    The output will reflect “E: 1 servername (Target IP) 2, where 2 is a mirror definition, indicating a Resyncing status

    Return to Windows Failover Clustering and online the Failed DataKeeper Resource
Deleting mirrorendpoints

Steps to delete mirrorendpoints:


• Go in the registry for the node
• Go to HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\ExtMirr\Parameters\Jobs
• Find the job for that endpoint and remove the endpoint from that job
When executing a “emcmd . createmirror”, you receive a Status = 317

ISSUE:

Reasons for a Status = 317

  • The Target volume size (in bytes) is smaller than the Source when attempting to use the emcmd . createmirror command (See properties of the Volumes).
  • When attempting to create a mirror using “emcmd . createmirror “ via 2×1, shared buddy configuration, you may receive a Status = 317.


SOLUTION:

There are two methods:

  1. Ensure that the potential Target location is available or eligible for mirroring (make sure that the volume is seen in
    • Windows Disk Management (Properties of the disk) – Source/Target SAME size in bytes or the Target is LARGER
    • File Manager or Disk Properties, in Disk Management) – Source/Target SAME size in bytes or the Target is LARGER.
  1. For a shared buddy
    • On the offending target(s), execute the following command; emcmd . clearblocktarget {drive letter}.
    • Then verify that target(s) are set to FALSE by running emcmd . getblocktarget {drive letter}

Note: The bitmask for a “shared buddy” should be set to 384.

  • This can be verified by executing “emcmd . getconfiguration {drive letter}”.
  • If not, execute a “emcmd . setconfiguration 384” and execute “emcmd . getconfiguration {drive letter}” to ensure it has been applied.

Additional Assistance: Rebuilding a Mirror

If there is ever a need to rebuild a mirror while maintaining the DataKeeper Job and DataKeeper Volume Resource in the cluster/WSFC/Failover Cluster Manager the steps are as follows:

  1. Always begin from the Source:
    1. Launch an elevated Administrator command prompt
    2. “cd %extmirrbase%” (This is a shortcut to the \DataKeeper directory)

  2. Next we’ll need to identify the status of the mirror from this node/Source:
    1. Type “emcmd . getmirrorvolinfo (drive letter)” (see the link below for the output this command produces)
    2. You can change the “.” to the destination servers, IP, Hostname, FQDN to determine status

  3. Now you need to remove the old mirror remnants:
    1. Execute “cd Support”
    2. The next command will delete the local mirror only (NO DATA), removes a switchover flag and notifies the SIOS DataKeeper Service & Driver that “No Mirror” is no longer present
    3. “cleanupmirror (drive letter)”
    4. “cd ..” (return to the DataKeeper directory)
    5. “emcmd . getmirrorvolinfo (drive letter)” The output should reflect “0 servername 0”
    6. Repeat the aforementioned commands on the Target(s)

  4. Now you are in a position to manually recreate the mirror(s):
    1. Launch Failover Cluster Manager to determine what server is the Current Owner of all resources.
      1. This will indicated the node that will be the Source when creating the mirror from the command prompt
  1. From the Source:
    1. Execute “emcmd . getjobinfo”
    2. Use the output to verify the IP addresses, drive letter(s) and Mirror Type(s) Async (A)/Sync(S) for recreating the mirror


The syntax for creating or recreating a mirror is as follows:

EMCmd <Source IP> CREATEMIRROR <volume letter> <Target IP> <Type> [options]

Example:
# emcmd 10.x.x.x createmirror D 10.x.x.x A

The completed output should say “Status = 0”.
  1. Return to the DataKeeper UI as you will view the mirror you’ve just created.
    1. Any data at this point that needs to be Resynced to the Target will indicate “Resyncing”


Once all Mirror(s) are in a “Mirroring” state, your WSFC roles a eligible for a Move/Switchover.

Commands to Consider:

Click here for access the list of all the emcmd commands

The most frequently use commands are the emcmd . get… commands:

For Mirror Status:
emcmd . getmirrorvolinfo
emcmd . getserviceinfo
emcmd . getjobinfo
emcmd . getcompletevolumelist

Feedback

Was this helpful?

Yes No
You indicated this topic was not helpful to you ...
Could you please leave a comment telling us why? Thank you!
Thanks for your feedback.

Post your comment on this topic.

Post Comment