Introduction
The following is a detailed, step-by-step guide on how to cluster SAP ASCS and ERS running on Windows in the AWS cloud. The examples assume that there are two nodes in the cluster (SIOS1, SIOS2) and each node resides in a different availability zone in the same AWS region. When following the examples please be sure to substitute the computer names, IP address and other variables with those that are applicable to your environment.
The first few sections are covered in greater detail in this blog article, written by Microsoft Cloud MVP, David Bermingham. It is recommended that you review the article prior to proceeding with the steps below.
The guide makes the assumption that you are reasonably familiar with Windows, AWS and SAP.
Step 1: Provision Servers
Pick an Instance Type
It is important to pick an instance that has Instance Store Storage attached, also known as ephemeral disks. We will use this disk to store the DataKeeper bitmap file as described later in the documentation. A list of support instance types can be found here.
Add Primary and Secondary Address on each Instance
Add EBS Volumes to each Instance
When provisioning the server be sure to attach at least one additional EBS volume to each server.
Open Security Groups
In this example, we had previously created a security group that allowed all traffic to flow freely between instances in our VPC (10.0.0.0/16). We also opened up the RDP port to allow remote access directly to each instance. You may want to consult with your cloud administrators to ensure you adhere to your security protocols. Refer to the AWS documentation for additional information on Security Groups.
Step 2: Configure Server Settings on each Cluster Node
- Rename and add it to the Domain
- Turn off the Windows firewall
- Turn off IE Enhanced Security Configuration
- Install Chrome and make it the default browser
- Download the software to each server
- SAP Installer
- DataKeeper Software and License
- Vcredist_x64
- Install vcredist_x64 on each cluster node
- Create DNS A Records for both the ASCS and ERS Virtual Name. In our case we will create the following records:
SAPASCS 10.0.2.152
SAPERS 10.0.2.153
Step 3: Create the Windows Server Failover Cluster
Enable the Failover Clustering Feature
Run the following Powershell Script on each cluster node:
Install-WindowsFeature -Name Failover-Clustering -IncludeManagementTools
Run Cluster Validation
Run the following command on one of the nodes to validate the cluster:
Test-Cluster -Node sios1,sios2
The validation report will show some warnings about a single network interface and no shared storage, you can ignore those warnings as those are expected in a SANless cluster running in AWS.
Create the Cluster
When creating the cluster in AWS it is important to do it via PowerShell rather than the WSFC interface. When we create the cluster we will use two of the secondary IP addresses we added to the interfaces in an earlier step.
Run the following PowerShell command on one of the cluster nodes:
New-Cluster -Name sapcluster -Node sios1,sios2 -StaticAddress 10.0.2.151, 10.0.3.151 -NoStorage
Add the File Share Witness
First, create a folder on another server in a 3rd availability zone. Once you have the folder created, share it and give the Cluster Name Object (CNO). Change permissions at the Share level and Modify permissions at the Security level.
Once the permissions are assigned, run the following PowerShell command to update the cluster quorum to add this file share witness.
Set-ClusterQuorum -FileShareWitness \\dc1\Sapcluster
Step 4: Format the EBS Volumes
Earlier we added additional EBS volumes to each cluster node. The nodes also have an ephemeral drive attached to them. The volumes can be initialized either MBR or GPT.
Initialize the Disk
Format the Volumes
Once they are initialized, format each volume NTFS and assign them drive letters.
Configure EC2launch
The ephemeral drive will be used to hold the DataKeeper bitmap file. It is important to ensure that the ephemeral drive is mounted each time the server starts up. Configure EC2Launch to ensure this process is automated.
Step 5: Adjust Pagefile
Windows in AWS has automatic pagefile memory management enabled by default. We want to disable that and at the same time increase the default size of the pagefile to meet the requirements of SAP. SAP recommends a minimum size of 20480 MB.
Step 6: Install DataKeeper
We are now ready to install DataKeeper. As part of the installation, you will be asked to specify a service account. The only requirement is that the account must be a member of the Local Administrators group on each of the cluster nodes. Create that account and add it to the local Administrators group on each node before you continue with the installation.
Run the DataKeeper setup on each node and accept the default options as shown below.
Step: 7 Relocate the Intent Log (Bitmap file)
For optimal performance, you must relocate the bitmap file to the local ephemeral storage, aka local instance store volume. Please refer to the SIOS documentation for instructions on relocating the bitmap file.
Step: 8 Configure DataKeeper
It is now time to use DataKeeper to replicate the EBS volume(s) between each of the cluster nodes and to register the DataKeeper Volume Resource(s) in the cluster’s Available Storage. The steps below only need to be run on one of the cluster nodes.
Connect to Server
Use the DataKeeper interface to connect to each of the cluster nodes. DataKeeper can be run from either of the cluster nodes.
Once you connect, your Server Overview report should look something like this:
Create DataKeeper Job
You are now ready to replicate your volumes. Click on Create Job and follow the screenshots below.
Register DataKeeper Volume in Available Storage
The last step of the Job Creation process asks if you want to register the volume in Windows Server Failover Clustering (WSFC). Click Yes.
Open the WSFC Manager and you will now see the DataKeeper Volume resource in Available Storage.
Step: 9 SAP High-Availability System Installation
First Cluster Node Installation
Before starting the SAP Installation, make sure you have created a DNS Type A record for SAP Virtual Instance Host on the DNS Server as we described in an earlier step.
You will perform a regular SAP Windows Cluster installation. For more information regarding SAP installation on Microsoft Windows Failover Cluster, please refer to official SAP documentation here.
Current Host Server
Before you begin, make sure the primary server is the Current Host Server.
Grant the Cluster Name Object (CNO) Permissions on the OU
In order to avoid possible security issues with creating the cluster objects in the next steps, it is advisable to grant the CNO permissions on the OU that contain the computer objects. This can be done on one of the domain controllers by a domain admin.
Run SAPINST
In the next steps, we will install the ASCS/ERS cluster on the first cluster node. Follow along with the screenshots below.
Log on as Domain Admin user at the first Windows Cluster instance and run the SAP Installation through sapinst.
Click Next.
The computer will log off at this point. Log back on to continue the installation.
It may take a few moments before the next screen appears.
The SAP System ID is just three letters that identify this system. The Network Name for ASCS and ERS are the same as the names we entered in DNS in an earlier step. The DataKeeper Volume D is the DataKeeper Volume Resource we created earlier.
Specify the folder where you have downloaded the SAP software.
Click Next and it will find your package location.
Click Next to continue.
Notice at this point there are two cluster resource groups with related resources being created in WSFC.
Step 10: Second Cluster Node Installation
The first cluster node installation was completed without too much effort. In a normal cluster where all the cluster nodes reside in the same subnet, the installation of the second cluster node would run just as smoothly. However, because this is AWS and the nodes reside in different subnets, there are some steps needed to address the unique requirements of a multi-subnet cluster.
Update DNS
Before we proceed, we have to fix the A records we created earlier. It was necessary to create those A records so that the first SAP node could be created properly. However, you will see that those A records in DNS are “static”. Static records cannot be updated by WSFC, which is necessary in a multi-subnet cluster.
Delete the A records from DNS so that the WSFC Manager can re-register them as dynamic records. Right-click on each of the two A records we created earlier and delete them.
In the WSFC Manager, bring the two name resources offline in each of the two cluster groups and then bring the resources back online. This process will re-register the two A records in DNS. This time they will be dynamic records.
Refresh your DNS zone and it should now look like this.
Change TTL
By default, the time to live (TTL) on each of those A records is 20 minutes. That is much too long for a client to wait after a failover to receive the new IP address. Instead, we are going to adjust the TTL to 15 seconds.
To adjust the TTL on a cluster name resource, run the following PowerShell command once for each name resource. This command can be run from either cluster node.
Get-ClusterResource -Name “SAP DAB ERS NetName” | Set-ClusterParameter -Name HostRecordTTL -Value 15
Get-ClusterResource -Name “SAP DAB NetName” | Set-ClusterParameter -Name HostRecordTTL -Value 15
Change RegisterAllProvidersIP
A multi-subnet cluster can handle client redirection a few ways. It’s beyond the scope of this article to discuss the differences. To handle SAP client redirection we need to ensure the RegisterAllProvidersIP property of each cluster name resource is set to 0. Run the following PowerShell command on each cluster node.
Get-ClusterResource -Name “SAP DAB ERS NetName” | Set-ClusterParameter RegisterAllProvidersIP 0
Get-ClusterResource -Name “SAP DAB NetName” | Set-ClusterParameter RegisterAllProvidersIP 0
It is important to bring both of the cluster resource offline and online again to ensure that the changes made in the last two sections are applied.
Add IP Address Resource to Support Multi-Subnet Cluster
When configuring clusters for applications like SQL Server, the installer recognizes when the cluster is a multi-subnet cluster. However, the SAP installer does NOT recognize that fact, so we need to perform some of the configuration steps manually. One of those steps is to create the cluster IP address resources that reside in the subnet of the secondary node. The steps are as follows:
- Create the IP Resources
- Assign the IP Address
- Create the “Or” Dependency
Follow the screenshots below to complete these steps on one of the cluster nodes for each of the two cluster resource groups.
Add a new IP address.
Configure the IP address so it is associated with the Network of the secondary node. Give it one of the unused secondary addresses configured earlier.
Make the server name resource dependent on this additional IP address using the “OR” functionality.
Complete the same process for the other server name resource as shown below.
It is normal for these addresses to be offline. They will only be online if the cluster workload is running in that subnet.
Change Cluster Resource Restart Policy
On occasion, the SAP ASCS Service will fail to start upon a switchover or failover. The reason this most often happens is that the service is dependent upon the clustered file share to be available. With the TTL set to 15 seconds, we have observed on occasion that the file share is not available before the ASCS service tries to start. When this occurs it fails to come online. Simply bringing the resource online again after the failure usually fixes the issue. However, that requires user intervention and defeats the purpose of failover clustering.
The fix to this problem is to adjust the Maximum restarts and Delay between restarts properties of the ASCS Service resource to give the ASCS service a little time to come online if the file server resource IP is not yet available. The settings pictured below are some sample settings that should be more than sufficient. If you have a large complex DNS environment, increase these parameters to meet the needs of your environment.
The other parameter that can impact the reconnection is the TTL. We set it to 15 seconds earlier. However, if you have a large AD environment that takes time to update the DNS zones, you may need to decrease the TTL even further and/or allow more restart attempts and increase the delay between restarts.
For good measure, it is recommended to do the same for your SAP Instance Resource.
Adjust Permissions on the USR Folder
Add permissions for the person who is doing the install to the USR folder that was created on the replicated volume. The folder will be on the D drive (or whatever replicated volume you used).
From the secondary node, confirm that you can see the file share that was created using the server name resource.
Run the SAPINST on the Second Node
Follow the screenshots below to complete the installation of the additional cluster nodes.
If the above screen seems to hang here for a while, click on the message and the installation should to start progressing again.
The next step will timeout and fail after 5 minutes.
The process of installing the second moved all the resources to the second node.
To get the installer to complete after the Error and the installer times out, you may need to move both of the cluster resources back to the primary node and click on Retry to complete the installation.
Once both cluster roles are back in service on the primary node, click the Retry button on the SAPinstaller. The installation will then be completed as shown below.
Step 11: Primary Application Server and Additional Application Server Installation
For more information regarding SAP installation on Windows, including the Primary Application Server and Additional Application Server, refer to official SAP documentation here.
After installing the PAS and AAS, you must deactivate the internal cache for hostnames and services names. For more information please refer to SAP note 1425520 – Disable NI cache for host and service names.
Conclusion
In this Step by Step guide, we have shown how to configure SIOS DataKeeper and integrate with Windows Failover Clustering for ABAP SAP Central Services and Enqueue Replication Server.
Post your comment on this topic.