Essbase Clustering 11.1.2.2

http://database.developer-works.com/article/17609016/Essbase+Active+Passive+Clustering+11.1.2.3 http://hyperionvirtuoso.blogspot.com/2014/01/essbase-clustering-part-1.html https://blogs.oracle.com/pa/entry/epm_11_1_2_23 Essbase v11.1.2.x Cluster Configuration on Unix Systems (Doc ID 1429843.1) To BottomTo Bottom In this Document Purpose Scope Details References APPLIES TO: Hyperion Essbase - Version 11.1.2.1.000 and later Generic UNIX *** checked for currency 10-07-2014 *** PURPOSE This document will help in understanding setting up an Essbase active/passive failover environment. SCOPE This document is intended for System Administrators responsible for installing and configuring the Essbase clustered environment. DETAILS Pre-requisites Identical User IDs must be configured on both servers, for example "epmadmin". A shared drive must be created that is accessible and writable from both Essbase nodes. These types of shared drive are supported: SAN storage device with a shared disk file system supported on the installation platform such as OCFS. NAS device over a supported network protocol. Note: Any networked file system that can communicate with a NAS storage device is supported, but the cluster nodes must be able to access the same shared disk over that file system at the same time. SAN or a fast NAS device is recommended because of shorter I/O latency and failover times. The mount point on both servers for the shared disk must be identical. To successfully cluster Essbase 11.1.2.x in a Linux environment, the User IDs and passwords used to install/configure Essbase must match on both the Active and Passive nodes. The following must also match: Primary Group IDs Numeric UIDs Numeric Primary Group IDs The easiest way to accomplish this is to create the users on both nodes prior to installing the software, making sure that all of the above criteria are met. To check the numeric UIDs and Primary Group GIDs using the following command on both servers: # cat /etc/passwd | grep epmadin The following is displayed: epmadmin:x:502:502::/home/epmadmin:/bin/bash The two numbers ":502:502" are the UID and GID of the user. Tips on Configuring the Cluster For instructions on configuring the Essbase cluster, please read/review chapter, "OPMN Service Failover for Essbase Server" in the "Oracle Hyperion Enterprise Performance Management System Installation and Configuration Guide" and "Essbase Server Clustering and Failover" in the "Oracle Hyperion Enterprise Performance Management System High Availability and Disaster Recovery Guide ". Use different instance names when configuring the Essbase nodes. For example on the Active Node, use the instance name epminstance2 and on the Passive Node, use the instance name epminstance3. After the configuration is complete, modify the opmn.xml files on both the active and the passive nodes. Refer to the "Setting Up Active-Passive Essbase Clusters" chapter in the "Installation and Configuration Guide". Define the ias-component name in the opmn.xml file as the clustername defined during the configuration. This should be the same for both Essbase nodes. Testing the Cluster Once Essbase is installed and configured on both nodes, and the opmn.xml files have been modified, test starting the cluster and connecting to it from an Essbase Client. NOTE: The clustername used in the opmnctl commands are case-sensitive. 1. Start OPMN and Essbase on the Active Node using the opmnctl command: $ opmnctl startall 2. Verify that OPMN and Essbase are up and running on the Active Node: $./opmnctl status Confirm Essbase is running as a process: $ ps -ae | grep ESS 11517 ? 00:00:07 ESSBASE 3. Start OPMN (and Essbase) on the passive server: $./opmnctl start (or startall) 4. Verify that OPMN is running on the passive server but Essbase is Down: $./opmnctl status 5. Stop Essbase on the Active Node: $./opmnctl stopproc ias-component=EssbaseCluster-1 6. Check to see that Essbase stopped on the Active Node: $./opmnctl status 7. Check to see if Essbase started on the Passive Node: $./opmnctl status 8. Using MaxL, connect to the Essbase cluster. Use the full path to the Analytic Provider Service, for example: login admin password on "http://:/aps/Essbase?ClusterName=EssbaseCluster-1"; To connect using the cluster name, refer to How To Connect To an Essbase v11.1.2.x Cluster via MaxL or ESSCMD? NOTE: If you then run the same stopproc command on the second node, Essbase will NOT failover to the first node as it was explicitly stopped. To re-establish the failover, stop OPMN on both nodes and start up the first node then the second node. Notes: Only the Agent (ESSBASE) fails over. Applications are not restarted on an Agent failover. Clients will need to reconnect/relogin and re-submit their request. TIPS: Restart-on-Death Setting When restart-on-death is set to TRUE, OPMN will attempt to restart Essbase on the same node first for 3 times (first start + 2 retries, configurable in opmn.xml) and when all attempts fail, Essbase will failover to the second node. When set to FALSE, OPMN will immediately failover to the second node without a retry. If set to TRUE and Essbase crashes, it will be restarted on the same node in a few seconds. The only time it will take longer to restart Essbase is when the current node is having trouble restarting Essbase after all retries. Check the communications status of OPMN To confirm the communication between the Active and Passive Nodes, use the following command on both servers: $netstat -a | grep 671 You should see at least one entry in the results on each server showing a communications link to the other. For example: tcp 0 0 Passive.example.com:37409 Active.example.com:6712 ESTABLISHED Logs $MIDDLEWARE_HOME/user_projects/epmsystem1/diagnostics/logs/OPMN/opmn: console~Essbase2~EssbaseAgent~AGENT~1.log EssbasePing.log opmn.out console~EssbaseCluster-1~EssbaseAgent~AGENT~1.log opmn.log $MIDDLEWARE_HOME/user_projects/epmsystem1/diagnostics/logs/essbase/essbase_0: ESSBASE_ODL_1328087597.log ESSBASE_ODL.log leasemanager_essbase_{servername}.log OPMN The behavior of the OPMN process and all of the OPMN components are controlled and configured in the opmn.xml file. The following command can be used to detect any syntax errors in that file: opmnctl validate NOTE: This will only check for syntax and format related issues. It will not tell you if the information that you may have entered into the opmn.xml file is incorrect. Common OPMN Commands The OPMN process and the opmnctl command are used for the control of the Essbase process as of version 11.1.2. Here are a few commonly used opmnctl examples as they are used with Essbase: The following command starts the OPMN process only, not Essbase: opmnctl start To check the current status of OPMN and all associated components, use the following command: opmnctl status Once the OPMN process is running, you can then start Essbase in the following manner: opmnctl startproc ias-component=EssbaseCluster1 NOTE: You will need to use the correct name of the ias-component to use this command. You can find that name by using the "opmnctl status" command as described above. The command to stop Essbase is similar: opmnctl stopproc ias-component=EssbaseCluster1 The following command will start OPMN and all ias-components as defined in the opmn.xml file, (this will start OPMN and Essbase): opmnctl startall Similarly, if you'd like to stop all OPMN components and OPMN itself, you would use the following: opmnctl stopall Configurable Settings (essbase.cfg) Take care when setting these values, especially if the timeout value has been modified in the opmn.xml file. If the timeout is set too low, you may run into a situation where both nodes will try to start and obtain a lease. Essbase Agent: AGENTLEASEEXPIRATIONTIME - Specifies the number of seconds before a lease expires. AGENTLEASEMAXRETRYCOUNT - Specifies the number of times that Essbase Agent attempts to acquire or renew a lease. If the attempts are unsuccessful, the agent terminates itself. AGENTLEASERENEWALTIME - Specifies the time interval, in seconds, after which Essbase Agent attempts to renew a lease. This value must be less than the value of AGENTLEASEEXPIRATIONTIME. Essbase Server (ESSSVR/applications): SERVERLEASEEXPIRATIONTIME - Sets the maximum amount of time that Essbase Server can own a lease before the lease is terminated. SERVERLEASEMAXRETRYCOUNT - Specifies the number of times that Essbase Server attempts to acquire or renew a lease. If the attempts are unsuccessful, the server terminates itself. SERVERLEASERENEWALTIME - Specifies the time interval after which Essbase Server renews its lease. Bin Directories The cluster configuration will lay down two EssbaseServer/bin directories, one under the $MIDDLEWARE_HOME directory and another on the shared mount point. When troubleshooting issues, you will need to check both directories. Essbase Clustering Part 1 Essbase clustering can be used in order to mitigate the risk of an Essbase server going down and affecting your Planning or Reporting application with it. When it comes to Essbase clustering, however, there is one commonly used method which is to use the Essbase clusters that can be configured via the main installation program. These Essbase clusters are more like a “hot spare” configuration or active/passive. The problem with this approach is that you need to have two identical servers (ideally) but can only use one at a time not using any of the horsepower from the “passive” server. In order to have a cluster of this type you will need to configure a Microsoft Cluster to monitor both servers, services and DNS entries. Configure a cluster disk (outside of the Essbase cluster) to store the Essbase data files (ARBORPATH) and configure opmn to monitor Essbase process running on both servers. Ideally, MCS will detect a down server and point the cluster name to the other server and switch the shared disk. Then OPMN will detect that Essbase is not running on the downed server and start it on the server that is still available. Does this sound like a lot? There’s another option… Another method of clustering is an active/active cluster, which can mitigate the risk of having only one Essbase server and also serve as a load balancer.The main “gotcha” for an active/active cluster is that you can not write back to it which is required for Planning applications. This is the reason this type of clusters is not as popular and is almost exclusively used with ASO reporting applications. Another gotcha of active/active clusters is that you have to build your cubes and load data on both servers. These need to be maintained in-sync in order to make sure that users being sent to one server see exactly the same as other servers in the cluster. This method uses APS and the JAPI to establish server pools like so: The main advantage for this setup is that you don’t have to configure any MS Clustering and you can treat each server individually with its own ARBORPATH’s. In EAS, each will show as an individual server with their own set of applications. On the downside, you will have to build each application on each server. Again, this is most suitable for ASO applications or BSO apps that do not require write back (which rules out Planning) and can take advantage of the horse power of both servers at all times (unless a server goes down) I will be writing two follow up posts on how to configure each type of cluster identified here.