EBS XML Publisher Request Fails Due to Output Post Processing Issue

Please refer and subscribe the blogger below . http://r12ebs.blogspot.qa/2014/10/xml-bi-publisher-concurrent-requests.html http://madhanappsdba.blogspot.qa/2013/12/output-post-processor-opp-in-oracle.html XML Publisher Request Fails Due to Output Post Processing Issue OPP Log file: To obtain the OPP log file, follow the bellow steps: § Select the System Administrator responsibility. § Navigate to Concurrentà ManageràAdminister. § Select the Output Post Processor Service. § Select the Processes button. § Select Manager Log. Issues: BI Publisher concurrent request completed with warning due to a timeout caused by the Output Post Processor (OPP). Given bellow the possible common errors may occur due to OPP issue: Issue 1. OPP No Response: +------------- 1) PUBLISH -------------+ The Output Post-processor is running but has not picked up this request. No further attempts will be made to post-process this request, and the request will be marked with Warning status. Setting the profile option Concurrent: OPP Response Timeout to a higher value may be necessary. +--------------------------------------+ Issue 2. OPP Process Timeout: +------------- 1) PUBLISH -------------+ The concurrent manager has timed out waiting for the Output Post-processor to finish this request. Check that there are enough Output Post-processor service processes running. More information may be found in the service process logfile. +--------------------------------------+ Issue 3. OPP No Response and OPP Process Timeout: +------------- 1) PUBLISH -------------+ The Output Post-processor is running but has not picked up this request. No further attempts will be made to post-process this request, and the request will be marked with Warning status. Setting the profile option Concurrent: OPP Response Timeout to a higher value may be necessary. +--------------------------------------+ Issue 4. The OPP is not running or does not pick up the request: +------------- 1) PUBLISH -------------+ Unable to find an Output Post Processor service to post-process request [request_id]. Check that the Output Post Processor service is running. +--------------------------------------+ Issue 5. OPP log contains error java.lang.OutOfMemoryError: Java heap space Cause: 1 1. OPP No Response: The concurrent manager process generates the XML data file. Upon completion it will trigger the OPP in order to merge the XML data file and the template which was selected on the Submit Request form. The number of concurrent requests that the OPP can handle in parallel depends upon: the number of Processes the number of Threads Per Process The default values are 2 Processes and 2 Threads per Process so a total of 4 reports can be processed in parallel. In case there are other concurrent requests running which have already invoked the OPP then it might happen that no additional requests can be picked up for a period of time. The pending request will be picked up as soon as one of the running jobs completes. By default a timeout will occur if it takes longer than 120 seconds (2 min.) for the Output Post Processor to pick up the request from the concurrent manager process. In that case, the concurrent request will complete with status Warning and the request log file will contain Error Message 1 (see above). 2. OPP Process Timeout: Once the OPP picks up the request, the BI Publisher engine is invoked to generate the final output file. The time that this takes will depends on various elements such as: size of the XML Data File complexity of the template performance of the server By default a timeout will occur if it takes longer than 300 seconds (5 min.) for the BI Publisher engine to generate the output file. The concurrent request will complete with status Warning and the request log file will contain Error Message 2. 3. OPP No Response and OPP Process Timeout: The OPP service is not picking up any new requests. The OPP JAVA process is still running, but it is unresponsive. This can be due to errors in previous requests or just to the amount of time the manager has been running. The OPP becomes unresponsive (stale) after running for more than one week. 4. The OPP is not running or does not pick up the request: The error generally occurs if the OPP service is not running or the OPP service is not active 5. OPP log contains error java.lang.OutOfMemoryError: Java heap space: Solution: Two profiles options actually control the timeouts of OPP. i. Concurrent:OPP Response Timeout ii. Concurrent:OPP Process Timeout OPP No Response: i. Increase the value (in seconds) for the profile option 'Concurrent:OPP Response Timeout'. If the current value is 120 then set it to 240. (Value *2) ii. Increase the number of processes or threads (or both) of the OPP via System Administrator iii. Stop and restart the OPP service OPP Process Timeout: Increase the value (in seconds) for the profile option 'Concurrent:OPP Process Timeout'. OPP No Response and OPP Process Timeout: i. Increase the value (in seconds) for the profile option 'Concurrent:OPP Response Timeout'. If the current value is 120 then set it to 240. (Value *2) ii. Increase the value (in seconds) for the profile option 'Concurrent:OPP Process Timeout' Note: Oracle recommends setting the profiles to the following values to minimize seeing the occurrence of the Warning in your log file, especially if you are submitting large reports: Concurrent: OPP Processing Timeout to 86400 (24 hours) Concurrent: OPP Response Timeout to 3600 (1 hour) iii. Bounce the concurrent managers and resubmit. 4. The OPP is not running or does not pick up the request: Insure the OPP manager is running. Note: If the OPP manager is not running then navigate to the Oracle Applications Manager responsibility - Concurrent Managers - Select Output Post Processor - View Details - Stop and then Start the Manager again. 5. OPP log contains error java.lang.OutOfMemoryError: Java heap space: Generic errors can be resolved by performing either of the following steps: § Log into the XML Publisher Administrator responsibility. § Navigate to Home – Administration – Configuration. § Under the General Properties select Temporary directory. § Select a temporary file location on your concurrent processing node. This should be at least 5Gb or 20 times larger than the largest XML data file you generate. OR § Get the heap size per OPP process is currently defined: SELECT developer_parameters FROM fnd_cp_services WHERE service_id = (SELECT manager_type FROM fnd_concurrent_queues WHERE concurrent_queue_name = 'FNDCPOPP' ); The default should be: J:oracle.apps.fnd.cp.gsf.GSMServiceController:-mx512m § Increase the Heap Space per Process to 1024: UPDATE fnd_cp_services SET developer_parameters ='J:oracle.apps.fnd.cp.gsf.GSMServiceController:-mx1024m' WHERE service_id = (SELECT manager_type FROM fnd_concurrent_queues WHERE concurrent_queue_name = 'FNDCPOPP' ); § Bounce the Concurrent Manager Output Post Processor (OPP) in Oracle Applications R12 and 11i What is Output Post Processor? Concurrent Processing now uses the Output Post Processor (OPP) to enforce post-processing actions for concurrent requests.Post-processing actions are actions taken on concurrent request output. An example of a post-processing action is that used in Concurrent Processing support of XML Publisher. If a request is submitted with an XML Publisher template specified as a layout for the concurrent request output, then after the concurrent manager finishes running the concurrent program, it will contact the OPP to apply the XML Publisher template and create the final output. OPP runs as a service that can be managed through Oracle Applications Manager (OAM) from the System Activity page (Navigation: Applications Dashboard > Applications Service (from the dropdown list) > Go). How to Increase the number of Output Post Processors? 1.Log on to Applications with “System Administrator” responsibility. 2.Navigate to Concurrent -> Manager -> Define. 3.Query for the “Output Post Processor” service. 4.Click on “Work Shifts” and increase the number of processes How to get OPP manager log file location? $APPLCSF/log//FNDOPP####.txt OR 1,System Administrator > Concurrent > Manager > Administer 2,Search for ‘Output Post Processor’ 3,Click the ‘Processes’ button . 4,Click the Manager Log button. This will open the ‘OPP’ Upload the OPP log file. In some cases, Output Post Processor is not start up and it shows Actual and Target are showing different values when we query for Output Post Processor. The log files shows that no error message.In this case apply the following possible solution for starting the OPP. 1. Shutdown the internal manager by using adcmctl.sh stop apps/apps 2. Make sure there is no FNDLIBR processe running: $ ps -ef| grep FNDLIBR OR ps -ef|grep applprod|grep FNDLIBR 3. If there is any FNDLIBR processe please kill it $ kill -9 pid 4. Run cmclean.sql script as document from Note 134007.1 5. Restart the internal manager by using adcmctl.sh start apps/apps or How to kill and start Output Post Processor (OPP) Background: This article explains how to kill OPP and restart the same Solution: 1,System Administator > Concurrent > Manager > Administer 2,Query "Output Post Processor" -> Processes button Get the sytem id of "Output Post Processor" 3,ps -ef|grep [system id] 4,kill -9 [system id] in Unix 5,System Administator > Concurrent > Manager > Administer 6,Query "Output Post Processor" -> Restart button Fixing Output Post-processor actions failed issues in EBS R12 Environment: Oracle EBS 12.1.3, Oracle Database 11gR2, RedHat Linux5 Symptoms: Users unable to open the out files. Concurrent requests failed with “Post-processing of request failed error message” One or more post-processing actions failed. Consult the OPP service log for details. No further attempts will be made to post-process this request. Cause: The concurrent manager process was able to successfully invoke the Output Post-Processor (OPP) but encountered a timeout as the OPP takes longer than the value assigned to complete the job. Solution: 1,Increase the value of profile Concurrent: OPP Response Timeout . Bounce Apache and retest. 2,If the issue still exists, perform the following steps. 3,Increase the number of Output Post Processors as follows: 4,Increase the number of processes for Output Post Processor. 5,Additionally, ensure there is a setting of oracle.apps.fnd.cp.opp.OPPServiceThread:2:0:max_threads=5 under Parameters. For more details Please check metalink I.E oracle.support Concurrent Requests Fail Due to Output Post Processing (OPP) Timeout [ID 352518.1] Ouput Post Processing Fails Due To java.lang.ThreadDeath [ID 427233.1] Why Does OPP Intermittently Completes With Warnings and Error 'java.lang.OutOfMemoryError'? [ID 978495.1] To know more this post is very nice http://knoworacleappsdba.blogspot.in/2012/07/all-about-output-post-processor-opp-in.html How to start the Output Post Processing (OPP) in 11i At least one OPP process active in the system. The concurrent processing uses the Output Post Processor (OPP) to enforce post-processing actions for concurrent requests. For example, post-processing action is that used in publishing concurrent requests with XML Publisher. Actually when you have submit a request with XML Publisher template specified as a layout for the concurrent request output. Once finishes the concurrent manager concurrent program, it will contact the OPP to apply the XML Publisher template and create the final output To activate the OPP follow these setps The Profile Option "Concurrent: GSM Enabled" must be set to Y Then: 1. Login to Apps with sysadmin responsibility 2. Navigate to: Concurrent -> Managers -> Define 3. Query for Manager = 'Output Post Processor%' or Short Name = FNDCPOPP 4. Check the checkbox "Enable" . 5. Click on 'Work Shifts button 6. see Work Shift of the OPP and Set Processes = 1 and Parameters = oracle.apps.fnd.cp.opp.OPPServiceThread:2:0:max_threads=5 and Sleep Second = 30 7. Save How to restart Oracle Apps 11i OPP via non unix command It can be done using adcmctl.sh script only and there is no specific script for OPP. You can restart it from the application via (System Administrator responsibility > Concurrent > Manager > Administer), select "Output Post Processor" and click on the "Restart" button. OPP Issue while generating the XML Output in apps 11i Problem Description: 1, Few XML type concurrent programs taking long time and completed with warning OPP log file registered below error Caused by: java.lang.ThreadDeath 2, CPU utilization taking 100% constantly while running the concurrent programs. Cause The java.lang.ThreadDeath error indicates that the Output Post Processor has reached its processing timeout. Solution The following configuration changes are recommended to optimize the environment for these type of reports: 1. Increase the value of the Concurrent: OPP Timeout profile option to 10800 seconds. 2. Enable the scalability feature of XML Publisher: 1. Login as SYSADMIN 2. Responsibility: XML Publisher Administrator 3. Function: Administration 4. Set the following properties: 5. Temporary Directory 6. Use XML Publisher's XSLT processor: True 7. Enable scalable feature of XSLT processor: True -- By default it’s false 8. Enable XSLT runtime optimization: True > After changing the value also the problem still exist. As per the SR suggestion, we increased the java heap size from 1024M to 2048M. a. Bring down the concurrent managers. b. Use the Update statement below, for example: update FND_CP_SERVICES set DEVELOPER_PARAMETERS = 'J:oracle.apps.fnd.cp.gsf.GSMServiceController:-mx2048m' where SERVICE_ID = (select MANAGER_TYPE from FND_CONCURRENT_QUEUES where CONCURRENT_QUEUE_NAME = 'FNDCPOPP'); c. Bring concurrent managers up again Please check below note's from oracle support 427233.1 :Output Post Processing Fails Due To java.lang.ThreadDeath 1268217.1: Output Post Processor (OPP) Log Contains Error “java.lang.OutOfMemoryError: Java heap space" 1266368.1: Output Post Processor (OPP) Log Contains Error "java.lang.OutOfMemoryError" 978495.1: Why Does OPP Intermittently Completes With Warnings and Error 'java.lang.OutOfMemoryError'? 352518.1 :Concurrent Requests Fail Due to Output Post Processing (OPP) Timeout 364547.1 :Troubleshooting Oracle XML Publisher For The Oracle E-Business Suite