Showing posts with label Apps DBA. Show all posts
Showing posts with label Apps DBA. Show all posts

21.6.26

How to Fix Performance Degradation in Oracle EBS Concurrent Managers

 When Oracle E-Business Suite (EBS) performance degrades, the root cause is frequently found within the Concurrent Processing (CP) subsystem. If users are complaining that routine reports are stuck in "Pending" status, invoices are processing at a snail's pace, or your night batch jobs are bleeding into business hours, your Concurrent Managers are hitting a bottleneck.

Fixing this issue requires looking beyond general database tuning. You must optimize how the Concurrent Managers handle workload distribution, queue definitions, and internal purging.
In this comprehensive guide, we will break down the exact steps to diagnose and remediate performance degradation in Oracle EBS Concurrent Managers.
Phase 1: Diagnosing the Performance Bottleneck
Before tweaking settings, you must identify whether the slowdown is caused by system resource starvation, queue misconfigurations, or database contention.
1. Check for Queue Backlogs
Log in to Oracle Applications as the System Administrator responsibility and navigate to:
Concurrent > Manager > Administer
Look at the Target versus Actual process counts. If a manager's Target process count is higher than its Actual count, that manager is failing to launch internal workers, causing jobs to queue up indefinitely.
2. Run the Concurrent Manager Status Script
Run the standard Oracle-provided script from the application tier ($FND_TOP/sql/afcmstat.sql) to view a live status report of your managers, their queues, and active processes.
3. Analyze the Concurrent Manager Log Files
Review the Internal Concurrent Manager (ICM) log file found in the $APPLCSF/$APPLLOG directory (usually named ICM*.mgr). Look for repeating error messages, memory allocation failures, or timeout warnings like ALERTER: Max continuous errors exceeded.
Phase 2: Core Reasons for Performance Degradation
Through thousands of real-world DBA cases, performance drops in Concurrent Managers usually boil down to these four critical issues:
  • Bloated FND Tables: The underlying tracking tables (FND_CONCURRENT_REQUESTS and FND_CONCURRENT_PROCESSES) have grown to millions of rows, slowing down the internal queries the ICM runs to fetch the next job.
  • Improper Cache Size Configuration: If a manager's Cache Size is set too low, it frequently queries the database to grab new requests, causing high database contention.
  • Suboptimal Worker Allocations: Standard managers (like the Standard Manager or Conflict Resolution Manager) do not have enough target processes allocated to handle sudden spikes in user request volume.
  • Database Level Wait Events: Concurrent processes are being blocked at the database layer by locks, slow I/O, or high CPU utilization.
Phase 3: Step-by-Step Performance Tuning Solutions
Follow these systematic solutions to restore speed to your Oracle EBS Concurrent Processing environment.
Step 1: Clean and Purge Concurrent Request Tables (The #1 Fix)
The single most effective action a DBA can take is cleaning up the historical tracking tables.
  1. Schedule the standard concurrent program: Purge Concurrent Request and/or Manager Data.
  2. Run it weekly or nightly with the parameter Criterion = Age and Days = 7 or 30 (depending on your business retention requirements).
  3. If the tables are already massively bloated, running the purge program might hang. In that case, look into Oracle Support Note 213824.1 for manual truncation and rebuild scripts for FND_CONCURRENT_REQUESTS.
Step 2: Optimize Manager Definitions (Target Processes & Cache)
Adjusting the way queues process requests relieves massive software bottlenecks:
  1. Navigate to Concurrent > Manager > Define.
  2. Query the Standard Manager.
  3. Click on Work Shifts.
  4. Increase Target Processes: Raise the number of workers to match your server's hardware capability. (e.g., if it is set to 4, increase it to 8 or 12).
  5. Adjust Cache Size: Increase the Cache Size parameter to 10 or 20. This allows the manager to look ahead and cache multiple requests in memory at once, dramatically reducing queries to the database.
Step 3: Isolate Heavy Reports Using Dedicated Managers
If a few specific, massive custom reports (like heavy financial or inventory extracts) are locking up the Standard Manager queue, isolate them:
  1. Create a Custom Concurrent Manager dedicated only to those heavy programs.
  2. Assign specialization rules to the Standard Manager to Exclude those specific programs.
  3. Assign specialization rules to your Custom Manager to Include only those programs.
  4. This ensures that a single massive report will never freeze the entire business operations queue.
Step 4: Reorganize and Reindex FND Tables
Because FND_CONCURRENT_REQUESTS experiences constant INSERT, UPDATE, and DELETE actions daily, its indexes become highly fragmented.
  • Coordinate a maintenance window to rebuild indexes on FND_CONCURRENT_REQUESTS and FND_CONCURRENT_PROCESSES.
  • Gather fresh schema statistics using the program: Gather Schema Statistics for the APPLSYS schema.

Conclusion
Tuning Oracle EBS Concurrent Managers is not a one-time task; it requires proactive maintenance. By establishing a rigid automated purging schedule, resizing your target processes to fit your hardware capabilities, and isolating heavy custom code from your standard flows, you will drastically decrease request queue times and improve system responsiveness.