SAP Basis Tech Supports

Symptom

Your J2EE Engine is stuck. XI/PI Message flow through the Adapter Framework has stopped. On 6.40 and 7.0 systems logon to the Visual Administrator isn’t possible any longer, in 7.1 systems SAP NetWeaver Administrator hangs. System startup failes, hangs or takes very long.

Other terms

Application Thread Manager, Max Thread Count, Messaging System, MessagingSystem, XI AF Adapter Framework, AdapterFramework hangs, System Threads, System Level Threads, Thread Dump, system hangs, system stuck, Engine stuck, sysDS.maximumConnections, DB Pool Size, DBPool, Process Integration, SAP NetWeaver for PI

Reason and Prerequisites

You are working with XI 3.0, PI 7.0, 7.1 or a higher release. According to the thread dump (see referenced note), all system/application threads are in use.
The configured number of application threads can be displayed with:
6.40, 7.0: Visual Administrator: Server -> Kernel -> ApplicationThreadManager -> MaxThreadCount7.1: Config Tool -> cluster-data -> -> -> managers -> ApplicationThreadManager -> MaxThreadCount

Regarding to the installation guide the MaxThreadCount should have been set to 150 at least – this depends on your scenario.
The configured number of system level threads can be found at: Server -> Kernel -> ThreadManager -> MaxThreadCount, InitialThreadCount, MinThreadCount.
Note: In 6.40 and 7.0 systems these values can be maintained directly in the Visual Administrator or, if the system doesn’t start, with the “Config Tool”. In 7.1 systems the “Config Tool” must be used exclusively.
Note: The “Config Tool” can be found in the file system at /usr/sap///j2ee/configtool/configtool.[suffix].

Solution

A possible solution to solve this problem is to increase the number of application and system level threads.
Application Level Threads: Start the “Config Tool” and browse to:
6.40, 7.0: cluster_data -> instance_ID -> managers -> ApplicationThreadManager7.1: cluster-data -> -> -> managers -> ApplicationThreadManager

Switch to Edit Mode and apply custome value MaxThreadCounts = 350 or more.
System Level Threads: Browse to
6.40, 7.0: cluster_data -> instance_ID -> managers -> ThreadManager7.1: cluster-data -> -> -> managers -> ThreadManager

Switch to Edit Mode and apply custome values MaxThreadCount = 120, MinThreadCount = 80, InitialThreadCount = 80.
Note: In some high load scenarios it might be required to increase the number of maximum database connections (DB pool size) from its default of 100 to a higher value as well:
6.40, 7.0: Visual Administrator: Browse to the “JDBC Connector” Service, choose the corresponding DataSource and open the “Additional” tab strip. Increase in section “Connection Pooling” the number of maximum connections respectively. See note 924524 for details. Save your settings. Adjust according to these settings the database specific session count as well !7.1: SAP NetWeaver Administrator: Browse to Configuration Management -> Infrastructure -> Application Resources. Select your data source and open the “Connection Polling” tab strip. Check number of maximum connections respectively. To change them start the “Config Tool” and navigate to the “dbpool” service.

Under some circumstances it makes sense to increase these DB related settings further, in accordance with the above number of application and system level threads. However, the maximum efficiency scales not linear with the number of DB connections. A second server node with a default database connections size might be more efficient. A general rule for that cannot be given in this note.
Note: Increase of application threads does not necessarily solve the problems regarding the stopped message flow through the Adapter Framework. See the attached FAQ notes under which circumstances this can happen.