PMON failed to acquire latch, see PMON dump

    这两天突然发现有套在运行的Oracle 10.2.0.1 for RHEL 5.8x86_x64的数据库关闭非常慢，长达4分钟，shutdown immediate之后alert.log报大量的”PMON failed to acquire latch, see PMON dump”，该库得连接数也不多，当前Process 是60，除了该错误之外没有报任何异常，也没有连接错误，但是进程数一直居高不下，到网上查了一下，据说是个bug，在Metalink上找到了解决办法，由于当前数据库暂时不停停机升级修复，故通过修改SQLNET.ora参数先测试用一段时间。

 ********************************************************************

To implement the solution, execute the following steps:

1. In $ORACLE_HOME/network/admin/listener.ora, of the database where the Metadata Repository is to be loaded, add the line:
INBOUND_CONNECT_TIMEOUT_LISTENER = 0

2. In the Oracle Server 10g sqlnet.ora, add the line:
SQLNET.INBOUND_CONNECT_TIMEOUT = 0

3. Stop and restart the database and the listener for the changes to take effect

这是metalink上的：

Applies to:
Oracle Server - Enterprise Edition - Version: 10.2.0.1.0 to 10.2.0.3.0

This problem can occur on any platform.

Symptoms
Database Instance hangs and connections to database using 'sqlplus' are also not possible.

Checking alert.log we see following messages

PMON failed to acquire latch, see PMON dump
Fri Oct 5 10:33:00 2007
PMON failed to acquire latch, see PMON dump
Fri Oct 5 10:34:05 2007
PMON failed to acquire latch, see PMON dump
Errors in file /dwrac/BDUMP/dwhp_pmon_1912834.trc:

This will also dump a systemstate dump and the location will be mentioned in alert.log

Also at OS level, we see that MMAN is consuming lot of CPU.

Cause

Currently this issue is being worked upon by development in

Bug 6488694
- DATABSE HUNG WITH PMON FAILED TO ACQUIRE LATCH MESSAGE

Solution
As of now only workaround is to disable Automatic Shared Memory Management (

ASMM
) i.e Setting

SGA_TARGET=0

Also as per bug, you can can set following event and restart the instance

EVENT="10235 trace name context forever, level 2"

Development is suspecting memory corruption in this case. So with the above event, database might hit ORA-600 before spin. The trace file of ORA-600 would help investigate the issue. These trace files need to be sent to Oracle support for investigation.

Note : - Event 10235 with level 2 or higher can impact latch contention.Though may not be quite critical. In case you see latch contention then unset the event

References
Bug 6488694
- DATABSE HUNG WITH PMON FAILED TO ACQUIRE LATCH MESSAGE

******************************************************************