一个多灾多难的系统：）

HP-UX 11.31 IA64 Oracle 10.2.0.1 两节点RAC

数据库集群自动重启，检查alert文件，报ORA-00600错误

Mon Nov 14 15:43:53 2011
Errors in file /u01/app/oracle/admin/ufsa8/udump/ufsa81_ora_4531.trc:
ORA-00600: internal error code, arguments: [KGHALO4], [0xC0000000BD0CD060], [], [], [], [], [], []
Mon Nov 14 15:43:58 2011
Trace dumping is performing id=[cdmp_20111114154358]
Mon Nov 14 15:43:58 2011
Errors in file /u01/app/oracle/admin/ufsa8/udump/ufsa81_ora_4531.trc:
ORA-00600: internal error code, arguments: [600], [], [], [], [], [], [], []

检查support.oracle.com，有不少BUG与此相关，无法马上定位具体的原因。

于是检查trace文件，在ufsa81_ora_4531.trc文件中发现：

[@more@]

C0000000BD0CD050 00000000 00000000 09820B00 00000000 [................]
C0000000BD0CD060 C0000002 A78EAE50 00000A98 4E554D41 [.......P....NUMA]
C0000000BD0CD070 20706F6F 6C203000 00000000 00E87FFF [ pool 0.........]
C0000000BD0CD080 008000B0 00000000 00000000 00000000 [................]
C0000000BD0CD090 00000000 00000038 C0000000 BD0CD098 [.......8........]

ksmasgn()+144 call 9ffffffffffe53e0 6000000000015190 ?

基本符合Bug 4414666 OERI[KGHALO4] can occur on NUMA的描述，猜测与此有关

Bug 4414666 OERI[KGHALO4] can occur on NUMA

This note gives a brief overview of bug 4414666.
The content was last updated on: 09-MAY-2008
Click here for details of each of the sections below.

Affects:

Product (Component)	Oracle Server (Rdbms)
Range of versions believed to be affected	Versions < 11
Versions confirmed as being affected	10.2.0.1
Platforms affected	Generic (all / most platforms affected)

Fixed:

This issue is fixed in

10.2.0.2 (Server Patch Set) 11.1.0.6 (Base Release)

Description

ORA-600 [KGHALO4] can on a NUMA machine
with ksmasgn in the call stack.

Workaround:
  Add the following parameter to init.ora:
   _enable_NUMA_optimization=FALSE
 Or 
  run with a larger shared_pool_size parameter.

References

Bug:4414666 (This link will only work for PUBLISHED bugs)
Note:245840.1 Information on the sections in this article