Fri Dec 10 03:34:36 2010

Thread 1 advanced to log sequence 111366

  Current log# 2 seq# 111366 mem# 0: /oradata/oradata/ngcclog/redo02.log

Fri Dec 10 03:34:36 2010

ARC1: Evaluating archive   log 1 thread 1 sequence 111365

ARC1: Beginning to archive log 1 thread 1 sequence 111365

Creating archive destination LOG_ARCHIVE_DEST_1: '/archivelogs/arch_639402486_1_111365.arc'

ARC1: Completed archiving  log 1 thread 1 sequence 111365

Fri Dec 10 04:20:08 2010

Thread 1 advanced to log sequence 111367

  Current log# 4 seq# 111367 mem# 0: /oradata/oradata/ngcclog/redo04.log

Fri Dec 10 04:20:08 2010

ARC0: Evaluating archive   log 2 thread 1 sequence 111366

ARC0: Beginning to archive log 2 thread 1 sequence 111366

Creating archive destination LOG_ARCHIVE_DEST_1: '/archivelogs/arch_639402486_1_111366.arc'

ARC0: Completed archiving  log 2 thread 1 sequence 111366

Fri Dec 10 04:41:09 2010

ARC1: Evaluating archive   log 4 thread 1 sequence 111367

Fri Dec 10 04:41:09 2010

  Current log# 5 seq# 111368 mem# 0: /oradata/oradata/ngcclog/redo05.log

Fri Dec 10 04:41:09 2010

ARC1: Beginning to archive log 4 thread 1 sequence 111367

Creating archive destination LOG_ARCHIVE_DEST_1: '/archivelogs/arch_639402486_1_111367.arc'

ARC1: Completed archiving  log 4 thread 1 sequence 111367

Fri Dec 10 05:09:26 2010

Errors in file /opt/ora92/admin/ngcclog/bdump/ngcclog_arc0_21423.trc:

ORA-00600: internal error code, arguments: [2103], [1], [0], [1], [900], [], [], []

Fri Dec 10 05:09:29 2010

Errors in file /opt/ora92/admin/ngcclog/bdump/ngcclog_arc0_21423.trc:

ORA-00600: internal error code, arguments: [2103], [1], [0], [1], [900], [], [], []

Fri Dec 10 05:09:29 2010

Errors in file /opt/ora92/admin/ngcclog/bdump/ngcclog_arc0_21423.trc:

ORA-00600: internal error code, arguments: [2103], [1], [0], [1], [900], [], [], []

Fri Dec 10 07:21:03 2010

Errors in file /opt/ora92/admin/ngcclog/bdump/ngcclog_ora_5581.trc:

ORA-00600: internal error code, arguments: [12235], [], [], [], [], [], [], []

Fri Dec 10 07:28:06 2010

Shutting down instance: further logons disabled

Fri Dec 10 07:45:14 2010

Errors in file /opt/ora92/admin/ngcclog/bdump/ngcclog_ora_23872.trc:

ORA-00600: internal error code, arguments: [12235], [], [], [], [], [], [], []

Fri Dec 10 09:00:54 2010

Starting ORACLE instance (normal)

LICENSE_MAX_SESSION = 0

这个库 每天晚上会跑一些后台 取数的脚本，用于 第二天的报表展示，所以，每天凌晨 1:00 到 6:00 应该说还是比较繁忙的。

可以看到，四点前有一些归档，可能 1 分钟 产生一个，但是到 4:41 到 5:09 分的时候，感觉突然就中断了，我估计是可能 归档太频繁，导致了 checkpoint not completed 的事件产生，后来 到 5:09 分就出现了 600 的错误，看了下 metalink，说是数据库太繁忙，导致一些 进程 死在那边。从数据库的现象上看，5 点后，数据库就死了，到 7:45 分的时候，完全宕机，后来就重启 数据库服务，之后就正常了。

Metalink：id 28045.1 and 33174.1