某客户的一个历史库遇到错误, 数据库不定期崩溃,从日志中可以看到如下内容:

Errors in file /u01/app/oracle/diag/rdbms/orcl2/orcl2/trace/orcl2_smon_6364.trc  (incident=336027):

ORA-00600: internal error code, arguments: [13013], [5001], [267], [8389014], [4], [8389014], [3], [], [], [], [], []

Incident details in: /u01/app/oracle/diag/rdbms/orcl2/orcl2/incident/incdir_336027/orcl2_smon_6364_i336027.trc

Wed Jul 01 01:10:43 2020

System state dump requested by (instance=1, osid=6334 (PMON)), summary=[abnormal instance termination].

System State dumped to trace file /u01/app/oracle/diag/rdbms/orcl2/orcl2/trace/orcl2_diag_6344_20200701011043.trc

Wed Jul 01 01:10:43 2020

PMON (ospid: 6334): terminating the instance due to error 474

Instance terminated by PMON, pid = 6334

此ORA-00600官方解释如下:

ORA-600 [13013] [a] [b] [c] [d] [e] [f]
This format relates to Oracle Server 8.0.3 to 10.1

Arg [a] Passcount
Arg [b] Data Object number
Arg [c] Tablespace Relative DBA of block containing the row to be updated
Arg [d] Row Slot number
Arg [e] Relative DBA of block being updated (should be same as [c])

Arg [f] Code

查询出错的数据库对象:

SQL> Select object_name,object_type,owner from dba_objects where data_object_id=267;

OBJECT_NAME      OBJECT_TYPE OWNER

-------------------- ------------------- ------------------------------

SMON_SCN_TIME      TABLE SYS

SMON_SCN_TO_TIME_AUX CLUSTER SYS

SQL> select dbms_utility.data_block_address_file(8389014) Rfile# ,dbms_utility.data_block_address_block(8389014) "Block#" from dual;

    RFILE#     Block#

---------- ----------

2   406

对表进行校验并进行DBV校验数据文件

SQL> Analyze table sys.smon_scn_time validate structure cascade online ;

Analyze table sys.smon_scn_time validate structure cascade online

*

ERROR at line 1:

ORA-01499: table/index cross reference failure - see trace file

SQL> /

Analyze table sys.smon_scn_time validate structure cascade online

*

ERROR at line 1:

ORA-01499: table/index cross reference failure - see trace file

分析表报告表结构有问题

[oracle@orcl2db trace]$ dbv file=/u02/oradata/orcl2/SYSAUX01.DBF

DBVERIFY: Release 11.2.0.4.0 - Production on Thu Jul 2 09:33:24 2020

Copyright (c) 1982, 2011, Oracle and/or its affiliates.  All rights reserved.

DBVERIFY - Verification starting : FILE = /u02/oradata/orcl2/SYSAUX01.DBF

DBVERIFY - Verification complete

Total Pages Examined         : 1148160

Total Pages Processed (Data) : 520451

Total Pages Failing   (Data) : 0

Total Pages Processed (Index): 505161

Total Pages Failing   (Index): 0

Total Pages Processed (Lob)  : 9746

Total Pages Failing   (Lob)  : 0

Total Pages Processed (Other): 32538

Total Pages Processed (Seg)  : 0

Total Pages Failing   (Seg)  : 0

Total Pages Empty            : 80264

Total Pages Marked Corrupt   : 0

Total Pages Influx           : 0

Total Pages Encrypted        : 0

Highest block SCN            : 1480172039 (0.1480172039)

DBV没有发现存在坏块

判断可能索引出了问题。查询问题表有哪些索引

SQL> select owner, index_name, table_owner from dba_indexes where table_name = 'SMON_SCN_TIME';

OWNER        INDEX_NAME       TABLE_OWNER

------------------------------ ------------------------------ ------------------------------

SYS        SMON_SCN_TIME_SCN_IDX       SYS

SYS        SMON_SCN_TIME_TIM_IDX       SYS

重建两个索引，并进行分析：

SQL> alter index  SMON_SCN_TIME_SCN_IDX rebuild online;

Index altered.

SQL> alter index SMON_SCN_TIME_TIM_IDX rebuild online;

Index altered.

SQL> 

SQL> 

SQL> 

SQL> Analyze table sys.smon_scn_time validate structure cascade online ;

Table analyzed.

analyze并未出错。问题应当已经解决