asm磁盘组增加磁盘进行扩容之后报ORA-15335: ASM metadata corruption detected in disk group ‘DATA’和ORA-15196: invalid ASM block header [kfc.c:26368] [check_kfbh] [2147483648] [7] [2183628676 != 686982479],磁盘组dismount,然后mount之后立马dismount掉.

Tue Jun 29 09:19:09 2021

SQL> ALTER DISKGROUP DATA ADD  DISK 
'/dev/raw/raw5' 
SIZE 102400M /* ASMCA */ 

NOTE: GroupBlock outside rolling migration privileged region

NOTE: Assigning number (2,1) to disk (
/dev/raw/raw5
)

NOTE: requesting all-instance membership refresh 
for 
group=2

NOTE: initializing header on grp 2 disk DATA_0001

NOTE: requesting all-instance disk validation 
for 
group=2

Tue Jun 29 09:19:11 2021

NOTE: skipping rediscovery 
for 
group 2
/0xb0c845ce 
(DATA) on 
local 
instance.

NOTE: requesting all-instance disk validation 
for 
group=2

NOTE: skipping rediscovery 
for 
group 2
/0xb0c845ce 
(DATA) on 
local 
instance.

NOTE: initiating PST update: grp = 2

Tue Jun 29 09:19:16 2021

GMON updating group 2 at 7 
for 
pid 27, osid 25020

NOTE: PST update grp = 2 completed successfully 

NOTE: membership refresh pending 
for 
group 2
/0xb0c845ce 
(DATA)

GMON querying group 2 at 8 
for 
pid 18, osid 3852

NOTE: cache opening disk 1 of grp 2: DATA_0001 path:
/dev/raw/raw5

NOTE: Attempting voting 
file 
refresh on diskgroup DATA

NOTE: Refresh completed on diskgroup DATA. No voting 
file 
found.

GMON querying group 2 at 9 
for 
pid 18, osid 3852

SUCCESS: refreshed membership 
for 
2
/0xb0c845ce 
(DATA)

Tue Jun 29 09:19:20 2021

SUCCESS: ALTER DISKGROUP DATA ADD  DISK 
'/dev/raw/raw5' 
SIZE 102400M /* ASMCA */

NOTE: starting rebalance of group 2
/0xb0c845ce 
(DATA) at power 1

Starting background process ARB0

Tue Jun 29 09:19:21 2021

ARB0 started with pid=33, OS 
id
=25176 

NOTE: assigning ARB0 to group 2
/0xb0c845ce 
(DATA) with 1 parallel I
/O

cellip.ora not found.

Tue Jun 29 09:19:24 2021

NOTE: Attempting voting 
file 
refresh on diskgroup DATA

NOTE: Refresh completed on diskgroup DATA. No voting 
file 
found.

Tue Jun 29 09:19:46 2021

WARNING: cache 
read  
a corrupt block: group=2(DATA) dsk=0 blk=7 disk=0 (DATA_0000) incarn=3915953476 au=0 blk=7 count=1

Errors 
in 
file 
/u01/app/grid/diag/asm/
+asm/+ASM1
/trace/
+ASM1_arb0_25176.trc:

ORA-15196: invalid ASM block header [kfc.c:26368] [check_kfbh] [2147483648] [7] [2183628676 != 686982479]

NOTE: a corrupted block from group DATA was dumped to 
/u01/app/grid/diag/asm/
+asm/+ASM1
/trace/
+ASM1_arb0_25176.trc

Errors 
in 
file 
/u01/app/grid/diag/asm/
+asm/+ASM1
/trace/
+ASM1_arb0_25176.trc:

ORA-15196: invalid ASM block header [kfc.c:26368] [check_kfbh] [2147483648] [7] [2183628676 != 686982479]

ORA-15196: invalid ASM block header [kfc.c:26368] [check_kfbh] [2147483648] [7] [2183628676 != 686982479]

ERROR: cache failed to 
read 
group=2(DATA) dsk=0 blk=7 from disk(s): 0(DATA_0000)

ORA-15196: invalid ASM block header [kfc.c:26368] [check_kfbh] [2147483648] [7] [2183628676 != 686982479]

ORA-15196: invalid ASM block header [kfc.c:26368] [check_kfbh] [2147483648] [7] [2183628676 != 686982479]

NOTE: cache initiating offline of disk 0 group DATA

NOTE: process _arb0_+asm1 (25176) initiating offline of disk 0.3915953476 (DATA_0000) with mask 0x7e 
in 
group 2

NOTE: initiating PST update: grp = 2, dsk = 0
/0xe968b544
, mask = 0x6a, 
op 
= 
clear

Tue Jun 29 09:19:46 2021

GMON updating disk modes 
for 
group 2 at 10 
for 
pid 33, osid 25176

ERROR: Disk 0 cannot be offlined, since diskgroup has external redundancy.

ERROR: too many offline disks 
in 
PST (grp 2)

Tue Jun 29 09:19:46 2021

NOTE: cache dismounting (not clean) group 2
/0xB0C845CE 
(DATA) 

NOTE: messaging CKPT to quiesce pins Unix process pid: 25395, image: oracle@frsrac1 (B000)

Tue Jun 29 09:19:46 2021

NOTE: halting all I
/Os 
to diskgroup 2 (DATA)

Tue Jun 29 09:19:46 2021

NOTE: LGWR doing non-clean dismount of group 2 (DATA)

NOTE: LGWR 
sync 
ABA=11.10715 last written ABA 11.10715

WARNING: Offline 
for 
disk DATA_0000 
in 
mode 0x7f failed.

Errors 
in 
file 
/u01/app/grid/diag/asm/
+asm/+ASM1
/trace/
+ASM1_arb0_25176.trc  (incident=54665):

ORA-15335: ASM metadata corruption detected 
in 
disk group 
'DATA'

ORA-15130: diskgroup 
"DATA" 
is being dismounted

ORA-15066: offlining disk 
"DATA_0000" 
in 
group 
"DATA" 
may result 
in 
a data loss

ORA-15196: invalid ASM block header [kfc.c:26368] [check_kfbh] [2147483648] [7] [2183628676 != 686982479]

ORA-15196: invalid ASM block header [kfc.c:26368] [check_kfbh] [2147483648] [7] [2183628676 != 686982479]

Incident details 
in
: 
/u01/app/grid/diag/asm/
+asm/+ASM1
/incident/incdir_54665/
+ASM1_arb0_25176_i54665.trc

Tue Jun 29 09:19:46 2021

kjbdomdet send to inst 2

detach from dom 2, sending detach message to inst 2

Tue Jun 29 09:19:46 2021

List of instances:

 
1 2

Dirty detach reconfiguration started (new ddet inc 1, cluster inc 24)

 
Global Resource Directory partially frozen 
for 
dirty detach

* dirty detach - domain 2 invalid = TRUE 

 
796 GCS resources traversed, 0 cancelled

Dirty Detach Reconfiguration complete

Tue Jun 29 09:19:46 2021

WARNING: dirty detached from domain 2

NOTE: cache dismounted group 2
/0xB0C845CE 
(DATA) 

SQL> alter diskgroup DATA dismount force /* ASM SERVER:2965915086 */ 

Tue Jun 29 09:19:47 2021

ERROR: ORA-15130 thrown 
in 
ARB0 
for 
group number 2

Errors 
in 
file 
/u01/app/grid/diag/asm/
+asm/+ASM1
/trace/
+ASM1_arb0_25176.trc:

ORA-15130: diskgroup 
"DATA" 
is being dismounted

ORA-15335: ASM metadata corruption detected 
in 
disk group 
'DATA'

ORA-15130: diskgroup 
"DATA" 
is being dismounted

ORA-15066: offlining disk 
"DATA_0000" 
in 
group 
"DATA" 
may result 
in 
a data loss

ORA-15196: invalid ASM block header [kfc.c:26368] [check_kfbh] [2147483648] [7] [2183628676 != 686982479]

ORA-15196: invalid ASM block header [kfc.c:26368] [check_kfbh] [2147483648] [7] [2183628676 != 686982479]

Tue Jun 29 09:19:47 2021

NOTE: stopping process ARB0

Tue Jun 29 09:19:47 2021

Sweep [inc][54665]: completed

Tue Jun 29 09:19:47 2021

Sweep [inc2][54665]: completed

NOTE: cache deleting context 
for 
group DATA 2
/0xb0c845ce

Errors 
in 
file 
/u01/app/grid/diag/asm/
+asm/+ASM1
/trace/
+ASM1_rbal_3852.trc:

ORA-15130: diskgroup 
"DATA" 
is being dismounted

GMON dismounting group 2 at 11 
for 
pid 27, osid 25395

NOTE: Disk DATA_0000 
in 
mode 0x7f marked 
for 
de-assignment

NOTE: Disk DATA_0001 
in 
mode 0x7f marked 
for 
de-assignment

SUCCESS: diskgroup DATA was dismounted

SUCCESS: alter diskgroup DATA dismount force /* ASM SERVER:2965915086 */

通过kfed分析报错block,确认错误

kfbh.endian:                          1 ; 0x000: 0x01

kfbh.hard:                          130 ; 0x001: 0x82

kfbh.
type
:                            3 ; 0x002: KFBTYP_ALLOCTBL

kfbh.datfmt:                          2 ; 0x003: 0x02

kfbh.block.blk:                       7 ; 0x004: blk=7

kfbh.block.obj:              2147483648 ; 0x008: disk=0

kfbh.check:                  2183628676 ; 0x00c: 0x82278784 <<======该值错误,应该为:686982479

kfbh.fcn.base:                     3430 ; 0x010: 0x00000d66

kfbh.fcn.wrap:                        0 ; 0x014: 0x00000000

kfbh.spare1:                          0 ; 0x018: 0x00000000

kfbh.spare2:                          0 ; 0x01c: 0x00000000

kfdatb.aunum:                      2240 ; 0x000: 0x000008c0

kfdatb.shrink:                      448 ; 0x004: 0x01c0

kfdatb.ub2pad:                        0 ; 0x006: 0x0000

通过修复该错误,并且禁止reblance操作[增加磁盘数据需要重新分布],mount磁盘组,然后open库,发现redo已经被覆盖(非归档),强制打开库报错

SQL> alter database 
open 
resetlogs;

alter database 
open 
resetlogs

*

ERROR at line 1:

ORA-00603: ORACLE server session terminated by fatal error

ORA-00600: internal error code, arguments: [2662], [0], [2691201882], [0],

[2691227745], [12583040], [], [], [], [], [], []

ORA-00600: internal error code, arguments: [2662], [0], [2691201881], [0],

[2691227745], [12583040], [], [], [], [], [], []

ORA-01092: ORACLE instance terminated. Disconnection forced

ORA-00600: internal error code, arguments: [2662], [0], [2691201879], [0],

[2691227745], [12583040], [], [], [], [], [], []

Process ID: 25110

Session ID: 287 Serial number: 3

通过对scn进行处理,数据库顺利open

SQL> startup 
mount 
pfile=
'/tmp/pfile'
;

ORACLE instance started.
 

Total System Global Area 5044088832 bytes

Fixed Size          2261928 bytes

Variable Size        1442843736 bytes

Database Buffers     3590324224 bytes

Redo Buffers            8658944 bytes

Database mounted.

SQL> alter database 
open
;
 

Database altered.

ITPub博客

ORA-15335: ASM metadata corruption detected in disk group ‘DATA’

最新文章