故障现象
客户反馈传输进程abended，查看日志发现是trail文件中的事务记录损坏，日志如下：
“Incompatible record”是指遇到不完整的记录，记录损坏，对于这类错误没有简单快速就能恢复的方法，处理不当还可能丢失记录，需要重新从源端同步以保持数据一致。
2025-05-13T08:13:30.516+0800 ERROR OGG-02171 Oracle GoldenGate Capture for Oracle, p2_md.prm: Error reading LCR from data source. Status 509, data source type 0. 2025-05-13T08:13:30.517+0800 ERROR OGG-02191 Oracle GoldenGate Capture for Oracle, p2_md.prm: Incompatible record 101 in /exptbk/odc/oggapp/dirdat/ms000006546, rba 483,188,832 when getting trail header. 2025-05-13T08:13:30.517+0800 ERROR OGG-01668 Oracle GoldenGate Capture for Oracle, p2_md.prm: PROCESS ABENDING
处理思路
情况一：如果源trail文件是好的，即trail文件是由传输进程写入的，只是目标端的trail文件损坏，导致应用进程异常，则可以停止传输和应用进程，并更改传输进程到失败点之前，然后重新发送trail文件，供应用进程使用；
情况二：如果源trail文件损坏，即trail文件是由挖掘进程写入的，导致传输进程读取并发送是异常，则可以跳过该错误事务并从下一个正常的事务开始发送，或者停止挖掘进程并找到错误点之前的scn号，更改挖掘进程到失败点之前，重新挖掘，传输进程从重挖之后的trail文件开始发送。
本次故障，我们属于情况二，即源trail文件已经损坏，所以对于情况一的处理方法，我们这里就简单讲下步骤，主要对情况二做详细描述及处理。
情况一处理方法
1.停止传输和应用进程
2.记下应用进程的上次应用的 SCN 和时间戳;
3.更改传输进程点，用于重新读取该 SCN 后的数据并发送，启动传输进程（可以往前多指定点也没关系）;
4.更改应用进程以上次记录的scn后开始应用，启动应用进程；
1.停止传输和应用进程。
2.删除或重命名损坏文件开始及之后的所有trail文件。
3.在源中启动 传输进程，传输进程会自动重建丢失的trail文件。
4.启动应用进程。
下面我们具体讲下本次故障处理方法（即情况二）。
前面我们也大致提到针对这种情况有两种处理方式，一种是定位到损坏点之前的scn号，并重新挖掘和传输，难点在于这个scn号的定位；另一种是直接跳过损坏的事务从下一个正常事务开始做传输，这个方法虽然能快速恢复进程，但问题是会造成少量事务不一致，为保证数据一致，后续需要做比对修复或者重新初始化。这次我们为确保数据不丢失，选择重新挖掘。
1.定位损坏点scn号
需要借助logdump工具
$ ./logdump Logdump 1 >log to 0513.txt --- Session log 0513.txt opened 2025/05/13 08:19:08.827.375 --- Logdump 2 >open /exptbk/odc/oggapp/dirdat/ms000006546 Current LogTrail is /exptbk/odc/oggapp/dirdat/ms000006546 Logdump 3 >pos 483188832 --错误日志中报错的位置 Bad record found at RBA 966377664, format 5.50 Header token) 0100 1080 | .... Reading forward from RBA 483188832 | .... Logdump 4 >sfh prev 2025/05/12 17:00:18.000.000 FieldComp Len 353 RBA 483188399 Name: SHSMCT.PS_EXCHANGE_T_LOGS (TDR Index: 36) After Image: Partition 12 G e 0000 0500 0000 0100 4c01 0016 0000 0012 00c8 a1cf | ........L........... fbd7 aab3 a1cc e1cf e4bc c6bb aecf e402 000a 0000 | .................... 0000 0000 0004 d727 8d03 0016 0000 0012 00c8 a1cf | .......'............ fbd7 aab3 a1cc e1cf e4bc c6bb aecf e404 0005 0000 | .................... 0001 0059 0500 0900 0000 0500 3138 3135 3206 0008 | ...Y........18152... 0000 0004 0053 4c57 4707 0005 0000 0001 004d 0800 | .....SLWG........M.. 1500 0000 3230 3235 2d30 352d 3132 3a31 373a 3030 | ....2025-05-12:17:00 Logdump 6 >count ** Count begins at RBA 483188399 Bad record found at RBA 483188832, format 5.50 Header token) 0100 1080 | .... LogTrail /exptbk/odc/oggapp/dirdat/ms000006546 has 1 records Total Data Bytes 353 Avg Bytes/Record 353 FieldComp 1 After Images 1 Average of 1 Transactions Bytes/Trans ..... 401 Records/Trans ... 1 Files/Trans ..... 1
重新定位
Logdump 22 >pos 483188832 Reading forward from RBA 483188832 Logdump 23 >sfh prev ___________________________________________________________________ Hdr-Ind : E (x45) Partition : . (x0c) UndoFlag : . (x00) BeforeAfter: A (x41) RecLength : 353 (x0161) IO Time : 2025/05/12 17:00:18.000.000 IOType : 15 (x0f) OrigNode : 255 (xff) TransInd : . (x02) FormatType : R (x52) SyskeyLen : 0 (x00) Incomplete : . (x00) AuditRBA : 98701 AuditPos : 29463068 Continued : N (x00) RecCount : 1 (x01) 2025/05/12 17:00:18.000.000 FieldComp Len 353 RBA 483188399 Name: SHSMCT.PS_EXCHANGE_T_LOGS (TDR Index: 36) After Image: Partition 12 G e 0000 0500 0000 0100 4c01 0016 0000 0012 00c8 a1cf | ........L........... fbd7 aab3 a1cc e1cf e4bc c6bb aecf e402 000a 0000 | .................... 0000 0000 0004 d727 8d03 0016 0000 0012 00c8 a1cf | .......'............ fbd7 aab3 a1cc e1cf e4bc c6bb aecf e404 0005 0000 | .................... 0001 0059 0500 0900 0000 0500 3138 3135 3206 0008 | ...Y........18152... 0000 0004 0053 4c57 4707 0005 0000 0001 004d 0800 | .....SLWG........M.. 1500 0000 3230 3235 2d30 352d 3132 3a31 373a 3030 | ....2025-05-12:17:00 3a31 3809 0004 00ff ff00 000a 0005 0000 0001 0033 | :18................3 0b00 0a00 0000 0600 5348 534d 4354 0c00 0a00 ffff | ........SHSMCT...... 0000 0000 0000 0000 0d00 0400 ffff 0000 0e00 0400 | .................... ffff 0000 0f00 0400 ffff 0000 1000 0700 0000 0300 | .................... 4e4f 5711 0015 00ff ff31 3930 302d 3031 2d30 313a | NOW......1900-01-01: 3030 3a30 303a 3030 1200 0a00 ffff 0000 0000 0000 | 00:00:00............ 0000 1300 0a00 0000 0000 0000 1251 3752 1400 0400 | .............Q7R.... ffff 0000 1500 0a00 ffff 0000 0000 0000 0000 1600 | .................... 0400 ffff 0000 1700 1500 ffff 3139 3030 2d30 312d | ............1900-01- 3031 3a30 303a 3030 3a30 3018 000a 00ff ff00 0000 | 01:00:00:00......... 0000 0000 0019 0004 00ff ff00 00 | ............. Column 0 (x0000), Len 5 (x0005) 0000 0100 4c | ....L Column 1 (x0001), Len 22 (x0016) 0000 1200 c8a1 cffb d7aa b3a1 cce1 cfe4 bcc6 bbae | .................... cfe4 | .. Column 2 (x0002), Len 10 (x000a) 0000 0000 0000 04d7 278d | ........'. Column 3 (x0003), Len 22 (x0016) 0000 1200 c8a1 cffb d7aa b3a1 cce1 cfe4 bcc6 bbae | .................... cfe4 | .. Column 4 (x0004), Len 5 (x0005) 0000 0100 59 | ....Y Column 5 (x0005), Len 9 (x0009) 0000 0500 3138 3135 32 | ....18152 Column 6 (x0006), Len 8 (x0008) 0000 0400 534c 5747 | ....SLWG Column 7 (x0007), Len 5 (x0005) 0000 0100 4d | ....M Column 8 (x0008), Len 21 (x0015) 0000 3230 3235 2d30 352d 3132 3a31 373a 3030 3a31 | ..2025-05-12:17:00:1 38 | 8 Column 9 (x0009), Len 4 (x0004) ffff 0000 | .... Column 10 (x000a), Len 5 (x0005) 0000 0100 33 | ....3 Column 11 (x000b), Len 10 (x000a) 0000 0600 5348 534d 4354 | ....SHSMCT Column 12 (x000c), Len 10 (x000a) ffff 0000 0000 0000 0000 | .......... Column 13 (x000d), Len 4 (x0004) ffff 0000 | .... Column 14 (x000e), Len 4 (x0004) ffff 0000 | .... Column 15 (x000f), Len 4 (x0004) ffff 0000 | .... Column 16 (x0010), Len 7 (x0007) 0000 0300 4e4f 57 | ....NOW Column 17 (x0011), Len 21 (x0015) ffff 3139 3030 2d30 312d 3031 3a30 303a 3030 3a30 | ..1900-01-01:00:00:0 30 | 0 Column 18 (x0012), Len 10 (x000a) ffff 0000 0000 0000 0000 | .......... Column 19 (x0013), Len 10 (x000a) 0000 0000 0000 1251 3752 | .......Q7R Column 20 (x0014), Len 4 (x0004) ffff 0000 | .... Column 21 (x0015), Len 10 (x000a) ffff 0000 0000 0000 0000 | .......... Column 22 (x0016), Len 4 (x0004) ffff 0000 | .... Column 23 (x0017), Len 21 (x0015) ffff 3139 3030 2d30 312d 3031 3a30 303a 3030 3a30 | ..1900-01-01:00:00:0 30 | 0 Column 24 (x0018), Len 10 (x000a) ffff 0000 0000 0000 0000 | .......... Column 25 (x0019), Len 4 (x0004) ffff 0000 | .... GGS tokens: TokenID x52 'R' ORAROWID Info x00 Length 20 4141 4164 546a 4142 2f41 4142 494d 4641 4133 0001 | AAAdTjAB/AABIMFAA3.. Logdump 24 >pos 483188832 Bad record found at RBA 483188832, format 5.50 Header token) 0100 1080 | .... Reading forward from RBA 483188832 Logdump 25 >sfh ___________________________________________________________________ Hdr-Ind : E (x45) Partition : . (x0c) UndoFlag : . (x00) BeforeAfter: A (x41) RecLength : 375 (x0177) IO Time : 2025/05/12 17:00:18.000.000 IOType : 5 (x05) OrigNode : 255 (xff) TransInd : . (x02) FormatType : R (x52) SyskeyLen : 0 (x00) Incomplete : . (x00) AuditRBA : 98701 AuditPos : 29558820 Continued : N (x00) RecCount : 1 (x01) 2025/05/12 17:00:18.000.000 Insert Len 375 RBA 483189336 Name: SHSMCT.UBK_BOOKING_CONTAINERS (TDR Index: 14) After Image: Partition 12 G e 0000 0a00 0000 0000 0000 0069 4cf9 0100 0a00 0000 | ...........iL....... 0000 0000 0069 4e67 0200 0600 0000 0200 5459 0300 | .....iNg........TY.. 0600 0000 0200 3131 0400 0a00 0000 0000 0000 0320 | ......11........... a71d 0500 0a00 0000 0000 0000 0000 0000 0600 0500 | .................... 0000 0100 5007 0004 00ff ff00 0008 0015 00ff ff31 | ....P..............1 3930 302d 3031 2d30 313a 3030 3a30 303a 3030 0900 | 900-01-01:00:00:00.. 1500 ffff 3139 3030 2d30 312d 3031 3a30 303a 3030 | ....1900-01-01:00:00 3a30 300a 0015 00ff ff31 3930 302d 3031 2d30 313a | :00......1900-01-01: 3030 3a30 303a 3030 0b00 0400 ffff 0000 0c00 0400 | 00:00:00............ ffff 0000 0d00 0700 0000 0300 4e4f 570e 0015 00ff | ............NOW..... ff31 3930 302d 3031 2d30 313a 3030 3a30 303a 3030 | .1900-01-01:00:00:00 0f00 1500 0000 3230 3235 2d30 352d 3132 3a31 373a | ......2025-05-12:17: 3030 3a31 3810 000d 0000 0009 0053 4950 472d 424f | 00:18........SIPG-BO 4f4b 1100 1500 0000 3230 3235 2d30 352d 3132 3a31 | OK......2025-05-12:1 373a 3030 3a31 3812 0004 00ff ff00 0013 0005 0000 | 7:00:18............. 0001 0046 1400 0f00 0000 0b00 5458 4755 3530 3036 | ...F........TXGU5006 3934 3615 0006 0000 0002 0047 5016 0006 0000 0002 | 946........GP....... 0034 3017 0004 00ff ff00 0018 0014 0000 0010 0032 | .40................2 3032 3530 3530 3832 3534 3833 3638 38 | 025050825483688 Column 0 (x0000), Len 10 (x000a) 0000 0000 0000 0069 4cf9 | .......iL. Column 1 (x0001), Len 10 (x000a) 0000 0000 0000 0069 4e67 | .......iNg Column 2 (x0002), Len 6 (x0006) 0000 0200 5459 | ....TY Column 3 (x0003), Len 6 (x0006) 0000 0200 3131 | ....11 Column 4 (x0004), Len 10 (x000a) 0000 0000 0000 0320 a71d | ....... .. Column 5 (x0005), Len 10 (x000a) 0000 0000 0000 0000 0000 | .......... Column 6 (x0006), Len 5 (x0005) 0000 0100 50 | ....P Column 7 (x0007), Len 4 (x0004) ffff 0000 | .... Column 8 (x0008), Len 21 (x0015) ffff 3139 3030 2d30 312d 3031 3a30 303a 3030 3a30 | ..1900-01-01:00:00:0 30 | 0 Column 9 (x0009), Len 21 (x0015) ffff 3139 3030 2d30 312d 3031 3a30 303a 3030 3a30 | ..1900-01-01:00:00:0 30 | 0 Column 10 (x000a), Len 21 (x0015) ffff 3139 3030 2d30 312d 3031 3a30 303a 3030 3a30 | ..1900-01-01:00:00:0 30 | 0 Column 11 (x000b), Len 4 (x0004) ffff 0000 | .... Column 12 (x000c), Len 4 (x0004) ffff 0000 | .... Column 13 (x000d), Len 7 (x0007) 0000 0300 4e4f 57 | ....NOW Column 14 (x000e), Len 21 (x0015) ffff 3139 3030 2d30 312d 3031 3a30 303a 3030 3a30 | ..1900-01-01:00:00:0 30 | 0 Column 15 (x000f), Len 21 (x0015) 0000 3230 3235 2d30 352d 3132 3a31 373a 3030 3a31 | ..2025-05-12:17:00:1 38 | 8 Column 16 (x0010), Len 13 (x000d) 0000 0900 5349 5047 2d42 4f4f 4b | ....SIPG-BOOK Column 17 (x0011), Len 21 (x0015) 0000 3230 3235 2d30 352d 3132 3a31 373a 3030 3a31 | ..2025-05-12:17:00:1 38 | 8 Column 18 (x0012), Len 4 (x0004) ffff 0000 | .... Column 19 (x0013), Len 5 (x0005) 0000 0100 46 | ....F Column 20 (x0014), Len 15 (x000f) 0000 0b00 5458 4755 3530 3036 3934 36 | ....TXGU5006946 Column 21 (x0015), Len 6 (x0006) 0000 0200 4750 | ....GP Column 22 (x0016), Len 6 (x0006) 0000 0200 3430 | ....40 Column 23 (x0017), Len 4 (x0004) ffff 0000 | .... Column 24 (x0018), Len 20 (x0014) 0000 1000 3230 3235 3035 3038 3235 3438 3336 3838 | ....2025050825483688 GGS tokens: TokenID x52 'R' ORAROWID Info x00 Length 20 4141 4164 4d31 4141 3441 4141 4739 5a41 4169 0001 | AAAdM1AA4AAAG9ZAAi..可以看到使用sfh查看到前后记录都是正常的。
补充：
TransInd : . (x00) First statement in transaction 事务的第一条记录，即开始
TransInd : . (x01) Statement in middle of transaction 事务的中间记录
Translnd : .(x02) last statement in transaction 事务的最后一条记录，即结束
Translnd : . (x03) Sole statement in transaction 该事务只有一条记录
Logdump 37 >pos 483187000 Reading forward from RBA 483187000 Logdump 38 >sfh ___________________________________________________________________ Hdr-Ind : E (x45) Partition : . (x0c) UndoFlag : . (x00) BeforeAfter: B (x42) RecLength : 353 (x0161) IO Time : 2025/05/12 17:00:18.000.000 IOType : 15 (x0f) OrigNode : 255 (xff) TransInd : . (x00) FormatType : R (x52) SyskeyLen : 0 (x00) Incomplete : . (x00) AuditRBA : 98701 AuditPos : 29458924 Continued : N (x00) RecCount : 1 (x01) 2025/05/12 17:00:18.000.000 FieldComp Len 353 RBA 483187017 Name: SHSMCT.PS_EXCHANGE_T_LOGS (TDR Index: 36) Before Image: Partition 12 G b 0000 0500 0000 0100 4c01 0016 0000 0012 00c8 a1cf | ........L........... fbd7 aab3 a1cc e1cf e4bc c6bb aecf e402 000a 0000 | .................... 0000 0000 0004 d727 8d03 0016 0000 0012 00c8 a1cf | .......'............ fbd7 aab3 a1cc e1cf e4bc c6bb aecf e404 0005 0000 | .................... 0001 0059 0500 0900 0000 0500 3138 3135 3206 0008 | ...Y........18152... 0000 0004 0053 4c57 4707 0005 0000 0001 004d 0800 | .....SLWG........M.. 1500 0000 3230 3235 2d30 352d 3132 3a31 373a 3030 | ....2025-05-12:17:00 3a31 3109 0004 00ff ff00 000a 0005 0000 0001 0031 | :11................1 0b00 0a00 0000 0600 5348 534d 4354 0c00 0a00 ffff | ........SHSMCT...... 0000 0000 0000 0000 0d00 0400 ffff 0000 0e00 0400 | .................... ffff 0000 0f00 0400 ffff 0000 1000 0700 0000 0300 | .................... 4e4f 5711 0015 00ff ff31 3930 302d 3031 2d30 313a | NOW......1900-01-01: 3030 3a30 303a 3030 1200 0a00 ffff 0000 0000 0000 | 00:00:00............ 0000 1300 0a00 0000 0000 0000 1251 3752 1400 0400 | .............Q7R.... ffff 0000 1500 0a00 ffff 0000 0000 0000 0000 1600 | .................... 0400 ffff 0000 1700 1500 ffff 3139 3030 2d30 312d | ............1900-01- 3031 3a30 303a 3030 3a30 3018 000a 00ff ff00 0000 | 01:00:00:00......... 0000 0000 0019 0004 00ff ff00 00 | ............. Column 0 (x0000), Len 5 (x0005) 0000 0100 4c | ....L Column 1 (x0001), Len 22 (x0016) 0000 1200 c8a1 cffb d7aa b3a1 cce1 cfe4 bcc6 bbae | .................... cfe4 | .. Column 2 (x0002), Len 10 (x000a) 0000 0000 0000 04d7 278d | ........'. Column 3 (x0003), Len 22 (x0016) 0000 1200 c8a1 cffb d7aa b3a1 cce1 cfe4 bcc6 bbae | .................... cfe4 | .. Column 4 (x0004), Len 5 (x0005) 0000 0100 59 | ....Y Column 5 (x0005), Len 9 (x0009) 0000 0500 3138 3135 32 | ....18152 Column 6 (x0006), Len 8 (x0008) 0000 0400 534c 5747 | ....SLWG Column 7 (x0007), Len 5 (x0005) 0000 0100 4d | ....M Column 8 (x0008), Len 21 (x0015) 0000 3230 3235 2d30 352d 3132 3a31 373a 3030 3a31 | ..2025-05-12:17:00:1 31 | 1 Column 9 (x0009), Len 4 (x0004) ffff 0000 | .... Column 10 (x000a), Len 5 (x0005) 0000 0100 31 | ....1 Column 11 (x000b), Len 10 (x000a) 0000 0600 5348 534d 4354 | ....SHSMCT Column 12 (x000c), Len 10 (x000a) ffff 0000 0000 0000 0000 | .......... Column 13 (x000d), Len 4 (x0004) ffff 0000 | .... Column 14 (x000e), Len 4 (x0004) ffff 0000 | .... Column 15 (x000f), Len 4 (x0004) ffff 0000 | .... Column 16 (x0010), Len 7 (x0007) 0000 0300 4e4f 57 | ....NOW Column 17 (x0011), Len 21 (x0015) ffff 3139 3030 2d30 312d 3031 3a30 303a 3030 3a30 | ..1900-01-01:00:00:0 30 | 0 Column 18 (x0012), Len 10 (x000a) ffff 0000 0000 0000 0000 | .......... Column 19 (x0013), Len 10 (x000a) 0000 0000 0000 1251 3752 | .......Q7R Column 20 (x0014), Len 4 (x0004) ffff 0000 | .... Column 21 (x0015), Len 10 (x000a) ffff 0000 0000 0000 0000 | .......... Column 22 (x0016), Len 4 (x0004) ffff 0000 | .... Column 23 (x0017), Len 21 (x0015) ffff 3139 3030 2d30 312d 3031 3a30 303a 3030 3a30 | ..1900-01-01:00:00:0 30 | 0 Column 24 (x0018), Len 10 (x000a) ffff 0000 0000 0000 0000 | .......... Column 25 (x0019), Len 4 (x0004) ffff 0000 | .... GGS tokens: TokenID x52 'R' ORAROWID Info x00 Length 20 4141 4164 546a 4142 2f41 4142 494d 4641 4133 0001 | AAAdTjAB/AABIMFAA3.. TokenID x4c 'L' LOGCSN Info x00 Length 14 3131 3235 3434 3834 3339 3838 3233 | 11254484398823 TokenID x36 '6' TRANID Info x00 Length 13 302e 382e 392e 3930 3338 3832 38 | 0.8.9.9038828 TokenID x69 'i' ORATHREADID Info x01 Length 2 0001 | .. Logdump 39 >SCANFORENDTRANS End of Transaction found at RBA 483187491 ___________________________________________________________________ Hdr-Ind : E (x45) Partition : . (x0c) UndoFlag : . (x00) BeforeAfter: A (x41) RecLength : 353 (x0161) IO Time : 2025/05/12 17:00:18.000.000 IOType : 15 (x0f) OrigNode : 255 (xff) TransInd : . (x02) FormatType : R (x52) SyskeyLen : 0 (x00) Incomplete : . (x00) AuditRBA : 98701 AuditPos : 29458924 Continued : N (x00) RecCount : 1 (x01) 2025/05/12 17:00:18.000.000 FieldComp Len 353 RBA 483187491 Name: SHSMCT.PS_EXCHANGE_T_LOGS (TDR Index: 36) After Image: Partition 12 G e 0000 0500 0000 0100 4c01 0016 0000 0012 00c8 a1cf | ........L........... fbd7 aab3 a1cc e1cf e4bc c6bb aecf e402 000a 0000 | .................... 0000 0000 0004 d727 8d03 0016 0000 0012 00c8 a1cf | .......'............ fbd7 aab3 a1cc e1cf e4bc c6bb aecf e404 0005 0000 | .................... 0001 0059 0500 0900 0000 0500 3138 3135 3206 0008 | ...Y........18152... 0000 0004 0053 4c57 4707 0005 0000 0001 004d 0800 | .....SLWG........M.. 1500 0000 3230 3235 2d30 352d 3132 3a31 373a 3030 | ....2025-05-12:17:00 3a31 3809 0004 00ff ff00 000a 0005 0000 0001 0032 | :18................2 0b00 0a00 0000 0600 5348 534d 4354 0c00 0a00 ffff | ........SHSMCT...... 0000 0000 0000 0000 0d00 0400 ffff 0000 0e00 0400 | .................... ffff 0000 0f00 0400 ffff 0000 1000 0700 0000 0300 | .................... 4e4f 5711 0015 00ff ff31 3930 302d 3031 2d30 313a | NOW......1900-01-01: 3030 3a30 303a 3030 1200 0a00 ffff 0000 0000 0000 | 00:00:00............ 0000 1300 0a00 0000 0000 0000 1251 3752 1400 0400 | .............Q7R.... ffff 0000 1500 0a00 ffff 0000 0000 0000 0000 1600 | .................... 0400 ffff 0000 1700 1500 ffff 3139 3030 2d30 312d | ............1900-01- 3031 3a30 303a 3030 3a30 3018 000a 00ff ff00 0000 | 01:00:00:00......... 0000 0000 0019 0004 00ff ff00 00 | ............. Column 0 (x0000), Len 5 (x0005) 0000 0100 4c | ....L Column 1 (x0001), Len 22 (x0016) 0000 1200 c8a1 cffb d7aa b3a1 cce1 cfe4 bcc6 bbae | .................... cfe4 | .. Column 2 (x0002), Len 10 (x000a) 0000 0000 0000 04d7 278d | ........'. Column 3 (x0003), Len 22 (x0016) 0000 1200 c8a1 cffb d7aa b3a1 cce1 cfe4 bcc6 bbae | .................... cfe4 | .. Column 4 (x0004), Len 5 (x0005) 0000 0100 59 | ....Y Column 5 (x0005), Len 9 (x0009) 0000 0500 3138 3135 32 | ....18152 Column 6 (x0006), Len 8 (x0008) 0000 0400 534c 5747 | ....SLWG Column 7 (x0007), Len 5 (x0005) 0000 0100 4d | ....M Column 8 (x0008), Len 21 (x0015) 0000 3230 3235 2d30 352d 3132 3a31 373a 3030 3a31 | ..2025-05-12:17:00:1 38 | 8 Column 9 (x0009), Len 4 (x0004) ffff 0000 | .... Column 10 (x000a), Len 5 (x0005) 0000 0100 32 | ....2 Column 11 (x000b), Len 10 (x000a) 0000 0600 5348 534d 4354 | ....SHSMCT Column 12 (x000c), Len 10 (x000a) ffff 0000 0000 0000 0000 | .......... Column 13 (x000d), Len 4 (x0004) ffff 0000 | .... Column 14 (x000e), Len 4 (x0004) ffff 0000 | .... Column 15 (x000f), Len 4 (x0004) ffff 0000 | .... Column 16 (x0010), Len 7 (x0007) 0000 0300 4e4f 57 | ....NOW Column 17 (x0011), Len 21 (x0015) ffff 3139 3030 2d30 312d 3031 3a30 303a 3030 3a30 | ..1900-01-01:00:00:0 30 | 0 Column 18 (x0012), Len 10 (x000a) ffff 0000 0000 0000 0000 | .......... Column 19 (x0013), Len 10 (x000a) 0000 0000 0000 1251 3752 | .......Q7R Column 20 (x0014), Len 4 (x0004) ffff 0000 | .... Column 21 (x0015), Len 10 (x000a) ffff 0000 0000 0000 0000 | .......... Column 22 (x0016), Len 4 (x0004) ffff 0000 | .... Column 23 (x0017), Len 21 (x0015) ffff 3139 3030 2d30 312d 3031 3a30 303a 3030 3a30 | ..1900-01-01:00:00:0 30 | 0 Column 24 (x0018), Len 10 (x000a) ffff 0000 0000 0000 0000 | .......... Column 25 (x0019), Len 4 (x0004) ffff 0000 | .... GGS tokens: TokenID x52 'R' ORAROWID Info x00 Length 20 4141 4164 546a 4142 2f41 4142 494d 4641 4133 0001 | AAAdTjAB/AABIMFAA3.. Logdump 40 >n --查下一条记录 ___________________________________________________________________ Hdr-Ind : E (x45) Partition : . (x0c) UndoFlag : . (x00) BeforeAfter: B (x42) RecLength : 353 (x0161) IO Time : 2025/05/12 17:00:18.000.000 IOType : 15 (x0f) OrigNode : 255 (xff) TransInd : . (x00) FormatType : R (x52) SyskeyLen : 0 (x00) Incomplete : . (x00) AuditRBA : 98701 AuditPos : 29463068 Continued : N (x00) RecCount : 1 (x01) 2025/05/12 17:00:18.000.000 FieldComp Len 353 RBA 483187924 Name: SHSMCT.PS_EXCHANGE_T_LOGS (TDR Index: 36) Before Image: Partition 12 G b 0000 0500 0000 0100 4c01 0016 0000 0012 00c8 a1cf | ........L........... fbd7 aab3 a1cc e1cf e4bc c6bb aecf e402 000a 0000 | .................... 0000 0000 0004 d727 8d03 0016 0000 0012 00c8 a1cf | .......'............ fbd7 aab3 a1cc e1cf e4bc c6bb aecf e404 0005 0000 | .................... 0001 0059 0500 0900 0000 0500 3138 3135 3206 0008 | ...Y........18152... 0000 0004 0053 4c57 4707 0005 0000 0001 004d 0800 | .....SLWG........M.. 1500 0000 3230 3235 2d30 352d 3132 3a31 373a 3030 | ....2025-05-12:17:00 3a31 3809 0004 00ff ff00 000a 0005 0000 0001 0032 | :18................2 0b00 0a00 0000 0600 5348 534d 4354 0c00 0a00 ffff | ........SHSMCT...... 0000 0000 0000 0000 0d00 0400 ffff 0000 0e00 0400 | .................... ffff 0000 0f00 0400 ffff 0000 1000 0700 0000 0300 | .................... 4e4f 5711 0015 00ff ff31 3930 302d 3031 2d30 313a | NOW......1900-01-01: 3030 3a30 303a 3030 1200 0a00 ffff 0000 0000 0000 | 00:00:00............ 0000 1300 0a00 0000 0000 0000 1251 3752 1400 0400 | .............Q7R.... ffff 0000 1500 0a00 ffff 0000 0000 0000 0000 1600 | .................... 0400 ffff 0000 1700 1500 ffff 3139 3030 2d30 312d | ............1900-01- 3031 3a30 303a 3030 3a30 3018 000a 00ff ff00 0000 | 01:00:00:00......... 0000 0000 0019 0004 00ff ff00 00 | ............. Column 0 (x0000), Len 5 (x0005) 0000 0100 4c | ....L Column 1 (x0001), Len 22 (x0016) 0000 1200 c8a1 cffb d7aa b3a1 cce1 cfe4 bcc6 bbae | .................... cfe4 | .. Column 2 (x0002), Len 10 (x000a) 0000 0000 0000 04d7 278d | ........'. Column 3 (x0003), Len 22 (x0016) 0000 1200 c8a1 cffb d7aa b3a1 cce1 cfe4 bcc6 bbae | .................... cfe4 | .. Column 4 (x0004), Len 5 (x0005) 0000 0100 59 | ....Y Column 5 (x0005), Len 9 (x0009) 0000 0500 3138 3135 32 | ....18152 Column 6 (x0006), Len 8 (x0008) 0000 0400 534c 5747 | ....SLWG Column 7 (x0007), Len 5 (x0005) 0000 0100 4d | ....M Column 8 (x0008), Len 21 (x0015) 0000 3230 3235 2d30 352d 3132 3a31 373a 3030 3a31 | ..2025-05-12:17:00:1 38 | 8 Column 9 (x0009), Len 4 (x0004) ffff 0000 | .... Column 10 (x000a), Len 5 (x0005) 0000 0100 32 | ....2 Column 11 (x000b), Len 10 (x000a) 0000 0600 5348 534d 4354 | ....SHSMCT Column 12 (x000c), Len 10 (x000a) ffff 0000 0000 0000 0000 | .......... Column 13 (x000d), Len 4 (x0004) ffff 0000 | .... Column 14 (x000e), Len 4 (x0004) ffff 0000 | .... Column 15 (x000f), Len 4 (x0004) ffff 0000 | .... Column 16 (x0010), Len 7 (x0007) 0000 0300 4e4f 57 | ....NOW Column 17 (x0011), Len 21 (x0015) ffff 3139 3030 2d30 312d 3031 3a30 303a 3030 3a30 | ..1900-01-01:00:00:0 30 | 0 Column 18 (x0012), Len 10 (x000a) ffff 0000 0000 0000 0000 | .......... Column 19 (x0013), Len 10 (x000a) 0000 0000 0000 1251 3752 | .......Q7R Column 20 (x0014), Len 4 (x0004) ffff 0000 | .... Column 21 (x0015), Len 10 (x000a) ffff 0000 0000 0000 0000 | .......... Column 22 (x0016), Len 4 (x0004) ffff 0000 | .... Column 23 (x0017), Len 21 (x0015) ffff 3139 3030 2d30 312d 3031 3a30 303a 3030 3a30 | ..1900-01-01:00:00:0 30 | 0 Column 24 (x0018), Len 10 (x000a) ffff 0000 0000 0000 0000 | .......... Column 25 (x0019), Len 4 (x0004) ffff 0000 | .... GGS tokens: TokenID x52 'R' ORAROWID Info x00 Length 20 4141 4164 546a 4142 2f41 4142 494d 4641 4133 0001 | AAAdTjAB/AABIMFAA3.. TokenID x4c 'L' LOGCSN Info x00 Length 14 3131 3235 3434 3834 3339 3838 3235 | 11254484398825 ------上一个事务的scn TokenID x36 '6' TRANID Info x00 Length 14 302e 382e 3238 2e39 3033 3836 3633 | 0.8.28.9038663 TokenID x69 'i' ORATHREADID Info x01 Length 2 0001 | .. Logdump 41 >n ___________________________________________________________________ Hdr-Ind : E (x45) Partition : . (x0c) UndoFlag : . (x00) BeforeAfter: A (x41) RecLength : 353 (x0161) IO Time : 2025/05/12 17:00:18.000.000 IOType : 15 (x0f) OrigNode : 255 (xff) TransInd : . (x02) FormatType : R (x52) SyskeyLen : 0 (x00) Incomplete : . (x00) AuditRBA : 98701 AuditPos : 29463068 Continued : N (x00) RecCount : 1 (x01) 2025/05/12 17:00:18.000.000 FieldComp Len 353 RBA 483188399 Name: SHSMCT.PS_EXCHANGE_T_LOGS (TDR Index: 36) After Image: Partition 12 G e 0000 0500 0000 0100 4c01 0016 0000 0012 00c8 a1cf | ........L........... fbd7 aab3 a1cc e1cf e4bc c6bb aecf e402 000a 0000 | .................... 0000 0000 0004 d727 8d03 0016 0000 0012 00c8 a1cf | .......'............ fbd7 aab3 a1cc e1cf e4bc c6bb aecf e404 0005 0000 | .................... 0001 0059 0500 0900 0000 0500 3138 3135 3206 0008 | ...Y........18152... 0000 0004 0053 4c57 4707 0005 0000 0001 004d 0800 | .....SLWG........M.. 1500 0000 3230 3235 2d30 352d 3132 3a31 373a 3030 | ....2025-05-12:17:00 3a31 3809 0004 00ff ff00 000a 0005 0000 0001 0033 | :18................3 0b00 0a00 0000 0600 5348 534d 4354 0c00 0a00 ffff | ........SHSMCT...... 0000 0000 0000 0000 0d00 0400 ffff 0000 0e00 0400 | .................... ffff 0000 0f00 0400 ffff 0000 1000 0700 0000 0300 | .................... 4e4f 5711 0015 00ff ff31 3930 302d 3031 2d30 313a | NOW......1900-01-01: 3030 3a30 303a 3030 1200 0a00 ffff 0000 0000 0000 | 00:00:00............ 0000 1300 0a00 0000 0000 0000 1251 3752 1400 0400 | .............Q7R.... ffff 0000 1500 0a00 ffff 0000 0000 0000 0000 1600 | .................... 0400 ffff 0000 1700 1500 ffff 3139 3030 2d30 312d | ............1900-01- 3031 3a30 303a 3030 3a30 3018 000a 00ff ff00 0000 | 01:00:00:00......... 0000 0000 0019 0004 00ff ff00 00 | ............. Column 0 (x0000), Len 5 (x0005) 0000 0100 4c | ....L Column 1 (x0001), Len 22 (x0016) 0000 1200 c8a1 cffb d7aa b3a1 cce1 cfe4 bcc6 bbae | .................... cfe4 | .. Column 2 (x0002), Len 10 (x000a) 0000 0000 0000 04d7 278d | ........'. Column 3 (x0003), Len 22 (x0016) 0000 1200 c8a1 cffb d7aa b3a1 cce1 cfe4 bcc6 bbae | .................... cfe4 | .. Column 4 (x0004), Len 5 (x0005) 0000 0100 59 | ....Y Column 5 (x0005), Len 9 (x0009) 0000 0500 3138 3135 32 | ....18152 Column 6 (x0006), Len 8 (x0008) 0000 0400 534c 5747 | ....SLWG Column 7 (x0007), Len 5 (x0005) 0000 0100 4d | ....M Column 8 (x0008), Len 21 (x0015) 0000 3230 3235 2d30 352d 3132 3a31 373a 3030 3a31 | ..2025-05-12:17:00:1 38 | 8 Column 9 (x0009), Len 4 (x0004) ffff 0000 | .... Column 10 (x000a), Len 5 (x0005) 0000 0100 33 | ....3 Column 11 (x000b), Len 10 (x000a) 0000 0600 5348 534d 4354 | ....SHSMCT Column 12 (x000c), Len 10 (x000a) ffff 0000 0000 0000 0000 | .......... Column 13 (x000d), Len 4 (x0004) ffff 0000 | .... Column 14 (x000e), Len 4 (x0004) ffff 0000 | .... Column 15 (x000f), Len 4 (x0004) ffff 0000 | .... Column 16 (x0010), Len 7 (x0007) 0000 0300 4e4f 57 | ....NOW Column 17 (x0011), Len 21 (x0015) ffff 3139 3030 2d30 312d 3031 3a30 303a 3030 3a30 | ..1900-01-01:00:00:0 30 | 0 Column 18 (x0012), Len 10 (x000a) ffff 0000 0000 0000 0000 | .......... Column 19 (x0013), Len 10 (x000a) 0000 0000 0000 1251 3752 | .......Q7R Column 20 (x0014), Len 4 (x0004) ffff 0000 | .... Column 21 (x0015), Len 10 (x000a) ffff 0000 0000 0000 0000 | .......... Column 22 (x0016), Len 4 (x0004) ffff 0000 | .... Column 23 (x0017), Len 21 (x0015) ffff 3139 3030 2d30 312d 3031 3a30 303a 3030 3a30 | ..1900-01-01:00:00:0 30 | 0 Column 24 (x0018), Len 10 (x000a) ffff 0000 0000 0000 0000 | .......... Column 25 (x0019), Len 4 (x0004) ffff 0000 | .... GGS tokens: TokenID x52 'R' ORAROWID Info x00 Length 20 4141 4164 546a 4142 2f41 4142 494d 4641 4133 0001 | AAAdTjAB/AABIMFAA3.. Logdump 42 >n --查下一条记录即为损坏记录 Bad record found at RBA 483188832, format 5.50 Header token) 0100 1080找到上一个交易的的scn号为11254484398825，尝试修改抽取进程重新生成trail文件
GGSCI (dcdga02) 8> info *2* EXTRACT E2_MD Last Started 2025-05-12 17:46 Status STOPPED Checkpoint Lag 00:00:02 (updated 00:00:13 ago) Log Read Checkpoint Oracle Redo Logs 2025-05-13 08:49:37 Thread 1, Seqno 98735, RBA 1690595156 SCN 2620.3272730868 (11256087046388) Log Read Checkpoint Oracle Redo Logs 2025-05-13 08:49:38 Thread 2, Seqno 97940, RBA 145551360 SCN 2620.3272731403 (11256087046923) Log Read Checkpoint Oracle Redo Logs 2025-05-13 08:49:38 Thread 3, Seqno 97931, RBA 302209344 SCN 2620.3272731196 (11256087046716) EXTRACT P2_MD Last Started 2025-05-13 08:36 Status ABENDED Checkpoint Lag 00:00:00 (updated 00:12:21 ago) Log Read Checkpoint File /exptbk/odc/oggapp/dirdat/ms000006546 2024-04-27 01:17:40.522457 RBA 48318883
2.重新指定挖掘点
GGSCI (dcdga02) 26> alter e2_md ETROLLOVER 2025-05-13 08:51:53 INFO OGG-01520 Rollover performed. For each affected output trail of Version 10 or higher format, after starting the source extract, issue ALTER EXTSEQNO for that trail's reader (either pump EXTRACT or REPLICAT) to move the reader's scan to the new trail file; it will not happen automatically. EXTRACT altered. GGSCI (dcdga02) 27> alter E2_MD scn 11254484398825 EXTRACT altered. GGSCI (dcdga02) 28> info *2* EXTRACT E2_MD Initialized 2025-05-13 08:52 Status STOPPED Checkpoint Lag 00:00:00 (updated 00:00:08 ago) Log Read Checkpoint Oracle Redo Logs First Record Thread 1, Seqno 0, RBA 0 SCN 2620.1670083305 (11254484398825) Log Read Checkpoint Oracle Redo Logs First Record Thread 2, Seqno 0, RBA 0 SCN 2620.1670083305 (11254484398825) Log Read Checkpoint Oracle Redo Logs First Record Thread 3, Seqno 0, RBA 0 SCN 2620.1670083305 (11254484398825) EXTRACT P2_MD Last Started 2025-05-13 08:36 Status ABENDED Checkpoint Lag 00:00:00 (updated 00:15:35 ago) Log Read Checkpoint File /exptbk/odc/oggapp/dirdat/ms000006546 2024-04-27 01:17:40.522457 RBA 483188832 GGSCI (dcdga02) 29> start E2_MD Sending START request to MANAGER ... EXTRACT E2_MD starting
3.指定传输进程从新挖出来的trail文件开始传输
GGSCI (dcdga02) 30> alter P2_MD extseqno 6556 extrba 0
检查应用进程，恢复同步。
来自 “ ITPUB博客 ” ，链接：https://blog.itpub.net/29956245/viewspace-3110158/ ，如需转载，请注明出处，否则将追究法律责任。
上一篇：
记一次OGG集成模式的bug
下一篇：
OGG升级需仔细！
请登录后发表评论
登录
全部评论
