一、环境描述

（1）Oracle 11.2.0.3 RAC ON Oracle Linux 6 x86_64，只有一个ASM外部冗余磁盘组——DATA；
（2）OCR，VOTEDISK，DATAFILE，CONTROLFILE，SPFILE全部位于这个磁盘组上；

二、故障描述

（1）存储故障导致ASM磁盘丢失。
（2）CRS因为OCR和VOTEDISK的丢失，除了OHAS还联机外，CLUSTERWARE服务都已经停止。

三、备份情况

（1）RMAN备份：包括controlfile，database，spfile，archivelog，
（2）OCR备份：没有进行过人工备份，在$CRS_HOME/cdata目录下有CRS自动备份文件。

四、操作步骤

说明：准使用CRS自动备份的文件恢复OCR，使用RMAN备份来恢复数据库；准备恢复数据的同时，调整ASM磁盘组，将OCR，VOTEDISK同数据库文件分开存放。

推荐阅读：

Oracle 11g从入门到精通 PDF+光盘源代码 http://www.linuxidc.com/Linux/2013-06/85670.htm

Ubuntu 12.04(amd64)安装完Oracle 11gR2后各种问题解决方法 http://www.linuxidc.com/Linux/2013-06/86155.htm

4.1 恢复OCR和VOTEDISK

(1) 在所有RAC节点上停止CRS服务

[root@rac1 ~]# crsctl stop has -f
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on'rac1'
CRS-2673: Attempting to stop 'ora.mdnsd'on'rac1'
CRS-2673: Attempting to stop 'ora.crf'on'rac1'
CRS-2677: Stop of'ora.mdnsd'on'rac1' succeeded
CRS-2677: Stop of'ora.crf'on'rac1' succeeded
CRS-2673: Attempting to stop 'ora.gipcd'on'rac1'
CRS-2677: Stop of'ora.gipcd'on'rac1' succeeded
CRS-2673: Attempting to stop 'ora.gpnpd'on'rac1'
CRS-2677: Stop of'ora.gpnpd'on'rac1' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on'rac1' has completed
CRS-4133: Oracle High Availability Services has been stopped.

[root@rac2 ~]# crsctl stop has -f
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on'rac2'
CRS-2673: Attempting to stop 'ora.mdnsd'on'rac2'
CRS-2673: Attempting to stop 'ora.crf'on'rac2'
CRS-2677: Stop of'ora.mdnsd'on'rac2' succeeded
CRS-2677: Stop of'ora.crf'on'rac2' succeeded
CRS-2673: Attempting to stop 'ora.gipcd'on'rac2'
CRS-2677: Stop of'ora.gipcd'on'rac2' succeeded
CRS-2673: Attempting to stop 'ora.gpnpd'on'rac2'
CRS-2677: Stop of'ora.gpnpd'on'rac2' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on'rac2' has completed
CRS-4133: Oracle High Availability Services has been stopped.

(2) 在一个节点上以NOCRS方式启动CRS，此操作会启动ASM实例。

[root@rac1 ~]# crsctl start crs -excl -nocrs
CRS-4123: Oracle High Availability Services has been started.
CRS-2672: Attempting to start 'ora.mdnsd'on'rac1'
CRS-2676: Start of'ora.mdnsd'on'rac1' succeeded
CRS-2672: Attempting to start 'ora.gpnpd'on'rac1'
CRS-2676: Start of'ora.gpnpd'on'rac1' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor'on'rac1'
CRS-2672: Attempting to start 'ora.gipcd'on'rac1'
CRS-2676: Start of'ora.cssdmonitor'on'rac1' succeeded
CRS-2676: Start of'ora.gipcd'on'rac1' succeeded
CRS-2672: Attempting to start 'ora.cssd'on'rac1'
CRS-2672: Attempting to start 'ora.diskmon'on'rac1'
CRS-2676: Start of'ora.diskmon'on'rac1' succeeded
CRS-2676: Start of'ora.cssd'on'rac1' succeeded
CRS-2679: Attempting to clean 'ora.cluster_interconnect.haip'on'rac1'
CRS-2672: Attempting to start 'ora.ctssd'on'rac1'
CRS-2681: Clean of'ora.cluster_interconnect.haip'on'rac1' succeeded
CRS-2672: Attempting to start 'ora.cluster_interconnect.haip'on'rac1'
CRS-2676: Start of'ora.ctssd'on'rac1' succeeded
CRS-2676: Start of'ora.cluster_interconnect.haip'on'rac1' succeeded
CRS-2672: Attempting to start 'ora.asm'on'rac1'
CRS-2676: Start of'ora.asm'on'rac1' succeeded

(3) 新添加了三块磁盘，已经使用UDEV进行了绑定，查看磁盘状态。

[root@rac1 ~]# su - grid
[grid@rac1 ~]$ sqlplus / as sysasm
SQL*Plus: Release 11.2.0.3.0 Production on Fri Jul 5 17:41:49 2013
Copyright (c) 1982, 2011, Oracle. All rights reserved.
Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
With the Real Application Clusters and Automatic Storage Management options
SQL> select group_number group#, disk_number disk#, OS_MB, state, path, header_status from v$asm_disk orderby 1,2;
GROUP# DISK# OS_MB STATE PATH HEADER_STATUS
---------- ---------- ---------- ---------- -------------------- ----------------------
0 0 1024 NORMAL /dev/asm-diskc CANDIDATE
0 1 5120 NORMAL /dev/asm-diskd CANDIDATE
0 2 20480 NORMAL /dev/asm-diskb CANDIDATE

(4) 创建三个磁盘组，SYSTEMDG给CRS使用，用于存放OCR，VOTEDISK和ASM实例的SPFILE。其余两个给ORACLE使用，DATADG用于存放datafile，controlfile，redolog，spfile；ARCLOGDG存放archivelog。

SQL> create diskgroup SYSTEMDG external redundancy
2 disk '/dev/asm-diskc'
3 ATTRIBUTE 'compatible.rdbms' = '11.2','compatible.asm' = '11.2';
Diskgroup created.
SQL> create diskgroup DATADG external redundancy
2 disk '/dev/asm-diskb'
3 ATTRIBUTE 'compatible.rdbms' = '11.2','compatible.asm' = '11.2';
Diskgroup created.
SQL> create diskgroup ARCLOGDG external redundancy
2 disk '/dev/asm-diskd'
3 ATTRIBUTE 'compatible.rdbms' = '11.2','compatible.asm' = '11.2';
Diskgroup created.

(5) 准备恢复OCR和VOTEDISK，/etc/oracle/ocr.loc中记录了OCR路径，修改ocrconfig_loc的值，以便将OCR恢复到新的磁盘组中。

[root@rac1 ~]# more /etc/oracle/ocr.loc
ocrconfig_loc=+DATA
local_only=FALSE
[root@rac1 ~]# vi /etc/oracle/ocr.loc
ocrconfig_loc=+SYSTEMDG
local_only=FALSE

(6) 恢复OCR

[root@rac1 ~]# ocrconfig -showbackup
PROT-26: Oracle Cluster Registry backup locations were retrieved from a local copy
rac1 2013/07/05 12:30:00 /u01/app/11.2.0/grid/cdata/rac-cluster/backup00.ocr
rac1 2013/07/05 08:30:00 /u01/app/11.2.0/grid/cdata/rac-cluster/backup01.ocr
rac1 2013/07/05 04:30:00 /u01/app/11.2.0/grid/cdata/rac-cluster/backup02.ocr
rac1 2013/07/05 00:29:59 /u01/app/11.2.0/grid/cdata/rac-cluster/day.ocr
rac1 2013/07/05 00:29:59 /u01/app/11.2.0/grid/cdata/rac-cluster/week.ocr
PROT-25: Manual backups for the Oracle Cluster Registry are not available
[root@rac1 ~]# ocrconfig -restore /u01/app/11.2.0/grid/cdata/rac-cluster/backup00.ocr
[root@rac1 ~]#
[root@rac1 ~]# ocrcheck
Status of Oracle Cluster Registry isas follows :
Version : 3
Total space (kbytes) : 262120
Used space (kbytes) : 2840
Available space (kbytes) : 259280
ID : 59415097
Device/File Name : +SYSTEMDG
Device/File integrity check succeeded
Device/File not configured
Device/File not configured
Device/File not configured
Device/File not configured
Cluster registry integrity check succeeded
Logical corruption check succeeded

(7) 创建VOTEDISK

[root@rac1 ~]# crsctl replace votedisk +SYSTEMDG
CRS-4602: Failed 27 toadd voting file afb0ca0f35684f1abfd43d5ec2dc1123.
Failed toreplace voting disk groupwith +SYSTEMDG.
CRS-4000: Command Replace failed, or completed with errors.

以上报错是因为使用UDEV绑定ASM磁盘时需要更改默认磁盘搜索路径为/dev/asm*，修改ASM磁盘搜索路径

[root@rac1 ~]# su - grid
[grid@rac1 ~]$ sqlplus / as sysasm
SQL*Plus: Release 11.2.0.3.0 Production on Fri Jul 5 19:03:25 2013
Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
With the Real Application Clusters and Automatic Storage Management options
SQL> show parameter asm_diskstring
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
asm_diskstring string
SQL>
SQL>
SQL> alter system set asm_diskstring = '/dev/asm*';
System altered.
SQL> create spfile from memory;
create spfile from memory
*
ERROR at line 1:
ORA-00349: failure obtaining block sizefor
'+DATA/rac-cluster/asmparameterfile/registry.253.819922365'
ORA-15001: diskgroup "DATA" does not exist orisnot mounted
SQL> create spfile='+SYSTEMDG'from memory;
File created.
SQL> startup force mount;
ORA-32004: obsolete or deprecated parameter(s) specified for ASM instance
ASM instance started
Total System Global Area 283930624 bytes
Fixed Size 2227664 bytes
Variable Size 256537136 bytes
ASM Cache 25165824 bytes
ASM diskgroups mounted

在次创建VOTEDISK，成功。

[root@rac1 init]# crsctl replace votedisk +SYSTEMDG
Successful addition of voting disk 8ebb7a63accb4fa8bfa7ab65df7a8c8a.
Successfully replaced voting disk groupwith +SYSTEMDG.
CRS-4266: Voting file(s) successfully replaced

(8) OCR和VOTEDISK都恢复完成后，重启CRS到正常模式。

[root@rac1 ~]# crsctl stop has -f
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on'rac1'
CRS-2673: Attempting to stop 'ora.mdnsd'on'rac1'
CRS-2673: Attempting to stop 'ora.ctssd'on'rac1'
CRS-2673: Attempting to stop 'ora.asm'on'rac1'
CRS-2677: Stop of'ora.mdnsd'on'rac1' succeeded
CRS-2677: Stop of'ora.asm'on'rac1' succeeded
CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip'on'rac1'
CRS-2677: Stop of'ora.ctssd'on'rac1' succeeded
CRS-2677: Stop of'ora.cluster_interconnect.haip'on'rac1' succeeded
CRS-2673: Attempting to stop 'ora.cssd'on'rac1'
CRS-2677: Stop of'ora.cssd'on'rac1' succeeded
CRS-2673: Attempting to stop 'ora.gipcd'on'rac1'
CRS-2677: Stop of'ora.gipcd'on'rac1' succeeded
CRS-2673: Attempting to stop 'ora.gpnpd'on'rac1'
CRS-2677: Stop of'ora.gpnpd'on'rac1' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on'rac1' has completed
CRS-4133: Oracle High Availability Services has been stopped.
[root@rac1 ~]# crsctl start crs
CRS-4123: Oracle High Availability Services has been started.
[root@rac1 ~]# crsctl check crs
CRS-4638: Oracle High Availability Services is online
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
[root@rac1 ~]#
4.2 修改CRS注册表中相关配置信息

(1) 挂载新的ASM磁盘组
1. [grid@rac1 ~]$ sqlplus / as sysasm
3. SQL*Plus: Release 11.2.0.3.0 Production on Sat Jul 6 00:16:05 2013
5. Copyright (c) 1982, 2011, Oracle. All rights reserved.
6. Connected to:
7. Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
8. With the Real Application Clusters and Automatic Storage Management options
10. SQL>
11. SQL> selectname,state from v$asm_diskgroup;
13. NAME STATE
14. ------------------------------ -----------
15. SYSTEMDG MOUNTED
16. ARCLOGDG DISMOUNTED
17. DATADG DISMOUNTED
19. SQL> alter diskgroup ARCLOGDG,DATADG mount;
21. Diskgroup altered.
(2) 更改CRS配置文件中数据库的磁盘组为DATADG和ARCLOGDG
1. [root@rac1 ~]# srvctl modifydatabase -d csdb -a "DATADG,ARCLOGDG"
(3) 禁用并删除原来的磁盘组DATA
1. [root@rac1 ~]# srvctl disable diskgroup -g DATA
2. [root@rac1 ~]# srvctl remove diskgroup -g DATA
4. [root@rac1 rac-cluster]# crs_stat -t -v
5. Name Type R/RA F/FT Target State Host
6. ----------------------------------------------------------------------
7. ora....OGDG.dg ora....up.type 0/5 0/ ONLINE ONLINE rac1
8. ora.DATADG.dg ora....up.type 0/5 0/ ONLINE ONLINE rac1
9. ora....ER.lsnr ora....er.type 0/5 0/ ONLINE ONLINE rac1
10. ora....N1.lsnr ora....er.type 0/5 0/0 ONLINE ONLINE rac1
11. ora.asm ora.asm.type 0/5 0/ ONLINE ONLINE rac1
12. ora.csdb.db ora....se.type 0/2 0/1 ONLINE OFFLINE
13. ora.cvu ora.cvu.type 0/5 0/0 ONLINE ONLINE rac1
14. ora.gsd ora.gsd.type 0/5 0/ OFFLINE OFFLINE
15. ora....network ora....rk.type 0/5 0/ ONLINE ONLINE rac1
16. ora.oc4j ora.oc4j.type 0/1 0/2 ONLINE ONLINE rac1
17. ora.ons ora.ons.type 0/3 0/ ONLINE ONLINE rac1
18. ora....SM1.asm application 0/5 0/0 ONLINE ONLINE rac1
19. ora....C1.lsnr application 0/5 0/0 ONLINE ONLINE rac1
20. ora.rac1.gsd application 0/5 0/0 OFFLINE OFFLINE
21. ora.rac1.ons application 0/3 0/0 ONLINE ONLINE rac1
22. ora.rac1.vip ora....t1.type 0/0 0/0 ONLINE ONLINE rac1
23. ora.rac2.vip ora....t1.type 0/0 0/0 ONLINE ONLINE rac1
24. ora.scan1.vip ora....ip.type 0/0 0/0 ONLINE ONLINE rac1
(4) 在OCR注册表中修改Oracle数据库参数文件的位置
1. [root@rac1 ~]# srvctl modifydatabase -d csdb -p +DATADG/csdb/spfilecsdb.ora
4.3 恢复数据库

(1) 查看备份文件路径和名称
1. [root@rac1 ~]# su - oracle
2. [oracle@rac1 ~]$
3. [oracle@rac1 ~]$ cd /u01/app/oracle/backup
4. [oracle@rac1 backup]$ ll
5. total 221796
6. -rw-r----- 1 oracle asmadmin 5357568 Jul 5 15:19 arc_819991156_9.bk
7. -rw-r----- 1 oracle asmadmin 2560 Jul 5 15:19 arc_819991158_11.bk
8. -rw-r----- 1 oracle asmadmin 203104256 Jul 5 15:18 CSDB_819991120_5.bk
9. -rw-r----- 1 oracle asmadmin 18546688 Jul 5 15:19 ctl_file_0coe04jq_1_1_20130705.ctl
10. -rw-r----- 1 oracle asmadmin 98304 Jul 5 15:19 spfile_0doe04js_1_1_20130705
11. [oracle@rac1 backup]$
(2) 创建一个基本的启动参数文件，以便启动数据库到nomout状态恢复spfile
1. [oracle@rac1 ~]$ touch /u01/app/oracle/backup/init.ora
2. [oracle@rac1 ~]$ vi /u01/app/oracle/backup/init.ora
4. *.db_name='csdb'
5. *.remote_login_passwordfile='exclusive'
(3) 使用刚创建的参数文件将数据库启动到nomount状态
1. [oracle@rac1 ~]$ sqlplus / as sysdba
3. SQL*Plus: Release 11.2.0.3.0 Production on Sat Jul 6 13:56:06 2013
5. Copyright (c) 1982, 2011, Oracle. All rights reserved.
7. Connected to an idle instance.
9. SQL> startup nomount pfile='/u01/app/oracle/backup/init.ora';
10. ORACLE instance started.
12. Total System Global Area 238034944 bytes
13. Fixed Size 2227136 bytes
14. Variable Size 180356160 bytes
15. Database Buffers 50331648 bytes
16. Redo Buffers 5120000 bytes
17. SQL>
(4) 使用RMAN恢复SPFILE到ASM磁盘组DATADG
1. [oracle@rac1 ~]$ rman target /
3. Recovery Manager: Release 11.2.0.3.0 - Production on Sat Jul 6 13:59:26 2013
5. Copyright (c) 1982, 2011, Oracle and/or its affiliates. All rights reserved.
7. connected to target database: CSDB (not mounted)
9. RMAN> restore spfile to'+DATADG/csdb/spfilecsdb.ora'from'/u01/app/oracle/backup/spfile_0doe04js_1_1_20130705';
11. Starting restore at 06-JUL-13
12. using channel ORA_DISK_1
14. channel ORA_DISK_1: restoring spfile from AUTOBACKUP /u01/app/oracle/backup/spfile_0doe04js_1_1_20130705
15. channel ORA_DISK_1: SPFILE restore from AUTOBACKUP complete
16. Finished restore at 06-JUL-13
(5) 使用恢复后spfile启动数据库，并修改control_files，db_recovery_file_dest，log_archive_dest等存在旧路径的参数值。
1. [oracle@rac1 ~]$ vi $ORACLE_HOME/dbs/initcsdb1.ora
3. SPFILE='+DATADG/csdb/spfilecsdb.ora'
5. [oracle@rac1 ~]$ sqlplus / as sysdba
7. SQL*Plus: Release 11.2.0.3.0 Production on Sat Jul 6 14:11:58 2013
9. Copyright (c) 1982, 2011, Oracle. All rights reserved.
12. Connected to:
13. Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
14. With the Partitioning, Real Application Clusters, OLAP, Data Mining
15. andReal Application Testing options
17. SQL> startup nomount force
18. ORACLE instance started.
20. Total System Global Area 1653518336 bytes
21. Fixed Size 2228904 bytes
22. Variable Size 1073745240 bytes
23. Database Buffers 570425344 bytes
24. Redo Buffers 7118848 bytes
25. SQL>
26. SQL> show parameter control_files
28. NAME TYPE VALUE
29. ------------------------------------ ----------- ------------------------------
30. control_files string +DATA/csdb/control01.ctl, +DAT
31. A/csdb/control02.ctl
32. SQL>
33. SQL> alter system set control_files='+DATADG/csdb/control01.ctl','+DATADG/csdb/control02.ctl' scope=spfile
35. System altered.
37. SQL> alter system set db_recovery_file_dest='+DATADG' scope=spfile;
39. System altered.
41. SQL> alter system set log_archive_dest_1='LOCATION=+ARCLOGDG' scope=spfile;
43. System altered.
45. SQL> startup force nomount;
46. ORACLE instance started.
48. Total System Global Area 1653518336 bytes
49. Fixed Size 2228904 bytes
50. Variable Size 1073745240 bytes
51. Database Buffers 570425344 bytes
52. Redo Buffers 7118848 bytes
(6) 查看数据库的DBID
1. [oracle@rac1 ~]$ strings /u01/app/oracle/backup/CSDB_819991120_5.bk | grep MAXVALUE,
3. 返回的值类似下面的例子，其中那一窜数字即为DBID。
4. ...
5. MAXVALUE, MAXVALUE!
6. 3042905279, MAXVALUE,
7. 3042905279, MAXVALUE,
8. ...
(7) 恢复控制文件到新的ASM磁盘组DATADG
1. [oracle@rac1 ~]$ rman target /
3. Recovery Manager: Release 11.2.0.3.0 - Production on Sat Jul 6 14:28:28 2013
5. Copyright (c) 1982, 2011, Oracle and/or its affiliates. All rights reserved.
7. connected to target database: CSDB (not mounted)
9. RMAN> set dbid=3042905279
11. executing command: SET DBID
13. RMAN> restore controlfile from'/u01/app/oracle/backup/ctl_file_0coe04jq_1_1_20130705.ctl';
15. Starting restore at 06-JUL-13
16. using target database control file insteadof recovery catalog
17. allocated channel: ORA_DISK_1
18. channel ORA_DISK_1: SID=18 instance=csdb1 device type=DISK
20. channel ORA_DISK_1: restoring control file
21. channel ORA_DISK_1: restore complete, elapsed time: 00:00:01
22. output file name=+DATADG/csdb/control01.ctl
23. output file name=+DATADG/csdb/control02.ctl
24. Finished restore at 06-JUL-13
(8) 进入SQLPLUS，查看旧数据文件信息
1. [oracle@rac1 ~]$ sqlplus / as sysdba
3. SQL*Plus: Release 11.2.0.3.0 Production on Sat Jul 6 14:41:12 2013
5. Copyright (c) 1982, 2011, Oracle. All rights reserved.
8. Connected to:
9. Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
10. With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,
11. Data Mining andReal Application Testing options
13. SQL> alterdatabase mount;
15. Database altered.
16. SQL> col name format a50
17. SQL> select file#,namefrom v$datafile;
19. FILE# NAME
20. ---------- --------------------------------------------------
21. 1 +DATA/csdb/datafile/system.260.819979847
22. 2 +DATA/csdb/datafile/sysaux.261.819979871
23. 3 +DATA/csdb/datafile/undotbs1.262.819979889
24. 4 +DATA/csdb/datafile/undotbs2.264.819979905
25. 5 +DATA/csdb/datafile/users.265.819979913
(9) 使用RMAN恢复数据库
1. RMAN> run{
2. 2> set newname for datafile 1 to'+DATADG/csdb/datafile/system.260.819979847';
3. 3> set newname for datafile 2 to'+DATADG/csdb/datafile/sysaux.261.819979871';
4. 4> set newname for datafile 3 to'+DATADG/csdb/datafile/undotbs1.262.819979889';
5. 5> set newname for datafile 4 to'+DATADG/csdb/datafile/undotbs2.264.819979905';
6. 6> set newname for datafile 5 to'+DATADG/csdb/datafile/users.265.819979913';
7. 7> restore database;
8. 8> switch datafile all;
9. 9> recover database;
10. 10> }
12. executing command: SET NEWNAME
13. released channel: ORA_DISK_1
15. executing command: SET NEWNAME
17. executing command: SET NEWNAME
19. executing command: SET NEWNAME
21. executing command: SET NEWNAME
23. Starting restore at 07-JUL-13
24. Starting implicit crosscheck backup at 07-JUL-13
25. allocated channel: ORA_DISK_1
26. Crosschecked 10 objects
27. Finished implicit crosscheck backup at 07-JUL-13
29. Starting implicit crosscheck copy at 07-JUL-13
30. using channel ORA_DISK_1
31. Finished implicit crosscheck copy at 07-JUL-13
33. searching forall files in the recovery area
34. cataloging files...
35. no files cataloged
37. using channel ORA_DISK_1
39. channel ORA_DISK_1: starting datafile backup set restore
40. channel ORA_DISK_1: specifying datafile(s) to restore from backup set
41. channel ORA_DISK_1: restoring datafile 00002 to +DATADG/csdb/datafile/sysaux.261.819979871
42. channel ORA_DISK_1: restoring datafile 00003 to +DATADG/csdb/datafile/undotbs1.262.819979889
43. channel ORA_DISK_1: reading from backup piece /u01/app/oracle/backup/CSDB_819991120_6.bk
44. channel ORA_DISK_1: piece handle=/u01/app/oracle/backup/CSDB_819991120_6.bk tag=ORCL_HOT_DB_BK
45. channel ORA_DISK_1: restored backup piece 1
46. channel ORA_DISK_1: restore complete, elapsed time: 00:00:46
47. channel ORA_DISK_1: starting datafile backup set restore
48. channel ORA_DISK_1: specifying datafile(s) to restore from backup set
49. channel ORA_DISK_1: restoring datafile 00001 to +DATADG/csdb/datafile/system.260.819979847
50. channel ORA_DISK_1: restoring datafile 00004 to +DATADG/csdb/datafile/undotbs2.264.819979905
51. channel ORA_DISK_1: restoring datafile 00005 to +DATADG/csdb/datafile/users.265.819979913
52. channel ORA_DISK_1: reading from backup piece /u01/app/oracle/backup/CSDB_819991120_5.bk
53. channel ORA_DISK_1: piece handle=/u01/app/oracle/backup/CSDB_819991120_5.bk tag=ORCL_HOT_DB_BK
54. channel ORA_DISK_1: restored backup piece 1
55. channel ORA_DISK_1: restore complete, elapsed time: 00:00:35
56. Finished restore at 07-JUL-13
58. datafile 1 switched to datafile copy
59. input datafile copy RECID=6 STAMP=820112831 file name=+DATADG/csdb/datafile/system.284.820112797
60. datafile 2 switched to datafile copy
61. input datafile copy RECID=7 STAMP=820112831 file name=+DATADG/csdb/datafile/sysaux.282.820112751
62. datafile 3 switched to datafile copy
63. input datafile copy RECID=8 STAMP=820112831 file name=+DATADG/csdb/datafile/undotbs1.283.820112751
64. datafile 4 switched to datafile copy
65. input datafile copy RECID=9 STAMP=820112831 file name=+DATADG/csdb/datafile/undotbs2.285.820112797
66. datafile 5 switched to datafile copy
67. input datafile copy RECID=10 STAMP=820112831 file name=+DATADG/csdb/datafile/users.286.820112797
69. Starting recover at 07-JUL-13
70. using channel ORA_DISK_1
72. starting media recovery
74. channel ORA_DISK_1: starting archived log restore todefault destination
75. channel ORA_DISK_1: restoring archived log
76. archived log thread=1 sequence=15
77. channel ORA_DISK_1: reading from backup piece /u01/app/oracle/backup/arc_819991156_9.bk
78. channel ORA_DISK_1: piece handle=/u01/app/oracle/backup/arc_819991156_9.bk tag=TAG20130705T151916
79. channel ORA_DISK_1: restored backup piece 1
80. channel ORA_DISK_1: restore complete, elapsed time: 00:00:01
81. archived log file name=+ARCLOGDG/csdb/archivelog/2013_07_07/thread_1_seq_15.256.820112833 thread=1 sequence=15
82. channel ORA_DISK_1: starting archived log restore todefault destination
83. channel ORA_DISK_1: restoring archived log
84. archived log thread=2 sequence=2
85. channel ORA_DISK_1: restoring archived log
86. archived log thread=1 sequence=16
87. channel ORA_DISK_1: reading from backup piece /u01/app/oracle/backup/arc_819991156_10.bk
88. channel ORA_DISK_1: piece handle=/u01/app/oracle/backup/arc_819991156_10.bk tag=TAG20130705T151916
89. channel ORA_DISK_1: restored backup piece 1
90. channel ORA_DISK_1: restore complete, elapsed time: 00:00:01
91. archived log file name=+ARCLOGDG/csdb/archivelog/2013_07_07/thread_2_seq_2.257.820112835 thread=2 sequence=2
92. channel default: deleting archived log(s)
93. archived log file name=+ARCLOGDG/csdb/archivelog/2013_07_07/thread_1_seq_15.256.820112833 RECID=5 STAMP=820112832
94. archived log file name=+ARCLOGDG/csdb/archivelog/2013_07_07/thread_1_seq_16.258.820112835 thread=1 sequence=16
95. channel default: deleting archived log(s)
96. archived log file name=+ARCLOGDG/csdb/archivelog/2013_07_07/thread_2_seq_2.257.820112835 RECID=7 STAMP=820112834
97. channel ORA_DISK_1: starting archived log restore todefault destination
98. channel ORA_DISK_1: restoring archived log
99. archived log thread=2 sequence=3
100. channel ORA_DISK_1: reading from backup piece /u01/app/oracle/backup/arc_819991158_11.bk
101. channel ORA_DISK_1: piece handle=/u01/app/oracle/backup/arc_819991158_11.bk tag=TAG20130705T151916
102. channel ORA_DISK_1: restored backup piece 1
103. channel ORA_DISK_1: restore complete, elapsed time: 00:00:01
104. archived log file name=+ARCLOGDG/csdb/archivelog/2013_07_07/thread_2_seq_3.257.820112837 thread=2 sequence=3
105. channel default: deleting archived log(s)
106. archived log file name=+ARCLOGDG/csdb/archivelog/2013_07_07/thread_2_seq_3.257.820112837 RECID=8 STAMP=820112835
107. unable to find archived log
108. archived log thread=2 sequence=4
109. RMAN-00571: ===========================================================
110. RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
111. RMAN-00571: ===========================================================
112. RMAN-03002: failure of recover command at 07/07/2013 01:07:16
113. RMAN-06054: media recovery requesting unknown archived log for thread 2 withsequence 4 and starting SCN of 323980
(10) 更改REDO LOG位置信息
1. SQL> select member from v$logfile;
3. MEMBER
4. --------------------------------------------------------------------------------
5. +DATA/csdb/redo01.log
6. +DATA/csdb/redo02.log
7. +DATA/csdb/redo03.log
8. +DATA/csdb/redo04.log
10. SQL> alterdatabase rename file '+DATA/csdb/redo01.log'to'+DATADG/csdb/redo01.log';
12. Database altered.
14. SQL> alterdatabase rename file '+DATA/csdb/redo02.log'to'+DATADG/csdb/redo02.log';
16. Database altered.
18. SQL> alterdatabase rename file '+DATA/csdb/redo03.log'to'+DATADG/csdb/redo03.log';
20. Database altered.
22. SQL> alterdatabase rename file '+DATA/csdb/redo04.log'to'+DATADG/csdb/redo04.log';
24. Database altered.
(11) 打开数据库
1. SQL> alterdatabaseopen resetlogs;
3. Database altered.
(12) 更改TEMP表空间文件位置
1. SQL> selectnamefrom v$tempfile;
3. NAME
4. --------------------------------------------------------------------------------
5. +DATA/csdb/tempfile/temp.263.819979895
7. SQL> alter tablespace tempadd tempfile '+DATADG';
9. Tablespace altered.
12. SQL> alter tablespace tempdrop tempfile '+DATA/csdb/tempfile/temp.263.819979895';
14. Tablespace altered
4.4 完成恢复操作

(1) 在其他RAC节点上更改OCR路径
1. [root@rac2 ~]# vi /etc/oracle/ocr.loc
3. ocrconfig_loc=+SYSTEMDG
4. local_only=FALSE
(2) 在恢复节点上重启CRS
1. [root@rac1 ~]# crsctl stop crs
2. [root@rac1 ~]# crsctl start has
(3) 在其他节点上启动CRS

ITPub博客

Oracle 11g RAC ASM磁盘全部丢失后的恢复

一、环境描述

二、故障描述

三、备份情况

四、操作步骤

4.1 恢复OCR和VOTEDISK

4.2 修改CRS注册表中相关配置信息

4.3 恢复数据库

4.4 完成恢复操作

最新文章