背景介绍：当你不小心rm -rf *.dbf时，千万不要慌张重启数据库或者操作系统，可以通过dbwn进程相关句柄找回数据文件。

测试环境：RHEL 7.6  Oracle 11.2.0.4 单机文件系统

实验如下：

1）删除一个数据文件

$ rm -rf users01*

[root@xmc ~]# ps -ef |grep dbw|grep -v grep

oracle   25330     1  0 07:41 ?        00:00:00 ora_dbw0_xmc

或者

#ps -ef |grep dbw

[root@xmc ~]# ll /proc/25330/fd

total 0

lr-x------ 1 oracle oinstall 64 Mar 24 09:42 0 -> /dev/null

l-wx------ 1 oracle oinstall 64 Mar 24 09:42 1 -> /dev/null

lrwx------ 1 oracle oinstall 64 Mar 24 09:42 10 -> /u01/app/oracle/product/11.2.0/db_1/dbs/lkCD_XMC

lr-x------ 1 oracle oinstall 64 Mar 24 09:42 11 -> /u01/app/oracle/product/11.2.0/db_1/rdbms/mesg/oraus.msb

l-wx------ 1 oracle oinstall 64 Mar 24 09:42 2 -> /dev/null

lrwx------ 1 oracle oinstall 64 Mar 24 09:42 256 -> /u01/app/oracle/oradata/xmc/control01.ctl

lrwx------ 1 oracle oinstall 64 Mar 24 09:42 257 -> /u01/app/oracle/oradata/xmc/system01.dbf

lrwx------ 1 oracle oinstall 64 Mar 24 09:42 258 -> /u01/app/oracle/oradata/xmc/sysaux01.dbf

lrwx------ 1 oracle oinstall 64 Mar 24 09:42 259 -> /u01/app/oracle/oradata/xmc/undotbs01.dbf

lrwx------ 1 oracle oinstall 64 Mar 24 09:42 260 -> /u01/app/oracle/oradata/xmc/rkdz01.dbf

lrwx------ 1 oracle oinstall 64 Mar 24 09:42 261 -> /u01/app/oracle/oradata/xmc/temp01.dbf

lrwx------ 1 oracle oinstall 64 Mar 24 09:42 262 -> /u01/app/oracle/oradata/xmc/temp02.dbf

lr-x------ 1 oracle oinstall 64 Mar 24 09:42 3 -> /dev/null

lr-x------ 1 oracle oinstall 64 Mar 24 09:42 4 -> /dev/null

lr-x------ 1 oracle oinstall 64 Mar 24 09:42 5 -> /dev/null

lr-x------ 1 oracle oinstall 64 Mar 24 09:42 6 -> /u01/app/oracle/product/11.2.0/db_1/rdbms/mesg/oraus.msb

lr-x------ 1 oracle oinstall 64 Mar 24 09:42 7 -> /proc/25330/fd

lr-x------ 1 oracle oinstall 64 Mar 24 09:42 8 -> /dev/zero

lrwx------ 1 oracle oinstall 64 Mar 24 09:42 9 -> /u01/app/oracle/product/11.2.0/db_1/dbs/hc_xmc.dat

 

然后这里没有users文件，后面想起这个文件被我offline了，为了确认

SQL> select name,status from v$datafile;

 

NAME                                                    STATUS

------------------------------------------------------------ -------

/u01/app/oracle/oradata/xmc/system01.dbf                SYSTEM

/u01/app/oracle/oradata/xmc/sysaux01.dbf                ONLINE

/u01/app/oracle/oradata/xmc/undotbs01.dbf              ONLINE

/u01/app/oracle/oradata/xmc/users01.dbf                  OFFLINE

/u01/app/oracle/oradata/xmc/rkdz01.dbf                    ONLINE

此时先将此文件drop

SQL> drop tablespace users;

drop tablespace users

*

ERROR at line 1:

ORA-12919: Can not drop the default permanent tablespace

有备份，先恢复吧

SQL> shutdown immediate;

Database closed.

Database dismounted.

ORACLE instance shut down.

SQL> startup

ORACLE instance started.

 

Total System Global Area  826753024 bytes

Fixed Size                 2257760 bytes

Variable Size           503319712 bytes

Database Buffers     314572800 bytes

Redo Buffers             6602752 bytes

Database mounted.

Database opened.

SQL> select name,status from v$datafile;

 

NAME                                                    STATUS

------------------------------------------------------------ -------

/u01/app/oracle/oradata/xmc/system01.dbf                SYSTEM

/u01/app/oracle/oradata/xmc/sysaux01.dbf                ONLINE

/u01/app/oracle/oradata/xmc/undotbs01.dbf              ONLINE

/u01/app/oracle/oradata/xmc/users01.dbf                  OFFLINE

/u01/app/oracle/oradata/xmc/rkdz01.dbf                    ONLINE

此时虽然这个users01没了，但是由于是offline的，所以可以正常开启关闭（ 过会儿验证），现在进行恢复

RMAN> restore datafile 4;

 

Starting restore at 24-MAR-21

using target database control file instead of recovery catalog

allocated channel: ORA_DISK_1

channel ORA_DISK_1: SID=22 device type=DISK

 

channel ORA_DISK_1: starting datafile backup set restore

channel ORA_DISK_1: specifying datafile(s) to restore from backup set

channel ORA_DISK_1: restoring datafile 00004 to /u01/app/oracle/oradata/xmc/users01.dbf

channel ORA_DISK_1: reading from backup piece /u01/app/oracle/fast_recovery_area/CD_XMC/backupset/2021_03_24/o1_mf_nnndf_TAG20210324T094803_j5o6lmoh_.bkp

channel ORA_DISK_1: piece handle=/u01/app/oracle/fast_recovery_area/CD_XMC/backupset/2021_03_24/o1_mf_nnndf_TAG20210324T094803_j5o6lmoh_.bkp tag=TAG20210324T094803

channel ORA_DISK_1: restored backup piece 1

channel ORA_DISK_1: restore complete, elapsed time: 00:00:01

Finished restore at 24-MAR-21

 

RMAN> recover datafile 4;

 

Starting recover at 24-MAR-21

using channel ORA_DISK_1

 

starting media recovery

media recovery complete, elapsed time: 00:00:00

 

Finished recover at 24-MAR-21

 

SQL> alter tablespace users online;

 

Tablespace altered.

 

再次进行删除

$rm -rf users01.dbf

[root@xmc ~]# ps -ef |grep dbw

oracle   27946     1  0 10:13 ?        00:00:00 ora_dbw0_xmc

root     28187 27479  0 10:19 pts/5    00:00:00 grep --color=auto dbw

[root@xmc ~]# ll /proc/27946/fd

total 0

lr-x------ 1 oracle oinstall 64 Mar 24 10:19 0 -> /dev/null

l-wx------ 1 oracle oinstall 64 Mar 24 10:19 1 -> /dev/null

lrwx------ 1 oracle oinstall 64 Mar 24 10:19 10 -> /u01/app/oracle/product/11.2.0/db_1/dbs/lkCD_XMC

lr-x------ 1 oracle oinstall 64 Mar 24 10:19 11 -> /u01/app/oracle/product/11.2.0/db_1/rdbms/mesg/oraus.msb

l-wx------ 1 oracle oinstall 64 Mar 24 10:19 12 -> /u01/app/oracle/diag/rdbms/cd_xmc/xmc/trace/xmc_dbw0_27946.trc

l-wx------ 1 oracle oinstall 64 Mar 24 10:19 13 -> /u01/app/oracle/diag/rdbms/cd_xmc/xmc/trace/xmc_dbw0_27946.trm

l-wx------ 1 oracle oinstall 64 Mar 24 10:19 2 -> /dev/null

lrwx------ 1 oracle oinstall 64 Mar 24 10:19 256 -> /u01/app/oracle/oradata/xmc/control01.ctl

lrwx------ 1 oracle oinstall 64 Mar 24 10:19 257 -> /u01/app/oracle/oradata/xmc/system01.dbf

lrwx------ 1 oracle oinstall 64 Mar 24 10:19 258 -> /u01/app/oracle/oradata/xmc/sysaux01.dbf

lrwx------ 1 oracle oinstall 64 Mar 24 10:19 259 -> /u01/app/oracle/oradata/xmc/undotbs01.dbf

lrwx------ 1 oracle oinstall 64 Mar 24 10:19 260 -> /u01/app/oracle/oradata/xmc/rkdz01.dbf

lrwx------ 1 oracle oinstall 64 Mar 24 10:19 261 -> /u01/app/oracle/oradata/xmc/temp01.dbf

lrwx------ 1 oracle oinstall 64 Mar 24 10:19 262 -> /u01/app/oracle/oradata/xmc/temp02.dbf

lrwx------ 1 oracle oinstall 64 Mar 24 10:19 263 -> /u01/app/oracle/oradata/xmc/users01.dbf (deleted)

lr-x------ 1 oracle oinstall 64 Mar 24 10:19 3 -> /dev/null

lr-x------ 1 oracle oinstall 64 Mar 24 10:19 4 -> /dev/null

lr-x------ 1 oracle oinstall 64 Mar 24 10:19 5 -> /dev/null

lr-x------ 1 oracle oinstall 64 Mar 24 10:19 6 -> /u01/app/oracle/product/11.2.0/db_1/rdbms/mesg/oraus.msb

lr-x------ 1 oracle oinstall 64 Mar 24 10:19 7 -> /proc/27946/fd

lr-x------ 1 oracle oinstall 64 Mar 24 10:19 8 -> /dev/zero

lrwx------ 1 oracle oinstall 64 Mar 24 10:19 9 -> /u01/app/oracle/product/11.2.0/db_1/dbs/hc_xmc.dat

 

红色部分看到刚删除的文件，此时进行恢复操作

[oracle@xmc ~]$ cp /proc/27946/fd/263 /u01/app/oracle/oradata/xmc/users01.dbf

# ll /proc/28337/fd

total 0

lr-x------ 1 oracle oinstall 64 Mar 24 10:24 0 -> /dev/null

l-wx------ 1 oracle oinstall 64 Mar 24 10:24 1 -> /dev/null

lrwx------ 1 oracle oinstall 64 Mar 24 10:24 10 -> /u01/app/oracle/product/11.2.0/db_1/dbs/lkCD_XMC

lr-x------ 1 oracle oinstall 64 Mar 24 10:24 11 -> /u01/app/oracle/product/11.2.0/db_1/rdbms/mesg/oraus.msb

l-wx------ 1 oracle oinstall 64 Mar 24 10:24 2 -> /dev/null

lrwx------ 1 oracle oinstall 64 Mar 24 10:24 256 -> /u01/app/oracle/oradata/xmc/control01.ctl

lrwx------ 1 oracle oinstall 64 Mar 24 10:24 257 -> /u01/app/oracle/oradata/xmc/system01.dbf

lrwx------ 1 oracle oinstall 64 Mar 24 10:24 258 -> /u01/app/oracle/oradata/xmc/sysaux01.dbf

lrwx------ 1 oracle oinstall 64 Mar 24 10:24 259 -> /u01/app/oracle/oradata/xmc/undotbs01.dbf

lrwx------ 1 oracle oinstall 64 Mar 24 10:24 260 -> /u01/app/oracle/oradata/xmc/users01.dbf

lrwx------ 1 oracle oinstall 64 Mar 24 10:24 261 -> /u01/app/oracle/oradata/xmc/rkdz01.dbf

lrwx------ 1 oracle oinstall 64 Mar 24 10:24 262 -> /u01/app/oracle/oradata/xmc/temp01.dbf

lrwx------ 1 oracle oinstall 64 Mar 24 10:24 263 -> /u01/app/oracle/oradata/xmc/temp02.dbf

lr-x------ 1 oracle oinstall 64 Mar 24 10:24 3 -> /dev/null

lr-x------ 1 oracle oinstall 64 Mar 24 10:24 4 -> /dev/null

lr-x------ 1 oracle oinstall 64 Mar 24 10:24 5 -> /dev/null

lr-x------ 1 oracle oinstall 64 Mar 24 10:24 6 -> /u01/app/oracle/product/11.2.0/db_1/rdbms/mesg/oraus.msb

lr-x------ 1 oracle oinstall 64 Mar 24 10:24 7 -> /proc/28337/fd

lr-x------ 1 oracle oinstall 64 Mar 24 10:24 8 -> /dev/zero

lrwx------ 1 oracle oinstall 64 Mar 24 10:24 9 -> /u01/app/oracle/product/11.2.0/db_1/dbs/hc_xmc.dat

查看红色部分，已经正常

查看/u01/app/oracle/oradata/xmc/目录已经恢复users01，查看数据文件状态，并将数据库重启

SQL> startup force; 生产上注意安全停库

ORACLE instance started.

 

Total System Global Area  826753024 bytes

Fixed Size                 2257760 bytes

Variable Size           503319712 bytes

Database Buffers     314572800 bytes

Redo Buffers             6602752 bytes

Database mounted.

Database opened.

 

2）删除所有数据文件

$ rm -rf *.dbf

# ps -ef |grep dbw

oracle   28337     1  0 10:23 ?        00:00:00 ora_dbw0_xmc

root     28499 27479  0 10:30 pts/5    00:00:00 grep --color=auto dbw

[root@xmc ~]# ll /proc/28337/fd

total 0

lr-x------ 1 oracle oinstall 64 Mar 24 10:24 0 -> /dev/null

l-wx------ 1 oracle oinstall 64 Mar 24 10:24 1 -> /dev/null

lrwx------ 1 oracle oinstall 64 Mar 24 10:24 10 -> /u01/app/oracle/product/11.2.0/db_1/dbs/lkCD_XMC

lr-x------ 1 oracle oinstall 64 Mar 24 10:24 11 -> /u01/app/oracle/product/11.2.0/db_1/rdbms/mesg/oraus.msb

l-wx------ 1 oracle oinstall 64 Mar 24 10:24 2 -> /dev/null

lrwx------ 1 oracle oinstall 64 Mar 24 10:24 256 -> /u01/app/oracle/oradata/xmc/control01.ctl

lrwx------ 1 oracle oinstall 64 Mar 24 10:24 257 -> /u01/app/oracle/oradata/xmc/system01.dbf ( deleted)

lrwx------ 1 oracle oinstall 64 Mar 24 10:24 258 -> /u01/app/oracle/oradata/xmc/sysaux01.dbf (deleted)

lrwx------ 1 oracle oinstall 64 Mar 24 10:24 259 -> /u01/app/oracle/oradata/xmc/undotbs01.dbf ( deleted)

lrwx------ 1 oracle oinstall 64 Mar 24 10:24 260 -> /u01/app/oracle/oradata/xmc/users01.dbf ( deleted)

lrwx------ 1 oracle oinstall 64 Mar 24 10:24 261 -> /u01/app/oracle/oradata/xmc/rkdz01.dbf ( deleted)

lrwx------ 1 oracle oinstall 64 Mar 24 10:24 262 -> /u01/app/oracle/oradata/xmc/temp01.dbf ( deleted)

lrwx------ 1 oracle oinstall 64 Mar 24 10:24 263 -> /u01/app/oracle/oradata/xmc/temp02.dbf ( deleted)

lr-x------ 1 oracle oinstall 64 Mar 24 10:24 3 -> /dev/null

lr-x------ 1 oracle oinstall 64 Mar 24 10:24 4 -> /dev/null

lr-x------ 1 oracle oinstall 64 Mar 24 10:24 5 -> /dev/null

lr-x------ 1 oracle oinstall 64 Mar 24 10:24 6 -> /u01/app/oracle/product/11.2.0/db_1/rdbms/mesg/oraus.msb

lr-x------ 1 oracle oinstall 64 Mar 24 10:24 7 -> /proc/28337/fd

lr-x------ 1 oracle oinstall 64 Mar 24 10:24 8 -> /dev/zero

lrwx------ 1 oracle oinstall 64 Mar 24 10:24 9 -> /u01/app/oracle/product/11.2.0/db_1/dbs/hc_xmc.dat

可以看到deleted全是刚删的，进行cp恢复

$ cp /proc/28337/fd/257 /u01/app/oracle/oradata/xmc/system01.dbf

$ cp /proc/28337/fd/258 /u01/app/oracle/oradata/xmc/sysaux01.dbf

$ cp /proc/28337/fd/259 /u01/app/oracle/oradata/xmc/undotbs01.dbf

$ cp /proc/28337/fd/260 /u01/app/oracle/oradata/xmc/users01.dbf

$ cp /proc/28337/fd/261 /u01/app/oracle/oradata/xmc/rkdz01.dbf

$ cp /proc/28337/fd/262 /u01/app/oracle/oradata/xmc/temp01.dbf

$ cp /proc/28337/fd/263 /u01/app/oracle/oradata/xmc/temp02.dbf

启动之后出现如下错误

SQL> startup force;

ORACLE instance started.

 

Total System Global Area  826753024 bytes

Fixed Size                 2257760 bytes

Variable Size           503319712 bytes

Database Buffers     314572800 bytes

Redo Buffers             6602752 bytes

Database mounted.

ORA-00600: internal error code, arguments: [kcratr_scan_lastbwr], [], [], [], [], [], [], [], [], [], [], []

上mos一看 ID 1267231.1

ERROR:

  Format: ORA-600 [kcratr_scan_lastbwr] [a] [b] [c] [d] [e]

DETAILS

SUGGESTIONS:

If you encounter this error, try to recover the database explicitly and see if it opens:

      SQL> startup mount
      SQL> recover database;
      SQL> alter database open;

 

实际操作：

SQL> select open_mode from v$database;

 

OPEN_MODE

--------------------

MOUNTED

 

SQL> recover database;

Media recovery complete.

SQL> alter database open;

 

Database altered.

问题解决。