Linux oracle 10gR2 deleting nodes completely

+++++++++++++++++Env+++++++++++++++++
Host:
    node1 192.168.24.11
    node2 192.168.24.12

DB name:  rac
Instance: rac1/rac2
          ASM1/ASM2
          
DB Version 10.2.0.5
+++++++++++++++++Env+++++++++++++++++

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
==================================Summary===================================
    1 Using DBCA in interactive mode to delete Instance from the existing node
    2 Deleting Nodes from Oracle Real Application Cluster Databases
            1) Asm Instance Clean-Up Procedure for Node Deletion
            2) NETCA remove listener
            3) Remove the Oracle Database software and update the Node List
               for the node to be deleted
            4) Remove the Node to be deleted from Oracle Clusterware
            5) Remove/rename any remaining Oracle Files from Node to be deleted


==================================Summary===================================

@@1 Using DBCA in interactive mode to delete Instance from the existing node

startx
xhost +
su - oracle
dbca &
Oracle Real Application Clusters Database->Instance Management->Delete an Instance
->Choose RAC name and enter sysdba user and password->Select node 2 to be deleted
->Change service perfer to another node if service is perferred on deleted node
->Finish Button(node instance and its OFA will be deleted

-@Verification

Verify if Instance removed:

select inst_id,instance_name,status,to_char(startup_time,'dd-mon-yyyy hh24:mi:ss')
from gv$instance order by inst_id
/
 
Verify if DB files and TBS removed:
select tablespace_name from dba_tablespaces order by 1;

Verify if online redo log removed;
select inst_id,group#,member from gv$logfile order by 1,2,3;

 
@@2 Deleting Nodes from Oracle Real Application Cluster Databases

1)Asm Instance Clean-Up Procedure
 
-@Remove ASM instance from existing node
srvctl stop asm -n node2
srvctl remove asm -n node2

-@Remove ASM dir on deleted node
cd $ORACLE_BASE/admin
rm -rf +ASM

cd $ORACLE_HOME/dbs
rm -rf *ASM*

delete the ASM entries on /etc/oratab on deleted node

-@Remove ASMLIB on the deleted node
#/etc/init.d/oracleasm stop
#rpm -qa |grep oracleasm
The command completed successfully
[oracle@node2 ~]$ rpm -qa |grep oracleasm
oracleasmlib-2.0.4-1.el5
oracleasm-support-2.1.7-1.el5
oracleasm-2.6.18-164.el5-2.0.5-1.el5
#rpm -ev oracleasmlib-2.0.4-1.el5 oracleasm-support-2.1.7-1.el5 oracleasm-2.6.18-164.el5-2.0.5-1.el5

#rm -f /etc/sysconfig/oracleasm.rpmsave
#rm -f /etc/rc.d/init.d/oracleasm
#rm -f /etc/rc0.d/*oracleasm*
#rm -f /etc/rc1.d/*oracleasm*
#rm -f /etc/rc2.d/*oracleasm*
#rm -f /etc/rc3.d/*oracleasm*
#rm -f /etc/rc4.d/*oracleasm*
#rm -f /etc/rc5.d/*oracleasm*
#rm -f /etc/rc6.d/*oracleasm*

2) Remove listener on the deleted node

xhost+
su - oracle
netca &
Cluster configuration->Select deleted node->Listener configuration->Delete LISTENER->Finsh

Verify TNS config
$hostname
node1
$ps -ef|grep lsnr|grep -v "grep"|grep -v 'ocfs'|awk '{print $9}'
LISTENER_NODE1
$ $ORA_CRS_HOME/bin/crs_stat ora.node1.LISTENER_NODE1.lsnr

$hostname
node2
$ps -ef|grep lsnr|grep -v "grep"|grep -v 'ocfs'|awk '{print $9}'
LISTENER_NODE2
$ $ORA_CRS_HOME/bin/crs_stat ora.node1.LISTENER_NODE2.lsnr

3) Remove the Oracle Database software and update the Node List
     for the node to be deleted

-@Update inventory node list for software with OUI
!Run on node to be deleted
xhost + (or DISPLAY=your local workstation):0.0; export DISPLAY
su - oracle
cd $ORACLE_HOME/oui/bin
./runInstaller -updateNodeList ORACLE_HOME=$ORACLE_HOME CLUSTER_NODES="" -local

Note: Although the OUI does not lauch an installer GUI, the xhost+(or DISPLAY) env still needs to be set!

-@De-install Oracle Database Software
!Run on node to be deleted
xhost + (DISPLAY=your local workstation):0.0; export DISPLAY
su - oracle
cd $ORACLE_HOME/oui/bin
./runInstaller

Deinstall products-OraDb10g_home1->Remove-Yes

-@Update Node list for remaining Nodes in the cluster on existing node
xhost + (DISPLAY=your local workstation):0.0; export DISPLAY
su - oracle
cd $ORACLE_HOME/oui/bin
./runInstaller -updateNodeList ORACLE_HOME=$ORACLE_HOME "CLUSTER_NODES={node1}"

Note: Although the OUI does not lauch an installer GUI, the xhost+(or DISPLAY) env still needs to be set!

4) Remove the Node to be deleted from Oracle Clusterware

-@Remove Node-Specific Interface Configuration on delete node
Get ons port number
$ grep '^remoteport' $CRS_HOME/opmn/conf/ons.config
[oracle@node2 ~]$ grep '^remoteport' $CRS_HOME/opmn/conf/ons.config
remoteport=6200
$ $CRS_HOME/bin/racgons remove_config node2:6200
$ $CRS_HOME/bin/oifcfg delif -node node2
PROC-4: The cluster registry key to be operated on does not exist.
PRIF-11: cluster registry error
--This can be ignored safely

-@Disable/Stop Oracle Clusterware Application 
!Run on node to be deleted

su - root
cd $CRS_HOME/install
./rootdelete.sh local nosharedvar nosharedhome       --should be only run once.
                                                                                                         --nosharedhome  --crs in local disk
                                                                                                         --nosharedvar   --assume the ocr.loc file is not on a shared file system
                                                                                                         --Will stop CRS stack and delete orc.loc file

Shutting down Oracle Cluster Ready Services (CRS):
Oct 09 23:09:37.834 | INF | daemon shutting down
Stopping resources. This could take several minutes.
Successfully stopped CRS resources.
Stopping CSSD.
Shutting down CSS daemon.
Shutdown request successfully issued.
Shutdown has begun. The daemons should exit soon.
Checking to see if Oracle CRS stack is down...

Checking to see if Oracle CRS stack is down...
Oracle CRS stack is not running.
Oracle CRS stack is down now.
Removing script. for Oracle Cluster Ready services
Updating ocr file for downgrade
Cleaning up SCR settings in '/etc/oracle/scls_scr'
Cleaning up Network socket directories

-@Delete Node from Cluster and Update OCR
!Run on existing nodes

Get node number:
$ $CRS_HOME/bin/olsnodes -n
node1   1
node2   2

node1:
su - root
cd $CRS_HOME/install
./rootdeletenode.sh node2,2

[root@node1 install]# ./rootdeletenode.sh node2,2
CRS-0210: Could not find resource 'ora.node2.ons'.
CRS-0210: Could not find resource 'ora.node2.vip'.
CRS-0210: Could not find resource 'ora.node2.gsd'.
CRS-0210: Could not find resource ora.node2.vip.
CRS nodeapps are deleted successfully
clscfg: EXISTING configuration version 3 detected.
clscfg: version 3 is 10G Release 2.
Successfully deleted 14 values from OCR.
Key SYSTEM.css.interfaces.nodenode2 marked for deletion is not there. Ignoring.
Successfully deleted 5 keys from OCR.
Node deletion operation successful.
'node2,2' deleted successfully

$ $CRS_HOME/bin/olsnodes -n
'node2' deleted successfully

-@Update Node List for Oracle Clusterware software
!Run on node to be deleted
xhost + (DISPLAY=your local workstation):0.0; export DISPLAY
su - oracle
cd $CRS_HOME/oui/bin
./runInstaller -updateNodeList ORACLE_HOME=$CRS_HOME CLUSTER_NODES="" -local CRS=true

Note: Although the OUI does not lauch an installer GUI, the xhost+(or DISPLAY) env still needs to be set!

-@Deinstall Oracle Clusterware Software
!Run on the node to be deleted

xhost + (DISPLAY=your local workstation):0.0; export DISPLAY
su - oracle
cd $CRS_HOME/oui/bin
./runInstaller

Deinstall products-OraCrs10g_home->remove-Yes

Note: Although the OUI does not lauch an installer GUI, the xhost+(or DISPLAY) env still needs to be set!

-@Update Node list for remaining Nodes in the cluster on existing node
xhost + (DISPLAY=your local workstation):0.0; export DISPLAY
su - oracle
cd $CRS_HOME/oui/bin
./runInstaller -updateNodeList ORACLE_HOME=$CRS_HOME "CLUSTER_NODES={node1}"  CRS=true

Note: Although the OUI does not lauch an installer GUI, the xhost+(or DISPLAY) env still needs to be set!

-@Verify Node to be deleted is no longer a member of the Cluster
$ srvctl status nodeapps -n node2
PRKC-1056 : Failed to get the hostname for node node2
PRKH-1001 : HASContext Internal Error
  [OCR Error(Native: getHostName:[21])]
[oracle@node1 ~]$ $CRS_HOME/bin/crs_stat |grep -i node2
no return

5) Remove/rename any remaining Oracle Files from Node to be deleted
root:
rm -f /etc/inittab.no_crs /etc/inittab
rm -f /etc/inittab.orig
rm -f /etc/inittab.crs

rm -f /etc/oracle
rm -f /etc/oratab
rm -f /etc/oraInst.loc
rm -f /etc/ORCLcluster
rm -rf /u01/app/oracle
rm -f /usr/local/bin/coraenv
rm -f /usr/local/bin/dbhome
rm -f /usr/local/bin/oraenv

-@Remove oracle user and group
#userdel -r oracle
#groupdel oinstall
#groupdel dba


++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++