查看集群资源状态,节点node2的instance没能启动
[grid@node1 ~]$ crsctl status res -t
--------------------------------------------------------------------------------
NAME TARGET STATE SERVER STATE_DETAILS
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.ARCH.dg
ONLINE ONLINE node1
ONLINE ONLINE node2
ora.DATA.dg
ONLINE ONLINE node1
ONLINE ONLINE node2
ora.LISTENER.lsnr
ONLINE ONLINE node1
ONLINE ONLINE node2
ora.ORCVOTE.dg
ONLINE ONLINE node1
ONLINE ONLINE node2
ora.asm
ONLINE ONLINE node1 Started
ONLINE ONLINE node2 Started
ora.gsd
OFFLINE OFFLINE node1
OFFLINE OFFLINE node2
ora.net1.network
ONLINE ONLINE node1
ONLINE ONLINE node2
ora.ons
ONLINE ONLINE node1
ONLINE ONLINE node2
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
1 ONLINE ONLINE node1
ora.cvu
1 ONLINE ONLINE node1
ora.node1.vip
1 ONLINE ONLINE node1
ora.node2.vip
1 ONLINE ONLINE node2
ora.oc4j
1 ONLINE ONLINE node1
ora.prod.db
1 ONLINE ONLINE node1 Open
2 ONLINE OFFLINE Instance Shutdown
ora.scan1.vip
1 ONLINE ONLINE node1
在节点node2 以oracle用户登录sqlplus
数据实例是idle的,尝试startup启动出错“ORA-01102: cannot mount database in EXCLUSIVE mode”
[oracle@node2 ~]$ sqlplus / as sysdba
SQL*Plus: Release 11.2.0.3.0 Production on Tue Jul 1 23:20:56 2014
Copyright (c) 1982, 2011, Oracle. All rights reserved.
Connected to an idle instance.
SYS@PROD2> startup
ORA-01102: cannot mount database in EXCLUSIVE mode
SYS@PROD2>
查看alert日志
[oracle@node2 ~]$ vi /s01/diag/rdbms/prod/PROD2/trace/alert_PROD2.log
…..
…...
Tue Jul 01 23:21:10 2014
MARK started with pid=26, OS id=4898
NOTE: MARK has subscribed
starting up 1 shared server(s) ...
lmon registered with NM - instance number 2 (internal mem no 1)
USER (ospid: 4743): terminating the instance due to error 1102
Instance terminated by USER, pid = 4743
用健康的节点node1登录,发现节点node1的实例PROD1的cluster_database属性为false !
[oracle@node1 ~]$ sqlplus / as sysdba
SQL*Plus: Release 11.2.0.3.0 Production on Tue Jul 1 23:34:10 2014
Copyright (c) 1982, 2011, Oracle. All rights reserved.
Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,
Data Mining and Real Application Testing options
SYS@PROD1> show parameter cluster_database;
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
cluster_database boolean FALSE
cluster_database_instances integer 1
修改spfile中的参数,初步断定问题的原因是此处导致!
SYS@PROD1> alter system set cluster_database=TRUE scope=spfile sid='PROD1';
System altered.
SYS@PROD1> alter system set cluster_database_instances=2 scope=spfile sid=‘PROD1';
System altered.
退出sqlplus,su到grid用户下关闭数据库
[grid@node1 ~]$ srvctl stop database -d prod;
查看集群资源状态,两个节点的数据库实例都是shutdown的状态
[grid@node1 ~]$ crsctl status res -t
--------------------------------------------------------------------------------
NAME TARGET STATE SERVER STATE_DETAILS
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.ARCH.dg
ONLINE ONLINE node1
ONLINE ONLINE node2
ora.DATA.dg
ONLINE ONLINE node1
ONLINE ONLINE node2
ora.LISTENER.lsnr
ONLINE ONLINE node1
ONLINE ONLINE node2
ora.ORCVOTE.dg
ONLINE ONLINE node1
ONLINE ONLINE node2
ora.asm
ONLINE ONLINE node1 Started
ONLINE ONLINE node2 Started
ora.gsd
OFFLINE OFFLINE node1
OFFLINE OFFLINE node2
ora.net1.network
ONLINE ONLINE node1
ONLINE ONLINE node2
ora.ons
ONLINE ONLINE node1
ONLINE ONLINE node2
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
1 ONLINE ONLINE node1
ora.cvu
1 ONLINE ONLINE node1
ora.node1.vip
1 ONLINE ONLINE node1
ora.node2.vip
1 ONLINE ONLINE node2
ora.oc4j
1 ONLINE ONLINE node1
ora.prod.db
1 OFFLINE OFFLINE Instance Shutdown
2 ONLINE OFFLINE Instance Shutdown
ora.scan1.vip
1 ONLINE ONLINE node1
重新启动数据库,启动成功!
[grid@node1 ~]$ srvctl stop database -d prod;
[grid@node1 ~]$ crsctl status res -t
--------------------------------------------------------------------------------
NAME TARGET STATE SERVER STATE_DETAILS
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.ARCH.dg
ONLINE ONLINE node1
ONLINE ONLINE node2
ora.DATA.dg
ONLINE ONLINE node1
ONLINE ONLINE node2
ora.LISTENER.lsnr
ONLINE ONLINE node1
ONLINE ONLINE node2
ora.ORCVOTE.dg
ONLINE ONLINE node1
ONLINE ONLINE node2
ora.asm
ONLINE ONLINE node1 Started
ONLINE ONLINE node2 Started
ora.gsd
OFFLINE OFFLINE node1
OFFLINE OFFLINE node2
ora.net1.network
ONLINE ONLINE node1
ONLINE ONLINE node2
ora.ons
ONLINE ONLINE node1
ONLINE ONLINE node2
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
1 ONLINE ONLINE node1
ora.cvu
1 ONLINE ONLINE node1
ora.node1.vip
1 ONLINE ONLINE node1
ora.node2.vip
1 ONLINE ONLINE node2
ora.oc4j
1 ONLINE ONLINE node1
ora.prod.db
1 ONLINE ONLINE node1 Open
2 ONLINE ONLINE node2 Open
ora.scan1.vip
1 ONLINE ONLINE node1
查看参数,spfile已经修改成功,cluster_database 参数为TRUE
SYS@PROD1> show parameter cluster_database
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
cluster_database boolean TRUE
cluster_database_instances integer 2