solaris10 oracle10.2.0.2 RAC+asm


oracle@infodb2 $ls
4518443 OPatch p4898608_10202_GENERIC.zip
5077516 p4518443_10202_SOLARIS64.zip p5071492_10202_SOLARIS64.zip

oracle@infodb2 $ crs_stat -t
Name Type Target State Host
------------------------------------------------------------
ora....SM1.asm application ONLINE OFFLINE
ora....8A.lsnr application ONLINE OFFLINE
ora.infodb1.gsd application ONLINE OFFLINE
ora.infodb1.ons application ONLINE OFFLINE
ora.infodb1.vip application ONLINE ONLINE infodb2
ora....SM2.asm application ONLINE ONLINE infodb2
ora....8B.lsnr application ONLINE ONLINE infodb2
ora.infodb2.gsd application ONLINE ONLINE infodb2
ora.infodb2.ons application ONLINE ONLINE infodb2
ora.infodb2.vip application ONLINE ONLINE infodb2
ora.ivsp.db application ONLINE OFFLINE
ora....p1.inst application ONLINE OFFLINE
ora....p2.inst application ONLINE OFFLINE //实例无法启动

solaris10 oracle10.2.0.2 RAC+asm


oracle@infodb2 $ls
4518443 OPatch p4898608_10202_GENERIC.zip
5077516 p4518443_10202_SOLARIS64.zip p5071492_10202_SOLARIS64.zip

oracle@infodb2 $ crs_stat -t
Name Type Target State Host
------------------------------------------------------------
ora....SM1.asm application ONLINE OFFLINE
ora....8A.lsnr application ONLINE OFFLINE
ora.infodb1.gsd application ONLINE OFFLINE
ora.infodb1.ons application ONLINE OFFLINE
ora.infodb1.vip application ONLINE ONLINE infodb2
ora....SM2.asm application ONLINE ONLINE infodb2
ora....8B.lsnr application ONLINE ONLINE infodb2
ora.infodb2.gsd application ONLINE ONLINE infodb2
ora.infodb2.ons application ONLINE ONLINE infodb2
ora.infodb2.vip application ONLINE ONLINE infodb2
ora.ivsp.db application ONLINE OFFLINE
ora....p1.inst application ONLINE OFFLINE
ora....p2.inst application ONLINE OFFLINE //实例无法启动


查ASM LOG
Fri Jan 11 03:53:58 2008
NOTE: recovering COD for group 1/0x93802c0b (ASMDISK)
SUCCESS: completed COD recovery for group 1/0x93802c0b (ASMDISK)
Fri Jan 11 03:54:07 2008
Errors in file /data/oracle/admin/+ASM/udump/+asm2_ora_2590.trc:
ORA-07445: exception encountered: core dump [kkxsyn()+740] [SIGSEGV] [Address not mapped to object] [0x000000168] [] [] //关键错误信息
Fri Jan 11 03:54:08 2008



查metalink 发现是patch有bug导致
Subject: Bug 5128575 - RAC install of 10.2.0.2 does not update libknlopt.a on all nodes
Doc ID: Note:5128575.8 Type: PATCH
Last Revision Date: 02-NOV-2006 Status: PUBLISHED
Click here for details of sections in this note.

Bug 5128575 RAC install of 10.2.0.2 does not update libknlopt.a on all nodes
This note gives a brief overview of bug 5128575.
This bug is marked as an important issue.

Affects:
Product (Component) Oracle Universal Installer (PATCH)
Range of versions believed to be affected Versions >= 10.2.0.2
Versions confirmed as being affected 10.2.0.2

Platforms affected Generic (all / most platforms affected)

Fixed:
This issue is fixed in Bug not yet fixed / confirmed as fixed


Symptoms: Related To:
Install Is Not Performed Correctly
ORA-81
ORA-600 [1403]
Dump in or under kkxcms / kkxsyn
RAC (Real Application Clusters) / OPS
Oracle Univeral Installer


Description
Installing the 10.2.0.2 Patch Set in a RAC installation
on any Unix platform does not correctly update the libknlopt.a
file on all nodes. The local node where the installer is run
does update libknlopt.a but remote nodes do not get the
updated file.

This can lead to dumps or internal errors on the remote nodes
if Oracle is subsequently relinked.
eg: Attempts to use PLSQL may dump in kkxcms or kkxsyn

The workaround steps below should be considered as a mandatory
post install steps for the 10.2.0.2 Patch Set.

Workaround:
Following install of 10.2.0.2 on a RAC system:
Ensure all instances are shut down
Manually copy $ORACLE_HOME/rdbms/lib/libknlopt.a from the local node
to all remote nodes and relink Oracle on all nodes.
eg: cd $ORACLE_HOME/rdbms/lib
make -f ins_rdbms.mk ioracle



没办法
比较了一下 1;2机的 libknlopt.a 发觉2 机还是旧的日期和大小都和1 机的不一致
ftp 把1 机的libknlopt.a 弄过来重新make


oracle@infodb2 $/usr/ccs/bin/make -f ins_rdbms.mk ioracle
chmod 755 /data/oracle/OracleHome/bin

- Linking Oracle
rm -f /data/oracle/OracleHome/rdbms/lib/oracle
/usr/ccs/bin/ld -o /data/oracle/OracleHome/rdbms/lib/oracle -L/data/oracle/OracleHome/rdbms/lib/ -L/data/oracle/OracleHome/lib/ -dy /data/oracle/OracleHome/lib/prod/lib/v9/crti.o /data/oracle/OracleHome/lib/prod/lib/v9/crt1.o /data/oracle/OracleHome/rdbms/lib/opimai.o /data/oracle/OracleHome/rdbms/lib/ssoraed.o /data/oracle/OracleHome/rdbms/lib/ttcsoi.o /data/oracle/OracleHome/rdbms/lib/defopt.o -z allextract -lperfsrv10 -z defaultextract /data/oracle/OracleHome/lib/nautab.o /data/oracle/OracleHome/lib/naeet.o /data/oracle/OracleHome/lib/naect.o /data/oracle/OracleHome/lib/naedhs.o /data/oracle/OracleHome/rdbms/lib/config.o -lserver10 -lodm10 -lnnet10 -lskgxp10 -lsnls10 -lnls10 -lcore10 -lsnls10 -lnls10 -lcore10 -lsnls10 -lnls10 -lxml10 -lcore10 -lunls10 -lsnls10 -lnls10 -lcore10 -lnls10 -lhasgen10 -lcore10 -lskgxn2 -locr10 -locrb10 -locrutl10 -lhasgen10 -lcore10 -lskgxn2 -lclient10 -lvsn10 -lcommon10 -lgeneric10 -lknlopt `if /usr/ccs/bin/ar tv /data/oracle/OracleHome/rdbms/lib/libknlopt.a | grep xsyeolap.o > /dev/null 2>&1 ; then echo "-loraolap10" ; fi` -lslax10 -lpls10 -lplp10 -lserver10 -lclient10 -lvsn10 -lcommon10 -lgeneric10 -lknlopt -lslax10 -lpls10 -lplp10 -ljox10 -lserver10 -lclsra10 -ldbcfg10 -locijdbcst10 -lwwg `cat /data/oracle/OracleHome/lib/ldflags` -lnsslb10 -lncrypt10 -lnsgr10 -lnzjs10 -ln10 -lnnz10 -lnl10 -lnro10 `cat /data/oracle/OracleHome/lib/ldflags` -lnsslb10 -lncrypt10 -lnsgr10 -lnzjs10 -ln10 -lnnz10 -lnl10 -lmm -lsnls10 -lnls10 -lcore10 -lsnls10 -lnls10 -lcore10 -lsnls10 -lnls10 -lxml10 -lcore10 -lunls10 -lsnls10 -lnls10 -lcore10 -lnls10 `cat /data/oracle/OracleHome/lib/ldflags` -lnsslb10 -lncrypt10 -lnsgr10 -lnzjs10 -ln10 -lnnz10 -lnl10 -lnro10 `cat /data/oracle/OracleHome/lib/ldflags` -lnsslb10 -lncrypt10 -lnsgr10 -lnzjs10 -ln10 -lnnz10 -lnl10 -lsnls10 -lnls10 -lcore10 -lsnls10 -lnls10 -lcore10 -lsnls10 -lnls10 -lxml10 -lcore10 -lunls10 -lsnls10 -lnls10 -lcore10 -lnls10 `if /usr/ccs/bin/ar tv /data/oracle/OracleHome/rdbms/lib/libknlopt.a | grep "kxmnsd.o" > /dev/null 2>&1 ; then echo " " ; else echo "-lordsdo10"; fi` -lctxc10 -lctx10 -lzx10 -lgx10 -lctx10 -lzx10 -lgx10 -lordimt10 -lsnls10 -lnls10 -lcore10 -lsnls10 -lnls10 -lcore10 -lsnls10 -lnls10 -lxml10 -lcore10 -lunls10 -lsnls10 -lnls10 -lcore10 -lnls10 -lsnls10 -lunls10 -lsnls10 -lnls10 -lcore10 -lsnls10 -lnls10 -lcore10 -lsnls10 -lnls10 -lxml10 -lcore10 -lunls10 -lsnls10 -lnls10 -lcore10 -lnls10 `cat /data/oracle/OracleHome/lib/sysliblist` -R /opt/SUNWcluster/lib/sparcv9:/data/oracle/OracleHome/lib:/opt/ORCLcluster/lib/ -Y P,:/opt/SUNWcluster/lib/sparcv9:/opt/ORCLcluster/lib/:/usr/ccs/lib/sparcv9:/usr/lib/sparcv9 -Qy -lc -laio -lposix4 -lkstat -lm /data/oracle/OracleHome/lib/prod/lib/v9/crtn.o
mv -f /data/oracle/OracleHome/bin/oracle /data/oracle/OracleHome/bin/oracleO
mv /data/oracle/OracleHome/rdbms/lib/oracle /data/oracle/OracleHome/bin/oracle
chmod 6751 /data/oracle/OracleHome/bin/oracle


#/etc/init.d/init.crs start

oracle@infodb2 $crs_stat -t
Name Type Target State Host
------------------------------------------------------------
ora....SM1.asm application ONLINE ONLINE infodb1
ora....8A.lsnr application ONLINE ONLINE infodb1
ora.infodb1.gsd application ONLINE ONLINE infodb1
ora.infodb1.ons application ONLINE ONLINE infodb1
ora.infodb1.vip application ONLINE ONLINE infodb1
ora....SM2.asm application ONLINE ONLINE infodb2
ora....8B.lsnr application ONLINE ONLINE infodb2
ora.infodb2.gsd application ONLINE ONLINE infodb2
ora.infodb2.ons application ONLINE ONLINE infodb2
ora.infodb2.vip application ONLINE ONLINE infodb2
ora.ivsp.db application ONLINE ONLINE infodb1
ora....p1.inst application ONLINE ONLINE infodb1
ora....p2.inst application ONLINE ONLINE infodb2



故障排除 1、2机数据库正常启动