作者：Digital Observer（施嘉伟）
Oracle ACE Pro: Database
PostgreSQL ACE Partner
11年数据库行业经验，现主要从事数据库服务工作
拥有Oracle OCM、DB2 10.1 Fundamentals、MySQL 8.0 OCP、WebLogic 12c OCA、KCP、PCTP、PCSD、OCI、PolarDB技术专家、达梦师资认证、数据安全咨询高级等认证
ITPUB认证专家、PolarDB开源社区技术顾问
国内某高校企业实践指导教师
公众号：Digital Observer；CSDN：施嘉伟；ITPUB：sjw1933；PGFans：施嘉伟。
作为一家乙方的oracle工程师，工作中会碰到很多客户rac添加节点的需求，常规的有以下几种情况：一个节点主机损坏、操作系统损坏、软件被误删除、新采购机器需要添加到集群等。在处理这些情况时，除了物理安装和基本配置外，还需要考虑一致性、网络配置、存储共享等多个方面。
本次分享的案例主要是在一切前期准备主机、操作系统、内核参数、共享存储都完成，在执行addNode.sh时候容易碰到的问题。
$ ./addNode.sh "CLUSTER_NEW_NODES={rac2}" "CLUSTER_NEW_VIRTUAL_HOSTNAMES={rac2-vip}"
出现如下报错：
PRCF-2022 : The file "/u01/11.2.0/grid/javavm/admin/classes.bin" became smaller while being transferred
PRCF-2022 : The file "/u01/11.2.0/grid/lib/libserver11.a" became smaller while being transferred
PRCF-2022 : The file "/u01/11.2.0/grid/install/usm/Novell/SLES11/x86_64/2.6.32.12-0.7/xen/bin/oracleacfs.ko" became smaller while being transferred
但此三个文件在节点2上又都可以找到，文件大小以及权限属性都和一节点一样，怀疑跟节点2有关（节点2本来是属于集群的，根据删除RAC节点删除节点2后，然后再添加的，可能跟这个有关）
解决方案：
在节点1的grid用户的$ORACLE_HOME/crs/install/install.excl文件中，添加如下：
javavm/admin/classes.bin
lib/libserver11.a
install/usm/Novell/SLES11/x86_64/2.6.32.12-0.7/xen/bin/oracleacfs.ko
不需要删除节点2上的grid目录，直接在节点1再跑一遍addnode.sh即可
出现报错：
PRKC-1025 : Failed to create a file under the filepath /opt because the filepath is not executable or writable
java.lang.OutOfMemoryError
解决办法：
Increase JRE_MEMORY_OPTIONS=" -mx1024m" or greater value in the oraparam.ini located in: $GRID_HOME/oui/
参考文档：Addnode.sh or Fresh Install Fails With PRKC-1025 and Java.Lang.OutOfMemoryError (文档 ID 1085893.1)
Performing tests to see whether nodes rac2,rac3 are available
.................SEVERE:OUI-35000: Fatal cluster error encountered (PRKN-1034 : Failed to retrieve IP address of host "rac2"). Correct the problem and try the operation again.
解决方案：
原因是由于inventory里面有rac的信息，根据实际根据更新inventory内容：
$ cd $ORACLE_HOME/oui/bin
$ ./runInstaller -updateNodeList ORACLE_HOME=<Oracle_home_location> "CLUSTER_NODES={remaining_node_list}" ---remaining_node_list表示所有其它节点
如果跑root.sh出现以下报错：
[root@hisrac01 grid]# /oracle/app/oraInventory/orainstRoot.sh
[064110]Start of resource "ora.cssd" failed
[064110]CRS-2672: Attempting to start 'ora.cssdmonitor' on 'hisrac01'
[064110]CRS-2672: Attempting to start 'ora.gipcd' on 'hisrac01'
[064110]CRS-2676: Start of 'ora.cssdmonitor' on 'hisrac01' succeeded
[064110]CRS-2676: Start of 'ora.gipcd' on 'hisrac01' succeeded
[064110]CRS-2672: Attempting to start 'ora.cssd' on 'hisrac01'
[064110]CRS-2672: Attempting to start 'ora.diskmon' on 'hisrac01'
[064110]CRS-2676: Start of 'ora.diskmon' on 'hisrac01' succeeded
[064110]CRS-2674: Start of 'ora.cssd' on 'hisrac01' failed
[064110]CRS-2679: Attempting to clean 'ora.cssd' on 'hisrac01'
[064110]CRS-2681: Clean of 'ora.cssd' on 'hisrac01' succeeded
[064110]CRS-2673: Attempting to stop 'ora.gipcd' on 'hisrac01'
[064110]CRS-2677: Stop of 'ora.gipcd' on 'hisrac01' succeeded
[064110]CRS-2673: Attempting to stop 'ora.cssdmonitor' on 'hisrac01'
[064110]CRS-2677: Stop of 'ora.cssdmonitor' on 'hisrac01' succeeded
[064110]CRS-5804: Communication error with agent process
[064110]CRS-4000: Command Start failed, or completed with errors.
[064110]Failed to start Oracle Grid Infrastructure stack
[064111]Failed to start Cluster Synchorinisation Service in clustered mode at /oracle/11.2.0/grid/crs/install/crsconfig_lib.pm line 1278.
[064111]/oracle/11.2.0/grid/perl/bin/perl -I/oracle/11.2.0/grid/perl/lib -I/oracle/11.2.0/grid/crs/install /oracle/11.2.0/grid/crs/install/rootcrs.pl execution failed
解决方案：
检查心跳网卡两边配置是否一样，子网掩码需要一致
如果跑root.sh出现以下报错：
[root@hisrac01 /]# /oracle/11.2.0/grid/root.sh
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Using configuration parameter file: /oracle/11.2.0/grid/crs/install/crsconfig_params
User ignored Prerequisites during installation
Installing Trace File Analyzer
OLR initialization - successful
Adding Clusterware entries to upstart
CRS-4402: The CSS daemon was started in exclusive mode but found an active CSS daemon on node hisrac02, number 2, and is terminating
An active cluster was found during exclusive startup, restarting to join the cluster
Start of resource "ora.asm" failed
CRS-2672: Attempting to start 'ora.drivers.acfs' on 'hisrac01'
CRS-2676: Start of 'ora.drivers.acfs' on 'hisrac01' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'hisrac01'
CRS-5017: The resource action "ora.asm start" encountered the following error:
ORA-48189: OS command to create directory failed
Linux-x86_64 Error: 13: Permission denied
Additional information: 2
. For details refer to "(:CLSN00107:)" in "/oracle/11.2.0/grid/log/hisrac01/agent/ohasd/oraagent_grid/oraagent_grid.log".
CRS-2674: Start of 'ora.asm' on 'hisrac01' failed
CRS-2679: Attempting to clean 'ora.asm' on 'hisrac01'
CRS-2681: Clean of 'ora.asm' on 'hisrac01' succeeded
CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'hisrac01'
CRS-2677: Stop of 'ora.drivers.acfs' on 'hisrac01' succeeded
CRS-4000: Command Start failed, or completed with errors.
Failed to start Oracle Grid Infrastructure stack
Failed to start ASM at /oracle/11.2.0/grid/crs/install/crsconfig_lib.pm line 1339.
/oracle/11.2.0/grid/perl/bin/perl -I/oracle/11.2.0/grid/perl/lib -I/oracle/11.2.0/grid/crs/install /oracle/11.2.0/grid/crs/install/rootcrs.pl execution failed
解决方案：
ORA-48141: error creating directory during ADR initialization [<oracle_base>/base/diag]
ORA-48189: OS command to create directory failed
创建$ORACLE_BASE/base/diag文件夹
报错如下：
PRCR-1079 : Failed to start resource ora.rac1.vip
CRS-2674: Start of 'ora.net1.network' on 'rac1' failed
CRS-2674: Start of 'ora.net1.network' on 'rac2' failed
CRS-2632: There are no more servers to try to place resource 'ora.rac1.vip' on that would satisfy its placement policy
start VIP on node:rac1 ... failed
Failed to perform new node configuration at /oracle/grid/crs_1/crs/install/crsconfig_lib.pm line 9209.
/oracle/grid/crs_1/perl/bin/perl -I/oracle/grid/crs_1/perl/lib -I/oracle/grid/crs_1/crs/install /oracle/grid/crs_1/crs/install/rootcrs.pl execution failed
解决方案：
这里是因为我在后面修改了rac2的NETMASK，虽然service network restart了，但是没有在集群里做修改，所以报了这个错，可以用下面的方法解决：
./srvctl modify nodeapps -n rac2 -A 10.0.0.101/255.255.255.0/eth0 ###public ip
./srvctl modify nodeapps -n rac2 -A 10.0.0.103/255.255.255.0/eth0 ###vip
修改完后，重启集群。
粽子香飘四溢，端午情深意浓。愿你幸福安康，快乐每一天！
教育的根是苦的，但其果实是甜的。
往期推荐:
点击"阅读原文"
原文作者：Digital Observer；如有侵权，请联系管理员删除。
上一篇：
Oracle鼓励ACE们展示自己!
下一篇：
我升级为Oracle ACE Pro了
请登录后发表评论
登录
全部评论
