前言：oracle 11g rac安装部署前些年，做了好多遍，最近熟人找我做rac数据迁移，感觉手都生了，平时由于事务性的工作比较多，忽略了动手能力，本段时间将之前的部署及问题解决的工作进行总结，为后期项目中的工作服务。

Oracle 11g rac for linux 安装文档(ORHEL7.8)

1 、操作系统 Oracle linux 7.8

2 、数据库 Oracle 11.2.0.4 grid database

3 、网络拓扑

4 、地址分配

节点名称	物理网卡（Public）	物理网卡(Private)	虚拟地址vip	rac-scan
rac1	10.10.10.11	11.11.11.11	10.10.10.13	10.10.10.15
rac2	10.10.10.12	11.11.11.12	10.10.10.14	10.10.10.15

5 、磁盘规划（持久存储）

名称	大小	备注
OCR01	10G	表决盘1 sdb1
OCR02	10G	表决盘2 sdc1
OCR03	10G	表决盘3 sdd1
DATA01	20G	数据盘1 sde1
DATA02	20G	数据盘2 sdf1
DATA03	20G	数据盘3 sdg1
FLA01	20G	归档盘1 sdh1

添加存储磁盘空间：
备注：本次实验操作是在虚拟机环境，采用的VMware Workstation平台；

添加磁盘并完成磁盘分区！

[root@rac1 ~]# fdisk -l

Disk /dev/sdb: 10.7 GB, 10737418240 bytes, 20971520 sectors

Units = sectors of 1 * 512 = 512 bytes

Sector size (logical/physical): 512 bytes / 512 bytes

I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk label type: dos

Disk identifier: 0x6aa9296e

Device Boot Start End Blocks Id System

/dev/sdb1 2048 20971519 10484736 83 Linux

Disk /dev/sdc: 10.7 GB, 10737418240 bytes, 20971520 sectors

Units = sectors of 1 * 512 = 512 bytes

Sector size (logical/physical): 512 bytes / 512 bytes

I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk label type: dos

Disk identifier: 0x835a768f

Device Boot Start End Blocks Id System

/dev/sdc1 2048 20971519 10484736 83 Linux

Disk /dev/sdd: 10.7 GB, 10737418240 bytes, 20971520 sectors

Units = sectors of 1 * 512 = 512 bytes

Sector size (logical/physical): 512 bytes / 512 bytes

I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk label type: dos

Disk identifier: 0x74767fc5

Device Boot Start End Blocks Id System

/dev/sdd1 2048 20971519 10484736 83 Linux

Disk /dev/sde: 21.5 GB, 21474836480 bytes, 41943040 sectors

Units = sectors of 1 * 512 = 512 bytes

Sector size (logical/physical): 512 bytes / 512 bytes

I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk label type: dos

Disk identifier: 0xf8637c08

Device Boot Start End Blocks Id System

/dev/sde1 2048 41943039 20970496 83 Linux

Disk /dev/sdf: 21.5 GB, 21474836480 bytes, 41943040 sectors

Units = sectors of 1 * 512 = 512 bytes

Sector size (logical/physical): 512 bytes / 512 bytes

I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk label type: dos

Disk identifier: 0xe1fa754d

Device Boot Start End Blocks Id System

/dev/sdf1 2048 41943039 20970496 83 Linux

Disk /dev/sdg: 21.5 GB, 21474836480 bytes, 41943040 sectors

Units = sectors of 1 * 512 = 512 bytes

Sector size (logical/physical): 512 bytes / 512 bytes

I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk label type: dos

Disk identifier: 0x7fa6121c

Device Boot Start End Blocks Id System

/dev/sdg1 2048 41943039 20970496 83 Linux

Disk /dev/sdh: 21.5 GB, 21474836480 bytes, 41943040 sectors

Units = sectors of 1 * 512 = 512 bytes

Sector size (logical/physical): 512 bytes / 512 bytes

I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk label type: dos

Disk identifier: 0x8c0c3532

Device Boot Start End Blocks Id System

/dev/sdh1 2048 41943039 20970496 83 Linux

①在虚拟化环境下使用两台机器使用相同的共享磁盘，会出现报错的情况，造成两台机器无法正常开机的情况；

需要编辑虚拟机.vmx配置文件：

编辑ORHEL.vmx添加文件末尾

disk.locking = "FALSE"

diskLib.dataCacheMaxSize = "0"

diskLib.dataCacheMaxReadAheadSize = "0"

diskLib.dataCacheMinReadAheadSize = "0"

diskLib.dataCachePageSize = "4096"

diskLib.maxUnsyncedWrites = "0"

②记录磁盘UUID，防止在重启后磁盘出现乱序的情况：

/usr/lib/udev/scsi_id -g -u -d /dev/sdc （VMware workstation下可以回显磁盘uuid，在Esxi环境下返回空值）

需要在虚拟机配置文件.vmx中添加参数：disk.EnableUUID = "TRUE"

[root@rac1 rules.d]# /usr/lib/udev/scsi_id -g -u -d /dev/sdc

36000c298341d440bdc5dda2b557d1ae3

也可以使用脚本将本次所有的磁盘全部写到99-oracle-asmdevices.rules文件中

for i in b c d e f h g;

do

echo "KERNEL==\"sd?\", SUBSYSTEM==\"block\", PROGRAM==\"/usr/lib/udev/scsi_id -g -u -d /dev/\$name\", RESULT==\"`/usr/lib/udev/scsi_id -g -u -d /dev/sd\$i`\", SYMLINK+=\"asmdisk$i\", OWNER=\"grid\", GROUP=\"asmadmin\", MODE=\"0660\"" >> /etc/udev/rules.d/99-oracle-asmdevices.rules

done

③udevadm配置重载生效：

[root@rac1 rules.d]# udevadm control --reload

/sbin/udevadm trigger --type=devices --action=change

确认udev绑定成功，已经生产绑定后的设备：

[root@rac1 rules.d]# ls -l /dev/asm*

brw-rw----. 1 grid asmadmin 8, 16 Apr 13 09:32 /dev/asmdiska

brw-rw----. 1 grid asmadmin 8, 32 Apr 13 09:32 /dev/asmdiskb

brw-rw----. 1 grid asmadmin 8, 48 Apr 13 09:32 /dev/asmdiskc

brw-rw----. 1 grid asmadmin 8, 64 Apr 13 09:32 /dev/asmdiskd

brw-rw----. 1 grid asmadmin 8, 80 Apr 13 09:32 /dev/asmdiske

brw-rw----. 1 grid asmadmin 8, 96 Apr 13 09:32 /dev/asmdiskf

brw-rw----. 1 grid asmadmin 8, 112 Apr 13 09:32 /dev/asmdiskg

备注：另外一个节点可以直接拷贝配置文件到相应目录

6 、安装前的系统配置（rac1、rac2）

①主机hosts文件/etc/hosts

[root@rac1 ~]# cat /etc/hosts

127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4

::1 localhost localhost.localdomain localhost6 localhost6.localdomain6

#ens32 public-ip

10.10.10.11 rac1

10.10.10.12 rac2

#public-vip

10.10.10.13 rac1-vip

10.10.10.14 rac2-vip

#ens35 ens36 private-ip

11.11.11.11 rac1-priv

11.11.11.12 rac2-priv

#scan-ip

10.10.10.15 rac-scan

10.10.10.20 mirrordb #(ADG 主机)

②修改资源限制参数（rac1、rac2）

cat /etc/security/limits.conf

oracle soft nproc 2047

oracle hard nproc 16384

oracle soft nofile 1024

oracle hard nofile 65536

oracle soft stack 10240

grid soft nproc 2047

grid hard nproc 16384

grid soft nofile 1024

grid hard nofile 65536

grid soft stack 10240

③修改nproc参数（rac1、rac2）

[root@rac1 ~]# cat /etc/security/limits.d/20-nproc.conf

# Default limit for number of user's processes to prevent

# accidental fork bombs.

# See rhbz #432903 for reasoning.

* soft nproc unlimited

控制用户分配的资源（使上面配置生效）

echo "session required pam_limits.so" >> /etc/pam.d/login

[root@rac1 ~]# cat /etc/pam.d/login

#%PAM-1.0

auth [user_unknown=ignore success=ok ignore=ignore default=bad] pam_securetty.so

auth substack system-auth

auth include postlogin

account required pam_nologin.so

account include system-auth

password include system-auth

# pam_selinux.so close should be the first session rule

session required pam_selinux.so close

session required pam_loginuid.so

session optional pam_console.so

# pam_selinux.so open should only be followed by sessions to be executed in the user context

session required pam_selinux.so open

session required pam_namespace.so

session optional pam_keyinit.so force revoke

session include system-auth

session include postlogin

-session optional pam_ck_connector.so

session required pam_limits.so

④修改内核参数（rac1、rac2）

[root@rac1 ~]# cat /etc/sysctl.conf

# sysctl settings are defined through files in

# /usr/lib/sysctl.d/, /run/sysctl.d/, and /etc/sysctl.d/.

#

# Vendors settings live in /usr/lib/sysctl.d/.

# To override a whole file, create a new file with the same in

# /etc/sysctl.d/ and put new settings there. To override

# only specific settings, add a file with a lexically later

# name in /etc/sysctl.d/ and put new settings there.

#

# For more information, see sysctl.conf(5) and sysctl.d(5).

fs.aio-max-nr = 1048576

fs.file-max = 6815744

kernel.shmall = 2097152

kernel.shmmax = 4294967295

kernel.shmmni = 4096

# semaphores: semmsl, semmns, semopm, semmni

kernel.sem = 250 32000 100 128

net.ipv4.ip_local_port_range = 9000 65500

net.core.rmem_default=262144

net.core.rmem_max=4194304

net.core.wmem_default=262144

net.core.wmem_max=1048586

⑤关闭透明页，numa功能（rac1、rac2）

cd /etc/default/

cp grub grub.bak

vi grub

将GRUB_CMDLINE_LINUX="crashkernel=auto spectre_v2=retpoline rhgb quiet" 修改为

GRUB_CMDLINE_LINUX="rhgb quiet transparent_hugepage=never"

grub2-mkconfig -o /boot/grub2/grub.cfg

reboot 重启操作系统

检查生效：

[root@rac1 ~]# grep AnonHugePages /proc/meminfo

AnonHugePages: 0 kB

关闭numa功能

[root@rac1 ~]# vi /etc/default/grub

GRUB_TIMEOUT=5

GRUB_DISTRIBUTOR="$(sed 's, release .*$,,g' /etc/system-release)"

GRUB_DEFAULT=saved

GRUB_DISABLE_SUBMENU=true

GRUB_TERMINAL_OUTPUT="console"

GRUB_CMDLINE_LINUX="crashkernel=auto rhgb quiet transparent_hugepage=never numa=off"

GRUB_DISABLE_RECOVERY="true"

重新编译grub2-mkconfig -o /etc/grub2.cfg

检查生效：（已经关闭）

[root@rac1 ~]# grep -i numa /var/log/dmesg

[ 0.000000] Command line: BOOT_IMAGE=/vmlinuz-4.14.35-1902.300.11.el7uek.x86_64 root=/dev/mapper/ol-root ro crashkernel=auto rhgb quiet transparent_hugepage=never numa=off

[ 0.000000] NUMA turned off

[ 0.000000] Kernel command line: BOOT_IMAGE=/vmlinuz-4.14.35-1902.300.11.el7uek.x86_64 root=/dev/mapper/ol-root ro crashkernel=auto rhgb quiet transparent_hugepage=never numa=off

备注：以下为开启numa状态

[root@rac2 bin]# grep -i numa /var/log/dmesg

[ 0.000000] NUMA: Node 0 [mem 0x00000000-0x0009ffff] + [mem 0x00100000-0xbfffffff] -> [mem 0x00000000-0xbfffffff]

[ 0.000000] NUMA: Node 0 [mem 0x00000000-0xbfffffff] + [mem 0x100000000-0x23fffffff] -> [mem 0x00000000-0x23fffffff]

⑥操作系统图形、字符启动模式（不必修改）

[root@rac2 bin]# systemctl get-default

graphical.target

设置字符启动模式

systemctl set-default multi-user.target （重启生效）

⑦修改共享内存段（rac1、rac2）

echo "shmfs /dev/shm tmpfs defaults,size=8g 0 0" >>/etc/fstab
[root@rac1 ~]# mount -a

⑧修改其他参数（rac1、rac2）

修改时区：

timedatectl set-timezone Asia/Shanghai

[root@rac1 ~]# timedatectl

Local time: Sun 2025-04-13 11:02:16 CST

Universal time: Sun 2025-04-13 03:02:16 UTC

RTC time: Sun 2025-04-13 03:02:12

Time zone: Asia/Shanghai (CST, +0800)

NTP enabled: no

NTP synchronized: no

RTC in local TZ: no

DST active: n/a

备注：有些发行版本修改

ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime

[root@rac1 ~]# hwclock -w

-s, --hctosys set the system time from the hardware clock

-w, --systohc set the hardware clock from the current system time

禁用SELINUX：

[root@rac1 ~]# getenforce

Permissive

[root@rac1 ~]# cat /etc/selinux/config

# This file controls the state of SELinux on the system.

# SELINUX= can take one of these three values:

# enforcing - SELinux security policy is enforced.

# permissive - SELinux prints warnings instead of enforcing.

# disabled - No SELinux policy is loaded.

SELINUX=permissive

# SELINUXTYPE= can take one of three values:

# targeted - Targeted processes are protected,

# minimum - Modification of targeted policy. Only selected processes are protected.

# mls - Multi Level Security protection.

SELINUXTYPE=targeted

关闭防火墙：

[root@rac1 ~]# systemctl stop firewalld.service

[root@rac1 ~]# systemctl status firewalld.service

● firewalld.service - firewalld - dynamic firewall daemon

Loaded: loaded (/usr/lib/systemd/system/firewalld.service; disabled; vendor preset: enabled)

Active: inactive (dead)

Docs: man:firewalld(1)

配置NOZEROCONF:（指在系统中禁用零配置网络，所有网络设置需要手动配置）

echo “NOZEROCONF=yes” >> /etc/sysconfig/network

[root@rac1 ~]# cat /etc/sysconfig/network

# Created by anaconda

NOZEROCONF=yes

修改/etc/nsswitch.conf文件：（保存域名检索顺序）

hosts: files dns myhostname nis

关闭不需要的服务：

avahi-daemon 守护进程配合缓存用户程序的答复，以帮助减少因答复而产生的网络流量；根据官方文档列出来的BUG，建议关闭这个进程。

Linux7 中：

systemctl stop avahi-daemon.socket avahi-daemon.service

systemctl disable avahi-daemon.socket avahi-daemon.service

关闭postfix邮件服务：

[root@rac1 ~]# systemctl stop postfix.service

[root@rac1 ~]# systemctl disable postfix.service

Removed symlink /etc/systemd/system/multi-user.target.wants/postfix.service.

禁用ntp服务：

[root@rac1 system]# systemctl stop chronyd.service

[root@rac1 system]# systemctl disable chronyd.service

[root@rac1 system]# systemctl status chronyd.service

● chronyd.service - NTP client/server

Loaded: loaded (/usr/lib/systemd/system/chronyd.service; disabled; vendor preset: enabled)

Active: inactive (dead)

Docs: man:chronyd(8)

man:chrony.conf(5)

[root@rac1 system]# mv /etc/chrony.conf /etc/chrony.conf.bak

备注：关于时间同步服务-redhat、centos、orhel 5/6使用ntp服务，7/8使用的是cherry服务。

配置limits等参数文件信息

vi /etc/profile

if [ $USER = "oracle" ] || [ $USER = "grid" ]; then

if [ $SHELL = "/bin/ksh" ]; then

ulimit -p 16384

ulimit -n 65536

else

ulimit -u 16384 -n 65536

fi

umask 022

fi

⑨创建用户和用户组，并设置用户密码（rac1、rac2）

groupadd oinstall

groupadd dba

groupadd oper

groupadd asmadmin

groupadd asmdba

groupadd asmoper

[root@rac1 ~]# useradd -g oinstall -G dba,asmdba,asmadmin,asmoper grid

[root@rac1 ~]# useradd -g oinstall -G dba,oper,asmdba oracle

设置grid和oracle密码： grid oracle

passwd grid

passwd oracle

⑩创建目录和配置环境变量（rac1、rac2）

创建grid、oracle目录：

mkdir -p /u01/app/11.2.0/grid

mkdir -p /u01/app/grid

mkdir -p /u01/app/oracle

chown grid:oinstall /u01/app/11.2.0/grid

chown grid:oinstall /u01/app/grid

chown -R oracle:oinstall /u01/app/oracle

chmod -R 775 /u01/

chown -R grid:oinstall /u01

创建oraInventory目录：

mkdir -p /u01/app/oraInventory

chown -R grid:oinstall /u01/app/oraInventory

chmod -R 775 /u01/app/oraInventory

创建oracle目录：

mkdir -p /u01/app/oracle

mkdir -p /u01/app/oracle/cfgtoollogs

chown -R oracle:oinstall /u01/app/oracle

chmod -R 775 /u01/app/oracle

mkdir -p /u01/app/oracle/product/11.2.0/db_1

chown -R oracle:oinstall /u01/app/oracle/product/11.2.0/db_1

chmod -R 775 /u01/app/oracle/product/11.2.0/db_1

配置grid用户环境变量：（rac1、rac2）

su – grid

vi .bash_profile

节点rac1：

export ORACLE_BASE=/u01/app/grid

export ORACLE_HOME=/u01/app/11.2.0/grid

export ORACLE_SID=+ASM1

export PATH=$ORACLE_HOME/bin:$ORACLE_HOME/OPatch:$PATH

export LANG=”English”

umask 022

source /home/grid/.bash_profile # 使生效

配置oracle用户环境变量：（rac1、rac2）

su - oracle

vi /home/oracle/.bash_profile

节点rac01：

export ORACLE_BASE=/u01/app/oracle

export ORACLE_UNQNAME=orcl

export ORACLE_SID=orcl1

export ORACLE_HOME=$ORACLE_BASE/product/11.2.0/db_1

export PATH=$PATH:$ORACLE_HOME/bin

export LD_LIBARY_PATH=$ORACLE_HOME/lib:$LD_LIBRARY_PATH

export export

umask 022

source /home/oracle/.bash_profile # 使生效

7 、grid图形化安装

①配置远程安装工具vncserver（确保应用安装不中断，建议安装vnc工具）

配置yum库：（上传本地镜像或挂载本地ios文件）

[root@rac1 ~]# mount /dev/sr0 /mnt

mount: /dev/sr0 is write-protected, mounting read-only

[root@rac1 ~]# df -hT

Filesystem Type Size Used Avail Use% Mounted on

devtmpfs devtmpfs 3.8G 0 3.8G 0% /dev

tmpfs tmpfs 3.8G 9.7M 3.8G 1% /run

tmpfs tmpfs 3.8G 0 3.8G 0% /sys/fs/cgroup

/dev/mapper/ol-root xfs 180G 23G 158G 13% /

/dev/sda1 xfs 2.0G 228M 1.8G 12% /boot

tmpfs tmpfs 769M 12K 769M 1% /run/user/42

tmpfs tmpfs 769M 0 769M 0% /run/user/0

shmfs tmpfs 8.0G 0 8.0G 0% /dev/shm

/dev/sr0 iso9660 4.5G 4.5G 0 100% /mnt

编辑yum文件：

[root@rac1 ~]# ll /etc/yum.repos.d/rhel.repo

-rw-r--r--. 1 root root 98 Mar 3 22:24 /etc/yum.repos.d/rhel.repo

[root@rac1 ~]# cat /etc/yum.repos.d/rhel.repo

[ORHEL]

name=ORHEL

baseurl=file:///mnt

gpgkey=file:///mnt/RPM-GPG-KEY-oracle

gpgcheck=0

enabled=1

安装vncserver服务端：（rac1或rac2安装任意节点即可）

安装vnc服务端（安装linux oracle过程中使用vnc，不使用其他的，容易造成安装中断）

yum install tigervnc tigervnc-server

[root@rac1 ~]# cat /etc/sysconfig/vncservers

# THIS FILE HAS BEEN REPLACED BY /lib/systemd/system/vncserver@.serviceG

VNCSERVERS="1:root 2:grid 3:oracle"

VNCSERVERARGS[1]="-geometry 1024x768 -alwaysshared -depth 24"

VNCSERVERARGS[2]="-geometry 1024x768 -alwaysshared -depth 24"

VNCSERVERARGS[3]="-geometry 1024x768 -alwaysshared -depth 24"

vncpasswd

vncserver

备注：不同的用户启用不同终端号码（vncserver:1）

检查防火墙状态：firewall-cmd --state

检查防火墙条目：firewall-cmd --list-all

安装oracle完成后，关闭vncserver进程

[root@db01 ~]# vncserver -list

TigerVNC server sessions:

X DISPLAY # PROCESS ID

:1 8448

:2 9251

[root@db01 ~]# vncserver -kill :1

Killing Xvnc process ID 8448

[root@db01 ~]# vncserver -kill :2

Killing Xvnc process ID 9251

systemctl stop vncserver@:1.service 或者 vncserver -kill :1

systemctl disenable vncserver@:1.service

vncserver 故障问题解决：

a 、vncserver远程桌面无法打开terminal终端？

解答：未启动startx图形化界面所导致，并非vncserver导致。

[oracle@mirrordb ~]$ cat .bashrc

# .bashrc

# Source global definitions

if [ -f /etc/bashrc ]; then

. /etc/bashrc

fi

export LC_ALL=C （添加本行参数即可）

b 、oracle安装过程中确认对话界面小，无法操作

解答：

[oracle@mirrordb database]$ ./runInstaller -jreloc /etc/alternatives/jre

②安装数据库集群中依赖的rpm包（rac1、rac2）

vi pack.sh

#!bin/bash

#start install required packages

echo "start installing required packages"

yum -y install binutils-*

yum -y install compat-libstdc++-*

yum -y install elfutils-libelf-*

yum -y install gcc-*

yum -y install glibc*

yum -y install pdksh*

yum -y install libaio-*

yum -y install libgcc-*

yum -y install libstdc++-*

yum -y install make-*

yum -y install sysstat-*

yum -y install unixODBC-*

yum -y install compat-libcap*

yum -y install compat-db*

yum -y install readline-devel*

yum -y install libXp*

yum -y install compat-gcc*

yum -y install ncurses-devel*

yum -y install celt051*

备注：缺少包，后期检测失败再上传。

③图形化安装grid集群

上传包并解压： unzip "*.zip" #解压软件

[root@rac1 u01]# du -sh *

8.7G app

2.5G database

1.3G grid

[root@rac1 u01]# ll （设定上传文件目录权限）

total 8

drwxr-xr-x. 6 root oinstall 66 Mar 3 22:15 app

drwxrwxr-x. 7 oracle oinstall 4096 Aug 27 2013 database

drwxrwxr-x. 7 grid oinstall 4096 Aug 27 2013 grid

解决图形框小问题：在安装的时候后面添加 -jreloc /etc/alternatives/jre 即可！

测试互信任状态：（避免安装软件过程中因为互信问题导致报错）

[grid@rac1 ~]$ ssh rac2 date

Wed Mar 5 04:15:29 CST 2025

[grid@rac1 ~]$ ssh rac2-priv date

Wed Mar 5 04:15:32 CST 2025

[grid@rac2 ~]$ ssh rac1 date

Wed Mar 5 04:15:22 CST 2025

[grid@rac2 ~]$ ssh rac1-priv date

Wed Mar 5 04:16:30 CST 2025

解决办法：互信问题！

上传缺少的系统缺少的rpm安装包；

报错问题：

Failed to start the Clusterware. Last 20 lines of the alert log follow: 2025-03-05 05:45:21.516:

[client(20511)]CRS-2101:The OLR was formatted using version 3.

解决：1、安装必要的依赖包

yum install perl-Env

2 、注意用root用户操作，回退root.sh 脚本操作

ORACLE_HOME=/u01/app/grid/11.2.0/

/u01/app/11.2.0/grid/crs/install/roothas.pl -deconfig -force -verbose （回退脚本）

[root@rac1 ~]# /u01/app/11.2.0/grid/crs/install/roothas.pl -deconfig -force -verbose

Using configuration parameter file: /u01/app/11.2.0/grid/crs/install/crsconfig_params

CRS-4535: Cannot communicate with Cluster Ready Services

CRS-4000: Command Stop failed, or completed with errors.

CRS-4535: Cannot communicate with Cluster Ready Services

CRS-4000: Command Delete failed, or completed with errors.

CRS-4544: Unable to connect to OHAS

CRS-4000: Command Stop failed, or completed with errors.

Successfully deconfigured Oracle Restart stack

3 、创建服务ohas.service

创建服务ohas.service的服务文件并赋予权限

cp /u01/app/11.2.0/grid/crs/init/init.ohasd /etc/init.d/init.ohasd

touch /usr/lib/systemd/system/ohas.service

chmod 777 /usr/lib/systemd/system/ohas.service

cat >> /usr/lib/systemd/system/ohas.service <<EOF

[Unit]

Description=Oracle High Availability Services

After=syslog.target

[Service]

ExecStart=/etc/init.d/init.ohasd run >/dev/null 2>&1 Type=simple

Restart=always

[Install]

WantedBy=multi-user.target

EOF

加载、启动服务

systemctl daemon-reload

systemctl enable ohas.service

systemctl start ohas.service

查看ohas服务状态

systemctl status ohas.service( 状态显示失败)

备注：通过执行2个脚本后，执行回退配置，启动ohas.service服务，再执行2个脚本，即可完成grid安装！

节点2继续执行以上步骤：

集群安装完成：

8 、安装oracle软件、建库（图形化）

测试节点间的互信状态；

解决：修改$ORACLE_HOME/sysman/lib/ins_emagent.mk，将$(MK_EMAGENT_NMECTL)修改为：$(MK_EMAGENT_NMECTL) -lnnz11

建磁盘组：

建库：

至此，11g rac集群安装完成！

9 、关闭数据库服务器过程

①节点运行情况

[oracle@rac1 ~]$ srvctl status database -d orcl

Instance orcl1 is running on node rac1

Instance orcl2 is running on node rac2

②关闭数据库

[root@rac1 ~]# su - oracle

Last login: Sun Apr 13 07:13:59 CST 2025 on pts/1

[oracle@rac1 ~]$ srvctl status database -d orcl

Instance orcl1 is running on node rac1

Instance orcl2 is running on node rac2

[oracle@rac1 ~]$ srvctl stop database -d orcl

[oracle@rac1 ~]$ srvctl status database -d orcl

Instance orcl1 is not running on node rac1

Instance orcl2 is not running on node rac2

[oracle@rac1 ~]$ sqlplus / as sysdba

SQL*Plus: Release 11.2.0.4.0 Production on Sun Apr 13 15:23:00 2025

Connected to an idle instance.

SQL>

③关闭集群服务（rac，rac2）

[root@rac1 ~]# cd /u01/app/11.2.0/grid/bin/

[root@rac1 bin]# ./crsctl stop has

CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'rac1'

CRS-2673: Attempting to stop 'ora.crsd' on 'rac1'

CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on 'rac1'

CRS-2673: Attempting to stop 'ora.LISTENER.lsnr' on 'rac1'

CRS-2673: Attempting to stop 'ora.DATA.dg' on 'rac1'

CRS-2673: Attempting to stop 'ora.FLA.dg' on 'rac1'

CRS-2673: Attempting to stop 'ora.OCR.dg' on 'rac1'

CRS-2677: Stop of 'ora.LISTENER.lsnr' on 'rac1' succeeded

CRS-2673: Attempting to stop 'ora.rac1.vip' on 'rac1'

CRS-2677: Stop of 'ora.DATA.dg' on 'rac1' succeeded

CRS-2677: Stop of 'ora.FLA.dg' on 'rac1' succeeded

CRS-2677: Stop of 'ora.rac1.vip' on 'rac1' succeeded

CRS-2672: Attempting to start 'ora.rac1.vip' on 'rac2'

CRS-2676: Start of 'ora.rac1.vip' on 'rac2' succeeded

CRS-2677: Stop of 'ora.OCR.dg' on 'rac1' succeeded

CRS-2673: Attempting to stop 'ora.asm' on 'rac1'

CRS-2677: Stop of 'ora.asm' on 'rac1' succeeded

CRS-2673: Attempting to stop 'ora.ons' on 'rac1'

CRS-2677: Stop of 'ora.ons' on 'rac1' succeeded

CRS-2673: Attempting to stop 'ora.net1.network' on 'rac1'

CRS-2677: Stop of 'ora.net1.network' on 'rac1' succeeded

CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'rac1' has completed

CRS-2677: Stop of 'ora.crsd' on 'rac1' succeeded

CRS-2673: Attempting to stop 'ora.ctssd' on 'rac1'

CRS-2673: Attempting to stop 'ora.evmd' on 'rac1'

CRS-2673: Attempting to stop 'ora.asm' on 'rac1'

CRS-2673: Attempting to stop 'ora.mdnsd' on 'rac1'

CRS-2677: Stop of 'ora.evmd' on 'rac1' succeeded

CRS-2677: Stop of 'ora.mdnsd' on 'rac1' succeeded

CRS-2677: Stop of 'ora.asm' on 'rac1' succeeded

CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'rac1'

CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'rac1' succeeded

CRS-2677: Stop of 'ora.ctssd' on 'rac1' succeeded

CRS-2673: Attempting to stop 'ora.cssd' on 'rac1'

CRS-2677: Stop of 'ora.cssd' on 'rac1' succeeded

CRS-2673: Attempting to stop 'ora.crf' on 'rac1'

CRS-2677: Stop of 'ora.crf' on 'rac1' succeeded

CRS-2673: Attempting to stop 'ora.gipcd' on 'rac1'

CRS-2677: Stop of 'ora.gipcd' on 'rac1' succeeded

CRS-2673: Attempting to stop 'ora.gpnpd' on 'rac1'

CRS-2677: Stop of 'ora.gpnpd' on 'rac1' succeeded

CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'rac1' has completed

CRS-4133: Oracle High Availability Services has been stopped.

[root@rac2 bin]# ./crsctl stop has

CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'rac2'

CRS-2673: Attempting to stop 'ora.crsd' on 'rac2'

CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on 'rac2'

CRS-2673: Attempting to stop 'ora.DATA.dg' on 'rac2'

CRS-2673: Attempting to stop 'ora.FLA.dg' on 'rac2'

CRS-2673: Attempting to stop 'ora.OCR.dg' on 'rac2'

CRS-2673: Attempting to stop 'ora.rac1.vip' on 'rac2'

CRS-2673: Attempting to stop 'ora.oc4j' on 'rac2'

CRS-2673: Attempting to stop 'ora.LISTENER.lsnr' on 'rac2'

CRS-2673: Attempting to stop 'ora.cvu' on 'rac2'

CRS-2673: Attempting to stop 'ora.LISTENER_SCAN1.lsnr' on 'rac2'

CRS-2677: Stop of 'ora.cvu' on 'rac2' succeeded

CRS-2677: Stop of 'ora.LISTENER.lsnr' on 'rac2' succeeded

CRS-2677: Stop of 'ora.LISTENER_SCAN1.lsnr' on 'rac2' succeeded

CRS-2673: Attempting to stop 'ora.scan1.vip' on 'rac2'

CRS-2677: Stop of 'ora.FLA.dg' on 'rac2' succeeded

CRS-2677: Stop of 'ora.DATA.dg' on 'rac2' succeeded

CRS-2677: Stop of 'ora.rac1.vip' on 'rac2' succeeded

CRS-2673: Attempting to stop 'ora.rac2.vip' on 'rac2'

CRS-2677: Stop of 'ora.scan1.vip' on 'rac2' succeeded

CRS-2677: Stop of 'ora.rac2.vip' on 'rac2' succeeded

CRS-2677: Stop of 'ora.oc4j' on 'rac2' succeeded

CRS-2677: Stop of 'ora.OCR.dg' on 'rac2' succeeded

CRS-2673: Attempting to stop 'ora.asm' on 'rac2'

CRS-2677: Stop of 'ora.asm' on 'rac2' succeeded

CRS-2673: Attempting to stop 'ora.ons' on 'rac2'

CRS-2677: Stop of 'ora.ons' on 'rac2' succeeded

CRS-2673: Attempting to stop 'ora.net1.network' on 'rac2'

CRS-2677: Stop of 'ora.net1.network' on 'rac2' succeeded

CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'rac2' has completed

CRS-2677: Stop of 'ora.crsd' on 'rac2' succeeded

CRS-2673: Attempting to stop 'ora.mdnsd' on 'rac2'

CRS-2673: Attempting to stop 'ora.ctssd' on 'rac2'

CRS-2673: Attempting to stop 'ora.evmd' on 'rac2'

CRS-2673: Attempting to stop 'ora.asm' on 'rac2'

CRS-2677: Stop of 'ora.evmd' on 'rac2' succeeded

CRS-2677: Stop of 'ora.mdnsd' on 'rac2' succeeded

CRS-2677: Stop of 'ora.asm' on 'rac2' succeeded

CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'rac2'

CRS-2677: Stop of 'ora.ctssd' on 'rac2' succeeded

CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'rac2' succeeded

CRS-2673: Attempting to stop 'ora.cssd' on 'rac2'

CRS-2677: Stop of 'ora.cssd' on 'rac2' succeeded

CRS-2673: Attempting to stop 'ora.crf' on 'rac2'

CRS-2677: Stop of 'ora.crf' on 'rac2' succeeded

CRS-2673: Attempting to stop 'ora.gipcd' on 'rac2'

CRS-2677: Stop of 'ora.gipcd' on 'rac2' succeeded

CRS-2673: Attempting to stop 'ora.gpnpd' on 'rac2'

CRS-2677: Stop of 'ora.gpnpd' on 'rac2' succeeded

CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'rac2' has completed

CRS-4133: Oracle High Availability Services has been stopped.

④关闭物理服务器

[root@rac1 bin]# shutdown -h now

[root@rac2 bin]# shutdown -h now

10 、开启数据服务器过程

①开启物理服务器

②开启数据库集群服务

[root@rac1 bin]# pwd

/u01/app/11.2.0/grid/bin

[root@rac1 bin]# ./crsctl start has

CRS-4123: Oracle High Availability Services has been started.

[root@rac2 bin]# ./crsctl start has

CRS-4123: Oracle High Availability Services has been started.

检查集群服务：

[root@rac1 bin]# ./crs_stat -t -v

Name Type R/RA F/FT Target State Host

----------------------------------------------------------------------

ora.DATA.dg ora....up.type 0/5 0/ ONLINE ONLINE rac1

ora.FLA.dg ora....up.type 0/5 0/ ONLINE ONLINE rac1

ora....ER.lsnr ora....er.type 0/5 0/ ONLINE ONLINE rac1

ora....N1.lsnr ora....er.type 0/5 0/0 ONLINE ONLINE rac2

ora.OCR.dg ora....up.type 0/5 0/ ONLINE ONLINE rac1

ora.asm ora.asm.type 0/5 0/ ONLINE ONLINE rac1

ora.cvu ora.cvu.type 0/5 0/0 ONLINE ONLINE rac2

ora.gsd ora.gsd.type 0/5 0/ OFFLINE OFFLINE

ora....network ora....rk.type 0/5 0/ ONLINE ONLINE rac1

ora.oc4j ora.oc4j.type 0/1 0/2 ONLINE ONLINE rac2

ora.ons ora.ons.type 0/3 0/ ONLINE ONLINE rac1

ora.orcl.db ora....se.type 0/2 0/1 OFFLINE OFFLINE

ora....SM1.asm application 0/5 0/0 ONLINE ONLINE rac1

ora....C1.lsnr application 0/5 0/0 ONLINE ONLINE rac1

ora.rac1.gsd application 0/5 0/0 OFFLINE OFFLINE

ora.rac1.ons application 0/3 0/0 ONLINE ONLINE rac1

ora.rac1.vip ora....t1.type 0/0 0/0 ONLINE ONLINE rac1

ora....SM2.asm application 0/5 0/0 ONLINE ONLINE rac2

ora....C2.lsnr application 0/5 0/0 ONLINE ONLINE rac2

ora.rac2.gsd application 0/5 0/0 OFFLINE OFFLINE

ora.rac2.ons application 0/3 0/0 ONLINE ONLINE rac2

ora.rac2.vip ora....t1.type 0/0 0/0 ONLINE ONLINE rac2

ora.scan1.vip ora....ip.type 0/0 0/0 ONLINE ONLINE rac2

[root@rac2 bin]# ./crs_stat -t -v

Name Type R/RA F/FT Target State Host

----------------------------------------------------------------------

ora.DATA.dg ora....up.type 0/5 0/ ONLINE ONLINE rac1

ora.FLA.dg ora....up.type 0/5 0/ ONLINE ONLINE rac1

ora....ER.lsnr ora....er.type 0/5 0/ ONLINE ONLINE rac1

ora....N1.lsnr ora....er.type 0/5 0/0 ONLINE ONLINE rac2

ora.OCR.dg ora....up.type 0/5 0/ ONLINE ONLINE rac1

ora.asm ora.asm.type 0/5 0/ ONLINE ONLINE rac1

ora.cvu ora.cvu.type 0/5 0/0 ONLINE ONLINE rac2

ora.gsd ora.gsd.type 0/5 0/ OFFLINE OFFLINE

ora....network ora....rk.type 0/5 0/ ONLINE ONLINE rac1

ora.oc4j ora.oc4j.type 0/1 0/2 ONLINE ONLINE rac2

ora.ons ora.ons.type 0/3 0/ ONLINE ONLINE rac1

ora.orcl.db ora....se.type 0/2 0/1 OFFLINE OFFLINE

ora....SM1.asm application 0/5 0/0 ONLINE ONLINE rac1

ora....C1.lsnr application 0/5 0/0 ONLINE ONLINE rac1

ora.rac1.gsd application 0/5 0/0 OFFLINE OFFLINE

ora.rac1.ons application 0/3 0/0 ONLINE ONLINE rac1

ora.rac1.vip ora....t1.type 0/0 0/0 ONLINE ONLINE rac1

ora....SM2.asm application 0/5 0/0 ONLINE ONLINE rac2

ora....C2.lsnr application 0/5 0/0 ONLINE ONLINE rac2

ora.rac2.gsd application 0/5 0/0 OFFLINE OFFLINE

ora.rac2.ons application 0/3 0/0 ONLINE ONLINE rac2

ora.rac2.vip ora....t1.type 0/0 0/0 ONLINE ONLINE rac2

ora.scan1.vip ora....ip.type 0/0 0/0 ONLINE ONLINE rac2

③开启数据服务

[oracle@rac1 ~]$ srvctl status database -d orcl

Instance orcl1 is not running on node rac1

Instance orcl2 is not running on node rac2

启动数据库：

[oracle@rac1 ~]$ srvctl start database -d orcl

[oracle@rac1 ~]$ srvctl status database -d orcl

Instance orcl1 is running on node rac1

Instance orcl2 is running on node rac2

[oracle@rac1 ~]$ sqlplus / as sysdba

SQL*Plus: Release 11.2.0.4.0 Production on Sun Apr 13 15:33:53 2025

Connected to:

Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production

With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,

Data Mining and Real Application Testing options

SQL> select name,open_mode from v$database;

NAME OPEN_MODE

--------- --------------------

ORCL READ WRITE

④测试数据库

SQL> select name,open_mode from v$database;

NAME OPEN_MODE

--------- --------------------

ORCL READ WRITE

SQL> select tablespace_name,status from dba_data_files;

TABLESPACE_NAME STATUS

------------------------------ ---------

UNDOTBS1 AVAILABLE

USERS AVAILABLE

UNDOTBS2 AVAILABLE

SYSAUX AVAILABLE

SYSTEM AVAILABLE

DATA AVAILABLE

SQL> archive log list;

Database log mode Archive Mode

Automatic archival Enabled

Archive destination +FLA/orcl/

Oldest online log sequence 194

Next log sequence to archive 196

Current log sequence 196

建表：

SQL> create table it(id number);

Table created.

SQL> insert into it values(1000);

1 row created.

SQL> commit;

Commit complete.

SQL> select *from it;

ID

----------

1000

登陆测试(tnsname.ora用于plsql连接工具)：

ITPub博客

【oracle 11g rac 部署安装--实验1 】

最新文章