总的来说，搭建DB节点虚拟机的难度比搭建Cell节点虚拟机的难度要小很多，因为绝大多数过程都与在Linux下安装Oracle Clusterware/Database 11gR2的步骤一样，所以我这里不会特别详细的介绍如何搭建一套标准的11gR2的oracle环境。对11gR2 Clusterware/Database安装不熟悉的同学建议观看ML同学的视频教程： Maclean教你一步一步使用Vbox在Linux 5上安装Oracle 11gR2 RAC 。

值得一提的是这里DB节点并不是仅局限于使用RAC。例如我这里就使用 Grid Infrastructure for a Standalone Server的方法（主要原因是我的机器太烂

）此链接提供了一个简单的安装文档可做参考。OK, Let get started.

首先需要安装一个Linux虚拟机，建议内存为1.5G以上，操作系统还是使用Oracle Linux 5.7。

安装的时候手工配置使用静态ip地址分配：我的网络配置如下：

[root@db ~]# cat /etc/hosts # Do not remove the following line, or various programs # that require network functionality will fail. 127.0.0.1 db localhost.localdomain localhost
::1 localhost6.localdomain6 localhost6
[root@db ~]# cat /etc/sysconfig/network-scripts/ifcfg-eth0  # Intel Corporation 82540EM Gigabit Ethernet Controller DEVICE=eth0 BOOTPROTO=none
BROADCAST=192.168.56.255 HWADDR=08:00:27:36:02:2B
IPADDR=192.168.56.102 NETMASK=255.255.255.0 NETWORK=192.168.56.0 ONBOOT=yes
GATEWAY=192.168.56.1 TYPE=Ethernet
USERCTL=no IPV6INIT=no PEERDNS=yes

在安装Linux的时候注意选择开发的安装包。没有选择没关系，可以通过配置yum安装oracle-validated。

安装完毕以后，mount Oracle Linux 5.7的iso文件作为yum源，我的做法如下：

[root@db media]# mount /dev/mapper/VolGroup00-LogVol00 on / type ext3 (rw)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
devpts on /dev/pts type devpts (rw,gid=5,mode=620)
/dev/sda1 on /boot type ext3 (rw)
tmpfs on /dev/shm type tmpfs (rw)
none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw) /dev/sr0 on /media/OL5.7 x86_64 dvd 20110728 type iso9660 (ro,nosuid,nodev,uid=0)
[root@db media]# umount /media/OL5.7\ x86_64\ dvd\ 20110728/ [root@db media]# mkdir /media/iso [root@db media]# mount /dev/sr0 /media/iso mount: block device /dev/sr0 is write-protected, mounting read-only
[root@db ~]# vi  /etc/yum.repos.d/ol5.repo [root@db ~]# cat /etc/yum.repos.d/ol5.repo  [OL5]
name=OL 5 enabled=1 gpgcheck=0 baseurl=file:///media/iso/Server/

然后执行yum -y install oracle-validated就安装了所有Oracle需要的rpm包并且默认创建好了所有的用户组和用户。

安装完成以后建议关闭一些不需要的服务以节省系统的资源：

chkconfig --level 2345 auditd off && service auditd stop

chkconfig --level 2345 autofs off && service autofs stop

chkconfig --level 2345 avahi-daemon off && service avahi-daemon stop

chkconfig --level 2345 bluetooth off && service bluetooth stop

chkconfig --level 2345 cups off && service cups stop

chkconfig --level 2345 ip6tables off && service ip6tables stop

chkconfig --level 2345 iptables off && service iptables stop

chkconfig --level 2345 isdn off && service isdn stop

chkconfig --level 2345 kudzu off && service kudzu stop

chkconfig --level 2345 mcstrans off && service auditd stop

chkconfig --level 2345 netfs off && service netfs stop

chkconfig --level 2345 pcscd off && service pcscd stop

chkconfig --level 2345 restorecond off && service restorecond stop

chkconfig --level 2345 rhnsd off && service rhnsd stop

chkconfig --level 2345 sendmail off && service sendmail stop

chkconfig --level 2345 setroubleshoot off && service settroubleshoot stop

chkconfig --level 2345 smartd off && service smartd stop

chkconfig --level 2345 xinet off && service xinet stop

chkconfig --level 2345 yum-updatesd off && service yum-updatesd stop

同时建议关闭SELinux：

在/etc/selinux/config中将SELINUX=enabled修改为SELINUX=disabled，然后

echo 0 >/selinux/enforce就关闭了SELinux。

在开始安装Clusterware之前需要创建两个Exadata所需的配置文件。Exadata所需要的配置文件有两个一个是cellinit.ora，此文件中填入DB节点的ip地址和子网掩码。另外一个是cellip.ora, 此文件填入Cell节点的ip地址，我本机的配置如下所示：

[root@db ~]# mkdir -p  /etc/oracle/cell/network-config

[root@db ~]# chown -R grid:oinstall  /etc/oracle/cell/network-config

[root@db ~]# chmod -R 775 /etc/oracle/cell/network-config

[root@db ~]# vi /etc/oracle/cell/network-config/cellinit.ora

[root@db ~]# cat /etc/oracle/cell/network-config/cellinit.ora

ipaddress1=192.168.56.102/24

[root@db ~]# vi /etc/oracle/cell/network-config/cellip.ora

[root@db ~]# cat /etc/oracle/cell/network-config/cellip.ora

cell="192.168.56.101"

直接rpm包的安装
rpm -ivh oracleasm-support-2.1.8-1.el5.x86_64.rpm
rpm -ivh oracleasm-2.6.18-164.el5-2.0.5-1.el5.x86_64.rpm
rpm -ivh oracleasmlib-2.0.4-1.el5.x86_64.rpm

初始化 asmlib，在所有节点执行；
[root@rac1 ~]# /usr/sbin/oracleasm configure -i
Configuring the Oracle ASM library driver.

This will configure the on-boot properties of the Oracle ASM library
driver. The following questions will determine whether the driver is
loaded on boot and what permissions it will have. The current values
will be shown in brackets ('[]'). Hitting without typing an
answer will keep that current value. Ctrl-C will abort.

Default user to own the driver interface []: grid
Default group to own the driver interface []: asmadmin
Start Oracle ASM library driver on boot (y/n) [n]: y
Scan for Oracle ASM disks on boot (y/n) [y]: y
Writing Oracle ASM library driver configuration: done
[root@rac1 ~]#

ASMlib 加载到内核，在所有节点执行；
[root@rac1 ~]# /usr/sbin/oracleasm init
Creating /dev/oracleasm mount point: /dev/oracleasm
Loading module "oracleasm": oracleasm
Mounting ASMlib driver filesystem: /dev/oracleasm

然后就可以开始安装Oracle Grid Infrastructure了。

如果前面的配置正确，在配置ASM一步就能看到Cell的Griddisk了。

这些以o/开头的ASM磁盘就是Exadata Cell端创建的griddisk。当然在安装的时候可以选择普通的裸盘（raw disk）而非griddisk作为磁盘组，到Grid Infrastructure安装完毕以后通过asmca来添加/新建新的磁盘组。

[root@node2 ~]# sh /u01/app/oraInventory/orainstRoot.sh
Changing permissions of /u01/app/oraInventory.
Adding read,write permissions for group.
Removing read,write,execute permissions for world.
Changing groupname of /u01/app/oraInventory to grid.
The execution of the script is complete.
[root@node2 ~]# sh /u01/app/11.2.0/grid/root.sh
Performing root user operation for Oracle 11g

The following environment variables are set as:
ORACLE_OWNER= grid
ORACLE_HOME= /u01/app/11.2.0/grid

Enter the full pathname of the local bin directory: [/usr/local/bin]:
Copying dbhome to /usr/local/bin ...
Copying oraenv to /usr/local/bin ...
Copying coraenv to /usr/local/bin ...

Creating /etc/oratab file...
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Using configuration parameter file: /u01/app/11.2.0/grid/crs/install/crsconfig_params
Creating trace directory
User ignored Prerequisites during installation
OLR initialization - successful
root wallet
root wallet cert
root cert export
peer wallet
profile reader wallet
pa wallet
peer wallet keys
pa wallet keys
peer cert request
pa cert request
peer cert
pa cert
peer root cert TP
profile reader root cert TP
pa root cert TP
peer pa cert TP
pa peer cert TP
profile reader pa cert TP
profile reader peer cert TP
peer user cert
pa user cert
Adding Clusterware entries to inittab
CRS-2672: Attempting to start 'ora.mdnsd' on 'node2'
CRS-2676: Start of 'ora.mdnsd' on 'node2' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'node2'
CRS-2676: Start of 'ora.gpnpd' on 'node2' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'node2'
CRS-2672: Attempting to start 'ora.gipcd' on 'node2'
CRS-2676: Start of 'ora.cssdmonitor' on 'node2' succeeded
CRS-2676: Start of 'ora.gipcd' on 'node2' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'node2'
CRS-2672: Attempting to start 'ora.diskmon' on 'node2'
CRS-2676: Start of 'ora.diskmon' on 'node2' succeeded
CRS-2676: Start of 'ora.cssd' on 'node2' succeeded

ASM created and started successfully.
Disk Group DATA created successfully.

clscfg: -install mode specified
Successfully accumulated necessary OCR keys.
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
CRS-4256: Updating the profile
Successful addition of voting disk 65ae048d9c064f27bf352074857a106b.
Successfully replaced voting disk group with +DATA.
CRS-4256: Updating the profile
CRS-4266: Voting file(s) successfully replaced
## STATE File Universal Id File Name Disk group
-- ----- ----------------- --------- ---------
1. ONLINE 65ae048d9c064f27bf352074857a106b (o/192.0.2.40/data_CD_disk01_cell) [DATA]
Located 1 voting disk(s).

Located 1 voting disk(s).
CRS-2672: Attempting to start 'ora.asm' on 'node2'
CRS-2676: Start of 'ora.asm' on 'node2' succeeded
CRS-2672: Attempting to start 'ora.DATA.dg' on 'node2'
CRS-2676: Start of 'ora.DATA.dg' on 'node2' succeeded
Configure Oracle Grid Infrastructure for a Cluster ... succeeded
[root@node1 ~]# sh /u01/app/oraInventory/orainstRoot.sh
Changing permissions of /u01/app/oraInventory.
Adding read,write permissions for group.
Removing read,write,execute permissions for world.

Changing groupname of /u01/app/oraInventory to grid.
The execution of the script is complete.
[root@node1 ~]#
[root@node1 ~]# sh /u01/app/oraInventory/orainstRoot.sh
Changing permissions of /u01/app/oraInventory.
Adding read,write permissions for group.
Removing read,write,execute permissions for world.

Changing groupname of /u01/app/oraInventory to grid.
The execution of the script is complete.
[root@node1 ~]# sh /u01/app/11.2.0/grid/root.sh
Performing root user operation for Oracle 11g

The following environment variables are set as:
ORACLE_OWNER= grid
ORACLE_HOME= /u01/app/11.2.0/grid

Enter the full pathname of the local bin directory: [/usr/local/bin]:
Copying dbhome to /usr/local/bin ...
Copying oraenv to /usr/local/bin ...
Copying coraenv to /usr/local/bin ...

Creating /etc/oratab file...
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Using configuration parameter file: /u01/app/11.2.0/grid/crs/install/crsconfig_params
Creating trace directory
User ignored Prerequisites during installation
OLR initialization - successful
Adding Clusterware entries to inittab
CRS-4402: The CSS daemon was started in exclusive mode but found an active CSS daemon on node node2, number 1, and is terminating
An active cluster was found during exclusive startup, restarting to join the cluster
Configure Oracle Grid Infrastructure for a Cluster ... succeeded

[grid@node1 bin]$ ./crs_stat -t
Name Type Target State Host
------------------------------------------------------------
ora.DATA.dg ora....up.type ONLINE ONLINE node1
ora....N1.lsnr ora....er.type ONLINE ONLINE node1
ora.asm ora.asm.type ONLINE ONLINE node1
ora.cvu ora.cvu.type ONLINE ONLINE node1
ora.gsd ora.gsd.type OFFLINE OFFLINE
ora....network ora....rk.type ONLINE ONLINE node1
ora....SM2.asm application ONLINE ONLINE node1
ora.node1.gsd application OFFLINE OFFLINE
ora.node1.ons application ONLINE ONLINE node1
ora.node1.vip ora....t1.type ONLINE ONLINE node1
ora....SM1.asm application ONLINE ONLINE node2
ora.node2.gsd application OFFLINE OFFLINE
ora.node2.ons application ONLINE OFFLINE
ora.node2.vip ora....t1.type ONLINE ONLINE node1
ora.oc4j ora.oc4j.type ONLINE ONLINE node2
ora.ons ora.ons.type ONLINE ONLINE node1
ora.scan1.vip ora....ip.type ONLINE ONLINE node1

创建完成以后如图所示：

如果没有找到griddisk，也可以尝试手工修改ASM_DISKSTRING到”o/cell的ip地址/*”。例如：我的ASM_DISKSTRING为： “o/192.168.56.101/data*”

创建完成以后可以通过SQLPLUS和asmcmd来查看其属性：

[grid@db bin]$ export ORACLE_HOME=/u01/app/grid/product/11.2.0/grid
[grid@db bin]$ export ORACLE_SID=+ASM
[grid@db bin]$ ./sqlplus
SQL> select disk_number, name, failgroup, path, header_status from v$asm_disk where path like 'o/%' order by disk_number; DISK_NUMBER NAME                           FAILGROUP                      PATH                                     HEADER_STATUS ----------- ------------------------------ ------------------------------ ---------------------------------------- ---------- 0 DATA_CD_DISK01_CELL1           DATA_CD_DISK01_CELL1           o/192.168.56.101/data_CD_disk01_cell1    MEMBER
          1 DATA_CD_DISK02_CELL1           DATA_CD_DISK02_CELL1           o/192.168.56.101/data_CD_disk02_cell1    MEMBER
          2 DATA_CD_DISK03_CELL1           DATA_CD_DISK03_CELL1           o/192.168.56.101/data_CD_disk03_cell1    MEMBER
          3 DATA_CD_DISK04_CELL1           DATA_CD_DISK04_CELL1           o/192.168.56.101/data_CD_disk04_cell1    MEMBER
          4 DATA_CD_DISK05_CELL1           DATA_CD_DISK05_CELL1           o/192.168.56.101/data_CD_disk05_cell1    MEMBER
          5 DATA_CD_DISK06_CELL1           DATA_CD_DISK06_CELL1           o/192.168.56.101/data_CD_disk06_cell1    MEMBER
          6 DATA_CD_DISK07_CELL1           DATA_CD_DISK07_CELL1           o/192.168.56.101/data_CD_disk07_cell1    MEMBER
          7 DATA_CD_DISK08_CELL1           DATA_CD_DISK08_CELL1           o/192.168.56.101/data_CD_disk08_cell1    MEMBER
          8 DATA_CD_DISK09_CELL1           DATA_CD_DISK09_CELL1           o/192.168.56.101/data_CD_disk09_cell1    MEMBER
          9 DATA_CD_DISK10_CELL1           DATA_CD_DISK10_CELL1           o/192.168.56.101/data_CD_disk10_cell1    MEMBER
         10 DATA_CD_DISK11_CELL1           DATA_CD_DISK11_CELL1           o/192.168.56.101/data_CD_disk11_cell1    MEMBER
         11 DATA_CD_DISK12_CELL1           DATA_CD_DISK12_CELL1           o/192.168.56.101/data_CD_disk12_cell1    MEMBER
ASMCMD> lsdg
State    Type    Rebal  Sector  Block AU  Total_MB  Free_MB  Req_mir_free_MB  Usable_file_MB  Offline_disks  Voting_files  Name
MOUNTED  EXTERN  N 512 4096 4194304 11328 11152 0 11152 0 N  DATA/
ASMCMD> lsattr -l -G data
Name Value access_control.enabled FALSE access_control.umask 066 au_size 4194304 cell.smart_scan_capable TRUE compatible.asm 11.2.0.0.0 compatible.rdbms 11.1.0.7.0 disk_repair_time 3.6h        
sector_size 512 ASMCMD> lsdsk -G data
Path
o/192.168.56.101/data_CD_disk01_cell1
o/192.168.56.101/data_CD_disk02_cell1
o/192.168.56.101/data_CD_disk03_cell1
o/192.168.56.101/data_CD_disk04_cell1
o/192.168.56.101/data_CD_disk05_cell1
o/192.168.56.101/data_CD_disk06_cell1
o/192.168.56.101/data_CD_disk07_cell1
o/192.168.56.101/data_CD_disk08_cell1
o/192.168.56.101/data_CD_disk09_cell1
o/192.168.56.101/data_CD_disk10_cell1
o/192.168.56.101/data_CD_disk11_cell1
o/192.168.56.101/data_CD_disk12_cell1

最后，继续按照标准的流程来安装Oracle Database，建立数据库实例，监听等。

这样一套简单Exadata的虚拟机就搭建好了。

ITPub博客

搭建Exadata虚拟机——DB节点

最新文章