ITPub博客

首页 > Linux操作系统 > Linux操作系统 > rhcs 4.2+gfs6.1.2

rhcs 4.2+gfs6.1.2

原创 Linux操作系统 作者:眼镜男 时间:2009-07-19 14:51:53 0 删除 编辑

一:安装rhcs所需的rpm包安装可有脚本完成,脚本如下:
#!/bin/sh
rpm -ivh cman-kernel-2.6.9-39.5.i686.rpm
rpm -ivh cman-kernel-hugemem-2.6.9-39.5.i686.rpm
rpm -ivh cman-kernheaders-2.6.9-39.5.i686.rpm
rpm -ivh cman-kernel-debuginfo-2.6.9-39.5.i686.rpm
rpm -ivh cman-kernel-smp-2.6.9-39.5.i686.rpm
rpm -ivh magma-1.0.1-4.i686.rpm
rpm -ivh magma-devel-1.0.1-4.i686.rpm
rpm -ivh magma-debuginfo-1.0.1-4.i686.rpm
rpm -ivh dlm-kernel-2.6.9-37.7.i686.rpm
rpm -ivh dlm-kernel-debuginfo-2.6.9-37.7.i686.rpm
rpm -ivh dlm-kernel-smp-2.6.9-37.7.i686.rpm
--nodeps
rpm -ivh dlm-kernel-hugemem-2.6.9-37.7.i686.rpm
--nodeps
rpm -ivh dlm-kernheaders-2.6.9-37.7.i686.rpm
rpm -ivh dlm-1.0.0-5.i686.rpm--nodeps
rpm -ivh dlm-debuginfo-1.0.0-5.i686.rpm
rpm -ivh dlm-devel-1.0.0-5.i686.rpm
rpm -ivh ccs-*
rpm -ivh gulm-*
rpm -ivh magma-plugins-*
rpm -ivh cman-1.0.4-0.i686.rpm
rpm -ivh cman-debuginfo-1.0.4-0.i686.rpm
rpm -ivh cman-devel-1.0.4-0.i686.rpm
rpm -ivh perl-Net-Telnet-3.03-1.2.el4.rf.noarch.rpm
rpm -ivh fence-*
rpm -ivh iddev-*
rpm -ivh piranha-* --nodeps
rpm -ivh rgmanager-*
rpm -ivh system-config-cluster-1.0.16-1.0.noarch.rpm
rpm -ivh GFS-kernel-2.6.9-42.1.i686.rpm
rpm -ivh GFS-kernheaders-2.6.9-42.1.i686.rpm
rpm -ivh GFS-kernel-debuginfo-2.6.9-42.1.i686.rpm
rpm -ivh GFS-kernel-smp-2.6.9-42.1.i686.rpm--nodeps
rpm -ivh GFS-kernel-hugemem-2.6.9-42.1.i686.rpm--nodeps
rpm -ivh GFS-6.1.2-0.i386.rpm
rpm -ivh GFS-debuginfo-6.1.2-0.i386.rpm
rpm -ivh gnbd-kernheaders-2.6.9-9.12.i686.rpm
rpm -ivh gnbd-kernel-2.6.9-9.12.i686.rpm
rpm -ivh gnbd-kernel-debuginfo-2.6.9-9.12.i686.rpm
rpm -ivh gnbd-kernel-smp-2.6.9-9.12.i686.rpm--nodeps
rpm -ivh gnbd-kernel-hugemem-2.6.9-9.12.i686.rpm--nodeps
rpm -ivh gnbd-1.0.1-1.i686.rpm
rpm -ivh gnbd-debuginfo-1.0.1-1.i686.rpm
rpm -ivh lvm2-cluster-*
rpm -ivh ipvsadm-*

二、修改各个节点上的/etc/hosts文件(每个节点都一样)
如下:
        [root@gfs-node01 etc]# cat hosts
        # Do not remove the following line, or various programs
        # that require network functionality will fail.
        #127.0.0.1              gfs-node01 localhost.localdomain localhost
        192.168.60.133            gfs-node01
        192.168.60.135            gfs-node02
  
192.168.60.133            gnbd-server
        [root@gfs-node01 etc]#

备注:我把gnbs-server和node01放在了同一台server上,实际运行如果有条件的话中最好分开

三、在gnbd-server(此例也就是node1)上进行分区#dmesg |grep scsi察看scsi设备,如下:
        [root@gfs-node01 ~]# dmesg|grep scsi
scsi0 : ioc0: LSI53C1030, FwRev=00000000h, Ports=1, MaxQ=128, IRQ=9
Attached scsi disk sda at scsi0, channel 0, id 0, lun 0
scsi1 : ioc1: LSI53C1030, FwRev=00000000h, Ports=1, MaxQ=128, IRQ=11
Attached scsi disk sdb at scsi1, channel 0, id 0, lun 0

说明认出了两个scsi设备,一个是系统盘,一个是共享存储盘,接着用fdisk进行分区,注意,只进行分区,不要格式化,在这里我使用fdisk分了一个大小为2G的分区, /dev/sdb1即是。
 
[root@gfs-node01 ~]#fdisk -l
Disk /dev/sda: 4294 MB, 4294967296 bytes
255 heads, 63 sectors/track, 522 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot
Start
End
Blocks
Id
System
/dev/sda1
*
1

13
104391
83
Linux
/dev/sda2
14
140
1020127+
82
Linux swap
/dev/sda3
141
522
3068415
83
Linux

Disk /dev/sdb: 2147 MB, 2147483648 bytes
255 heads, 63 sectors/track, 261 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot
Start
End
Blocks
Id
System
/dev/sdb1
1
261
2096451
83
Linux    
四、运行system-config-cluster进行配置

(注视“因为system-config-cluster需要X-windows运行,所以第一次测试时安装了X-windows,实际配置时候只需要创建一个/etc/cluster目录,然后根据下面的cluster.conf文件进行修改即可)

增加两个节点,节点的权置全部设置为1,即Quorum值设置为1
两个节点的名称为:
gfs-node01       
gfs-node02

修改cluster.conf文件,如下:

[root@gfs-node01 ~]# cat /etc/cluster/cluster.conf


       
       
               
                       
                               
                                       
                               

                       

               

               
                       
                               
                                       
                               

                       

               

       

      
     
               
       

       
               
                       
                               
                               
                       

               

       


[root@gfs-node01 ~]#

使用scp命令把这个配置文件copy到02节点上
五、在01/02节点上启动dlm,ccsd,fence等服务 

        5.1、在两个节点上加载dlm模块 

        [root@gfs-node01 cluster]# modprobe lock_dlm
        [root@gfs-node02 cluster]# modprobe lock_dlm

        5.2、启动ccsd服务 
        [root@gfs-node01 cluster]# ccsd
        [root@gfs-node02 cluster]# ccsd

        5.3、在两个节点上启动集群管理器(cman)
root@one # /sbin/cman_tool join  
root@tow # /sbin/cman_tool join 

        5.4、测试ccsd服务 
        (注意:ccsd的测试要等cman启动完成后,然后才可以进行下面的测试
        [root@gfs-node01 cluster]# ccs_test connect
        [root@gfs-node02 cluster]# ccs_test connect
      
# ccs_test connect 各个节点的返回如下:
        node 1:
        [root@gfs-node01 cluster]# ccs_test connect
        Connect successful.
        Connection descriptor = 0
        node 2:
        [root@gfs-node02 cluster]# ccs_test connect
        Connect successful.
        Connection descriptor = 30
        5.5、查看节点状态
        cat /proc/cluster/nodes,应该返回 
IXDBA.NET技术社区
        [root@gfs-node01 cluster]# cat /proc/cluster/nodes
        Node  Votes Exp Sts  Name
          1    1    3   M   gfs-node01
          2    1    3   M   gfs-node02
       
[root@gfs-node01 cluster]#
六、加入fence域: 
[root@gfs-node01 cluster]# /sbin/fence_tool join
[root@gfs-node02 cluster]# /sbin/fence_tool join 或者fence_tool join –c

七、 查看集群状态
Node 1:
[root@gfs-node01 cluster]# cat /proc/cluster/status
Protocol version: 5.0.1
Config version: 1
Cluster name: alpha_cluster
Cluster ID: 50356
Cluster Member: Yes
Membership state: Cluster-Member
Nodes: 3
Expected_votes: 3
Total_votes: 3
Quorum: 2  
Active subsystems: 1
Node name: gfs-node01
Node ID: 1
Node addresses: 192.168.60.133

Node 2
[root@gfs-node02 cluster]# cat /proc/cluster/status
Protocol version: 5.0.1
Config version: 1
Cluster name: alpha_cluster
Cluster ID: 50356
Cluster Member: Yes
Membership state: Cluster-Member
Nodes: 3
Expected_votes: 3
Total_votes: 3
Quorum: 2  
Active subsystems: 1
Node name: gfs-node02
Node ID: 2
Node addresses: 192.168.60.135

八、 在gnbd-server服务节点导出设备

        8.1、启动gnbd_serv进程 
        [root@gfs-node01 cluster]# /sbin/gnbd_serv –v –n
        gnbd_serv: startup succeeded
        导出设备 
        [root@gfs-node01 cluster]# gnbd_export -v -e gfs -d /dev/sdb1 -c
        gnbd_export: created GNBD gfs serving file /dev/sda1
        [root@gfs-node01 cluster]#
        查看export状态信息 
        [root@gfs-node01 cluster]# gnbd_export -v -l
        Server[1] : gfs
        --------------------------
              file : /dev/sda1
           sectors : 5879727
          readonly : no
           cached : yes
          timeout : no

        [root@gfs-node01 cluster]#
以上操作是在gnbd-server上完成,其它操作全部是在节点上完成。
8.2、 在01/02两个节点导入设备 
          
[root@gfs-node01 cluster] # modprobe gnbd 
        [root@gfs-node02 cluster] # modprobe gnbd 

        导入设备
        Node1:
        [root@gfs-node01 cluster]# gnbd_import -v -i gnbd-server
        gnbd_import: created gnbd device gfs
        gnbd_recvd: gnbd_recvd started
        [root@gfs-node01 cluster]#

        注视:此处的gnbd-server即是你写在/etc/hosts里面的冬冬
        Node2:
        [root@gfs-node02 cluster]# gnbd_import -v -i gnbd-server
        gnbd_import: created directory /dev/gnbd
        gnbd_import: created gnbd device gfs
        gnbd_recvd: gnbd_recvd started
        [root@gfs-node02 cluster]#
           
        8.3、查看导入状态信息
(两个节点都要做)
        [root@gfs-node01 cluster]# gnbd_import -v -l
        Device name : gfs
        ----------------------
           Minor # : 0
         sysfs name : /dev/gnbd0
             Server : gnbd-server
               Port : 14567
              State : Close Connected Clear
           Readonly : No
            Sectors : 5879727

        [root@gfs-node01 cluster]#

九、 建立gfs文件系统并且挂载 
        9.1、在两个节点加载gfs模块 
        [root@gfs-node01 cluster] # modprobe gfs
        [root@gfs-node02 cluster] # modprobe gfs

        9.2、在任意一个节点(我这里也是node01节点)上建立gfs文件系统 
        [root@gfs-node01 cluster]#
gfs_mkfs -p lock_dlm -t alpha_cluster:gfs -j 3 /dev/gnbd/gfs
        This will destroy any data on /dev/gnbd/gfs.
          It appears to contain a GFS filesystem.

        Are you sure you want to proceed? [y/n] y

        Device:                    /dev/gnbd/gfs
        Blocksize:                 4096
        Filesystem Size:           669344
        Journals:                  2
        Resource Groups:           12
        Locking Protocol:          lock_dlm
        Lock Table:                alpha_cluster:gfs

        Syncing...
        All Done
        [root@gfs-node01 cluster]#  
       
        9.3、在两个节点挂载文件系统
在两个节点的根目录下都创建一个名为gfs的目录,然后
        [root@gfs-node01 cluster] # mount -t gfs /dev/gnbd/gfs /gfs
        [root@gfs-node02 cluster] # mount -t gfs /dev/gnbd/gfs /gfs
          
        注释:要在node01挂载文件完全完成后,再在02节点上进行mount,02节点mount完成后,再在其它节点进行mount

十:遇到问题:

10.1、例如,我在进行node02的mount时出现了如下错误
        [root@gfs-node03 cluster]# mount -t gfs /dev/gnbd/gfs /gfstest
       mount: wrong fs type, bad option, bad superblock on /dev/gnbd/gfs,
               or too many mounted file systems
经查,发现我是按照
gfs_mkfs -p lock_dlm -t alpha_cluster:gfs -j 1 /dev/gnbd/gfs
做的,是一个节点的,于是umount节点1,从第九部开始重做,然后再mount就对了。
gfs_mkfs -p lock_dlm -t alpha_cluster:gfs -j 3 /dev/gnbd/gfs
      其中的3表示三个节点,实际上的话你是几个就写几个就可以了。
10.2、mount: permission denied错误
如果你配置了多个fence域,而该域又不在本结点且还没有配置好该结点,需要这样启动fence设备:
fence_tool leave
fence_tool join –c
否则会出现这样的错误:mount: permission denied 为这个问题我废了一天!建议凡是客户端配置除mount步骤外其它步骤同时在结点上进行,而mount时得一个结点mount 完了才能接着mount。

 

来自 “ ITPUB博客 ” ,链接:http://blog.itpub.net/21996817/viewspace-609621/,如需转载,请注明出处,否则将追究法律责任。

上一篇: 双网卡的绑定
请登录后发表评论 登录
全部评论

注册时间:2009-06-28

  • 博文量
    30
  • 访问量
    31681