ITPub博客
首页 博客 专栏 学院 社区 IT新媒体评选
发博

首页 > 大数据 > Hadoop > 【实验】Hadoop-2.7.2+zookeeper-3.4.6完全分布式环境搭建(HDFS、YARN HA)

【实验】Hadoop-2.7.2+zookeeper-3.4.6完全分布式环境搭建(HDFS、YARN HA)

原创 Hadoop 作者：hackeruncle 时间：2016-02-25 16:43:51 0 删除 编辑

Hadoop-2.7.2+Zookeeper-3.4.6完全分布式环境搭建

一.版本

组件名

版本

说明

JRE

java version "1.7.0_67"

Java(TM) SE Runtime Environment (build 1.7.0_67-b01)

Java HotSpot(TM) 64-Bit Server VM (build 24.65-b04, mixed mode)

 

Hadoop

hadoop-2.7.2.tar.gz

主程序包

Zookeeper

zookeeper-3.4.6.tar.gz

热切,Yarn 存储数据使用的协调服务

二.主机规划

IP

Host 及安装软件

部署模块

进程

172.16.101.55

sht-sgmhadoopnn-01

hadoop

NameNode

ResourceManager

NameNode

DFSZKFailoverController

ResourceManager

172.16.101.56

sht-sgmhadoopnn-02

hadoop

NameNode

ResourceManager

NameNode

DFSZKFailoverController

ResourceManager

172.16.101.58

sht-sgmhadoopdn-01

hadoop、zookeeper

DataNode

NodeManager

Zookeeper

DataNode

NodeManager

JournalNode

QuorumPeerMain

172.16.101.59

sht-sgmhadoopdn-02

Hadoop、zookeeper

DataNode

NodeManager

Zookeeper

DataNode

NodeManager

JournalNode

QuorumPeerMain

172.16.101.60

sht-sgmhadoopdn-03

Hadoop、zookeeper

DataNode

NodeManager

Zookeeper

DataNode

NodeManager

JournalNode

QuorumPeerMain

三.目录规划

名称

路径

$HADOOP_HOME

/hadoop/hadoop-2.7.2

Data

$ HADOOP_HOME/data

Log

$ HADOOP_HOME/logs

四.常用脚本及命令

1.启动集群

start-dfs.sh

start-yarn.sh

2.关闭集群

stop-yarn.sh

stop-dfs.sh

3.监控集群

hdfs dfsadmin -report

4.单个进程启动/关闭

hadoop-daemon.sh start|stop namenode|datanode| journalnode

yarn-daemon.sh start |stop resourcemanager|nodemanager

http://blog.chinaunix.net/uid-25723371-id-4943894.html

五.环境准备

1 .设置ip地址(5台)

点击(此处)折叠或打开

  1. [root@sht-sgmhadoopnn-01 ~]# vi /etc/sysconfig/network-scripts/ifcfg-eth0
  2. DEVICE="eth0"
  3. BOOTPROTO="static"
  4. DNS1="172.16.101.63"
  5. DNS2="172.16.101.64"
  6. GATEWAY="172.16.101.1"
  7. HWADDR="00:50:56:82:50:1E"
  8. IPADDR="172.16.101.55"
  9. NETMASK="255.255.255.0"
  10. NM_CONTROLLED="yes"
  11. ONBOOT="yes"
  12. TYPE="Ethernet"
  13. UUID="257c075f-6c6a-47ef-a025-e625367cbd9c"


执行命令: service network restart

验证:ifconfig

2 .关闭防火墙(5台)

执行命:service iptables stop

验证:service iptables status

3.关闭防火墙的自动运行(5台)

执行命令:chkconfig iptables off

验证:chkconfig --list | grep iptables

4 设置主机名(5台)

执行命令
(1)hostname sht-sgmhadoopnn-01

(2)vi /etc/sysconfig/network

点击(此处)折叠或打开

  1. [root@sht-sgmhadoopnn-01 ~]# vi /etc/sysconfig/network
  2. NETWORKING=yes
  3. HOSTNAME=sht-sgmhadoopnn-01.telenav.cn
  4. GATEWAY=172.16.101.1

5 ip与hostname绑定(5台)

点击(此处)折叠或打开

  1. [root@sht-sgmhadoopnn-01 ~]# vi /etc/hosts
  2. 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
  3. ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
  4. 172.16.101.55 sht-sgmhadoopnn-01.telenav.cn sht-sgmhadoopnn-01
  5. 172.16.101.56 sht-sgmhadoopnn-02.telenav.cn sht-sgmhadoopnn-02
  6. 172.16.101.58 sht-sgmhadoopdn-01.telenav.cn sht-sgmhadoopdn-01
  7. 172.16.101.59 sht-sgmhadoopdn-02.telenav.cn sht-sgmhadoopdn-02
  8. 172.16.101.60 sht-sgmhadoopdn-03.telenav.cn sht-sgmhadoopdn-03
  9. 验证:ping sht-sgmhadoopnn-01

6. 设置5台machines,SSH互相通信
http://blog.itpub.net/30089851/viewspace-1992210/

7 .安装JDK(5台)

点击(此处)折叠或打开

  1. (1)执行命令
  2. [root@sht-sgmhadoopnn-01 ~]# cd /usr/java
  3. [root@sht-sgmhadoopnn-01 java]# cp /tmp/jdk-7u67-linux-x64.gz ./
  4. [root@sht-sgmhadoopnn-01 java]# tar -xzvf jdk-7u67-linux-x64.gz
  5. (2)vi /etc/profile 增加内容如下:
  6. export JAVA_HOME=/usr/java/jdk1.7.0_67
  7. export HADOOP_HOME=/hadoop/hadoop-2.7.2
  8. export ZOOKEEPER_HOME=/hadoop/zookeeper
  9. export PATH=.:$HADOOP_HOME/bin:$JAVA_HOME/bin:$ZOOKEEPER_HOME/bin:$PATH
  10. #先把HADOOP_HOME, ZOOKEEPER_HOME配置了
  11. #本次实验机器已经配置好了jdk1.7.0_67-cloudera
  12. (3)执行 source /etc/profile
  13. (4)验证:java –version

8.创建文件夹(5台)

mkdir /hadoop

六.安装Zookeeper

sht-sgmhadoopdn-01/02/03

1.下载解压zookeeper-3.4.6.tar.gz

点击(此处)折叠或打开

  1. [root@sht-sgmhadoopdn-01 tmp]# wget http://mirror.bit.edu.cn/apache/zookeeper/zookeeper-3.4.6/zookeeper-3.4.6.tar.gz
  2. [root@sht-sgmhadoopdn-02 tmp]# wget http://mirror.bit.edu.cn/apache/zookeeper/zookeeper-3.4.6/zookeeper-3.4.6.tar.gz
  3. [root@sht-sgmhadoopdn-03 tmp]# wget http://mirror.bit.edu.cn/apache/zookeeper/zookeeper-3.4.6/zookeeper-3.4.6.tar.gz
  4. [root@sht-sgmhadoopdn-01 tmp]# tar -xvf zookeeper-3.4.6.tar.gz
  5. [root@sht-sgmhadoopdn-02 tmp]# tar -xvf zookeeper-3.4.6.tar.gz
  6. [root@sht-sgmhadoopdn-03 tmp]# tar -xvf zookeeper-3.4.6.tar.gz
  7. [root@sht-sgmhadoopdn-01 tmp]# mv zookeeper-3.4.6 /hadoop/zookeeper
  8. [root@sht-sgmhadoopdn-02 tmp]# mv zookeeper-3.4.6 /hadoop/zookeeper
  9. [root@sht-sgmhadoopdn-03 tmp]# mv zookeeper-3.4.6 /hadoop/zookeeper

2.修改配置

点击(此处)折叠或打开

  1. [root@sht-sgmhadoopdn-01 tmp]# cd /hadoop/zookeeper/conf
  2. [root@sht-sgmhadoopdn-01 conf]# cp zoo_sample.cfg zoo.cfg
  3. [root@sht-sgmhadoopdn-01 conf]# vi zoo.cfg
  4. 修改dataDir
  5. dataDir=/hadoop/zookeeper/data
  6. 添加下面三行
  7. server.1=sht-sgmhadoopdn-01:2888:3888
  8. server.2=sht-sgmhadoopdn-02:2888:3888
  9. server.3=sht-sgmhadoopdn-03:2888:3888
  10. [root@sht-sgmhadoopdn-01 conf]# cd ../
  11. [root@sht-sgmhadoopdn-01 zookeeper]# mkdir data
  12. [root@sht-sgmhadoopdn-01 zookeeper]# touch data/myid
  13. [root@sht-sgmhadoopdn-01 zookeeper]# echo 1 > data/myid
  14. [root@sht-sgmhadoopdn-01 zookeeper]# more data/myid
  15. 1
  16. ## sht-sgmhadoopdn-02/03,也修改配置,就如下不同
  17. [root@sht-sgmhadoopdn-02 zookeeper]# echo 2 > data/myid
  18. [root@sht-sgmhadoopdn-03 zookeeper]# echo 3 > data/myid

七.安装Hadoop(HDFS HA+YARN HA)

#step3~7,用SecureCRT ssh 到 linux的环境中,假如copy 内容从window 到 linux 中,中文乱码,请参照修改http://www.cnblogs.com/qi09/archive/2013/02/05/2892922.html

1.下载解压hadoop-2.7.2.tar.gz

点击(此处)折叠或打开

  1. [root@sht-sgmhadoopdn-01 tmp]# cd /hadoop/zookeeper/conf
  2. [root@sht-sgmhadoopdn-01 conf]# cp zoo_sample.cfg zoo.cfg
  3. [root@sht-sgmhadoopdn-01 conf]# vi zoo.cfg
  4. 修改dataDir
  5. dataDir=/hadoop/zookeeper/data
  6. 添加下面三行
  7. server.1=sht-sgmhadoopdn-01:2888:3888
  8. server.2=sht-sgmhadoopdn-02:2888:3888
  9. server.3=sht-sgmhadoopdn-03:2888:3888
  10. [root@sht-sgmhadoopdn-01 conf]# cd ../
  11. [root@sht-sgmhadoopdn-01 zookeeper]# mkdir data
  12. [root@sht-sgmhadoopdn-01 zookeeper]# touch data/myid
  13. [root@sht-sgmhadoopdn-01 zookeeper]# echo 1 > data/myid
  14. [root@sht-sgmhadoopdn-01 zookeeper]# more data/myid
  15. 1
  16. ## sht-sgmhadoopdn-02/03,也修改配置,就如下不同
  17. [root@sht-sgmhadoopdn-02 zookeeper]# echo 2 > data/myid
  18. [root@sht-sgmhadoopdn-03 zookeeper]# echo 3 > data/myid

2.修改$HADOOP_HOME/etc/hadoop/hadoop-env.sh

export JAVA_HOME="/usr/java/jdk1.7.0_67-cloudera"

3.修改$HADOOP_HOME/etc/hadoop/core-site.xml

点击(此处)折叠或打开

  1. <?xml version="1.0" encoding="UTF-8"?>
  2. <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
  3. <configuration>
  4.     <!--Yarn 需要使用 fs.defaultFS 指定NameNode URI -->
  5.     <property>
  6.         <name>fs.defaultFS</name>
  7.         <value>hdfs://mycluster</value>
  8.     </property>
  9.     <!--HDFS超级用户 -->
  10.     <property>
  11.         <name>dfs.permissions.superusergroup</name>
  12.         <value>root</value>
  13.     </property>
  14.     <!--==============================Trash机制======================================= -->
  15.     <property>
  16.         <!--多长时间创建CheckPoint NameNode截点上运行的CheckPointer 从Current文件夹创建CheckPoint;默认：0 由fs.trash.interval项指定 -->
  17.         <name>fs.trash.checkpoint.interval</name>
  18.         <value>0</value>
  19.     </property>
  20.     <property>
  21.         <!--多少分钟.Trash下的CheckPoint目录会被删除,该配置服务器设置优先级大于客户端，默认：0 不删除 -->
  22.         <name>fs.trash.interval</name>
  23.         <value>1440</value>
  24.     </property>
  25. </configuration>


4.修改$HADOOP_HOME/etc/hadoop/hdfs-site.xml


点击(此处)折叠或打开

  1. <?xml version="1.0" encoding="UTF-8"?>
  2. <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
  3. <configuration>
  4.     <!--开启web hdfs -->
  5.     <property>
  6.         <name>dfs.webhdfs.enabled</name>
  7.         <value>true</value>
  8.     </property>
  9.     <property>
  10.         <name>dfs.namenode.name.dir</name>
  11.         <value>/hadoop/hadoop-2.7.2/data/dfs/name</value>
  12.         <description> namenode 存放name table(fsimage)本地目录（需要修改）</description>
  13.     </property>
  14.     <property>
  15.         <name>dfs.namenode.edits.dir</name>
  16.         <value>${dfs.namenode.name.dir}</value>
  17.         <description>namenode粗放 transaction file(edits)本地目录（需要修改）</description>
  18.     </property>
  19.     <property>
  20.         <name>dfs.datanode.data.dir</name>
  21.         <value>/hadoop/hadoop-2.7.2/data/dfs/data</value>
  22.         <description>datanode存放block本地目录（需要修改）</description>
  23.     </property>
  24.     <property>
  25.         <name>dfs.replication</name>
  26.         <value>3</value>
  27.     </property>
  28.     <!-- 块大小 （默认） -->
  29.     <property>
  30.         <name>dfs.blocksize</name>
  31.         <value>268435456</value>
  32.     </property>
  33.     <!--======================================================================= -->
  34.     <!--HDFS高可用配置 -->
  35.     <!--nameservices逻辑名 -->
  36.     <property>
  37.         <name>dfs.nameservices</name>
  38.         <value>mycluster</value>
  39.     </property>
  40.     <property>
  41.         <!--设置NameNode IDs 此版本最大只支持两个NameNode -->
  42.         <name>dfs.ha.namenodes.mycluster</name>
  43.         <value>nn1,nn2</value>
  44.     </property>

  46.     <!-- Hdfs HA: dfs.namenode.rpc-address.[nameservice ID] rpc 通信地址 -->
  47.     <property>
  48.         <name>dfs.namenode.rpc-address.mycluster.nn1</name>
  49.         <value>sht-sgmhadoopnn-01:8020</value>
  50.     </property>
  51.     <property>
  52.         <name>dfs.namenode.rpc-address.mycluster.nn2</name>
  53.         <value>sht-sgmhadoopnn-02:8020</value>
  54.     </property>

  56.     <!-- Hdfs HA: dfs.namenode.http-address.[nameservice ID] http 通信地址 -->
  57.     <property>
  58.         <name>dfs.namenode.http-address.mycluster.nn1</name>
  59.         <value>sht-sgmhadoopnn-01:50070</value>
  60.     </property>
  61.     <property>
  62.         <name>dfs.namenode.http-address.mycluster.nn2</name>
  63.         <value>sht-sgmhadoopnn-02:50070</value>
  64.     </property>

  66.     <!--==================Namenode editlog同步 ============================================ -->
  67.     <!--保证数据恢复 -->
  68.     <property>
  69.         <name>dfs.journalnode.http-address</name>
  70.         <value>0.0.0.0:8480</value>
  71.     </property>
  72.     <property>
  73.         <name>dfs.journalnode.rpc-address</name>
  74.         <value>0.0.0.0:8485</value>
  75.     </property>
  76.     <property>
  77.         <!--设置JournalNode服务器地址，QuorumJournalManager 用于存储editlog -->
  78.         <!--格式：qjournal://<host1:port1>;<host2:port2>;<host3:port3>/<journalId> 端口同journalnode.rpc-address -->
  79.         <name>dfs.namenode.shared.edits.dir</name>
  80.         <value>qjournal://sht-sgmhadoopdn-01:8485;sht-sgmhadoopdn-02:8485;sht-sgmhadoopdn-03:8485/mycluster</value>
  81.     </property>

  83.     <property>
  84.         <!--JournalNode存放数据地址 -->
  85.         <name>dfs.journalnode.edits.dir</name>
  86.         <value>/hadoop/hadoop-2.7.2/data/dfs/jn</value>
  87.     </property>
  88.     <!--==================DataNode editlog同步 ============================================ -->
  89.     <property>
  90.         <!--DataNode,Client连接Namenode识别选择Active NameNode策略 -->
  91.         <name>dfs.client.failover.proxy.provider.mycluster</name>
  92.         <value>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider</value>
  93.     </property>
  94.     <!--==================Namenode fencing：=============================================== -->
  95.     <!--Failover后防止停掉的Namenode启动，造成两个服务 -->
  96.     <property>
  97.         <name>dfs.ha.fencing.methods</name>
  98.         <value>sshfence</value>
  99.     </property>
  100.     <property>
  101.         <name>dfs.ha.fencing.ssh.private-key-files</name>
  102.         <value>/root/.ssh/id_rsa</value>
  103.     </property>
  104.     <property>
  105.         <!--多少milliseconds 认为fencing失败 -->
  106.         <name>dfs.ha.fencing.ssh.connect-timeout</name>
  107.         <value>30000</value>
  108.     </property>

  110.     <!--==================NameNode auto failover base ZKFC and Zookeeper====================== -->
  111.     <!--开启基于Zookeeper及ZKFC进程的自动备援设置,监视进程是否死掉 -->
  112.     <property>
  113.         <name>dfs.ha.automatic-failover.enabled</name>
  114.         <value>true</value>
  115.     </property>
  116.     <property>
  117.         <name>ha.zookeeper.quorum</name>
  118.         <value>sht-sgmhadoopdn-01:2181,sht-sgmhadoopdn-02:2181,sht-sgmhadoopdn-03:2181</value>
  119.     </property>
  120.     <property>
  121.         <!--指定ZooKeeper超时间隔，单位毫秒 -->
  122.         <name>ha.zookeeper.session-timeout.ms</name>
  123.         <value>2000</value>
  124.     </property>
  125. </configuration>

5.修改$HADOOP_HOME/etc/hadoop/yarn-env.sh

#Yarn Daemon Options

#export YARN_RESOURCEMANAGER_OPTS

#export YARN_NODEMANAGER_OPTS

#export YARN_PROXYSERVER_OPTS

#export HADOOP_JOB_HISTORYSERVER_OPTS

#Yarn Logs

export YARN_LOG_DIR="/hadoop/hadoop-2.7.2/logs"

6.修改$HADOOP_HOEM/etc/hadoop/mapred-site.xml

点击(此处)折叠或打开

  1. [root@sht-sgmhadoopnn-01 hadoop]# cp mapred-site.xml.template mapred-site.xml
  2. [root@sht-sgmhadoopnn-01 hadoop]# vi mapred-site.xml
  3. <configuration>
  4.     <!-- 配置 MapReduce Applications -->
  5.     <property>
  6.         <name>mapreduce.framework.name</name>
  7.         <value>yarn</value>
  8.     </property>
  9.     <!-- JobHistory Server ============================================================== -->
  10.     <!-- 配置 MapReduce JobHistory Server 地址 ，默认: 0.0.0.0:10020 -->
  11.     <property>
  12.         <name>mapreduce.jobhistory.address</name>
  13.         <value>sht-sgmhadoopnn-01:10020</value>
  14.     </property>
  15.     <!-- 配置 MapReduce JobHistory Server web ui 地址， 默认: 0.0.0.0:19888 -->
  16.     <property>
  17.         <name>mapreduce.jobhistory.webapp.address</name>
  18.         <value>sht-sgmhadoopnn-01:19888</value>
  19.     </property>
  20. </configuration>

7.修改$HADOOP_HOME/etc/hadoop/yarn-site.xml

点击(此处)折叠或打开

  1. <configuration>
  2.     <!-- nodemanager 配置 ================================================= -->
  3.     <property>
  4.         <name>yarn.nodemanager.aux-services</name>
  5.         <value>mapreduce_shuffle</value>
  6.     </property>
  7.     <property>
  8.         <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
  9.         <value>org.apache.hadoop.mapred.ShuffleHandler</value>
  10.     </property>
  11.     <property>
  12.         <description>Address where the localizer IPC is.</description>
  13.         <name>yarn.nodemanager.localizer.address</name>
  14.         <value>0.0.0.0:23344</value>
  15.     </property>
  16.     <property>
  17.         <description>NM Webapp address.</description>
  18.         <name>yarn.nodemanager.webapp.address</name>
  19.         <value>0.0.0.0:23999</value>
  20.     </property>

  22.     <!-- HA 配置 =============================================================== -->
  23.     <!-- Resource Manager Configs -->
  24.     <property>
  25.         <name>yarn.resourcemanager.connect.retry-interval.ms</name>
  26.         <value>2000</value>
  27.     </property>
  28.     <property>
  29.         <name>yarn.resourcemanager.ha.enabled</name>
  30.         <value>true</value>
  31.     </property>
  32.     <property>
  33.         <name>yarn.resourcemanager.ha.automatic-failover.enabled</name>
  34.         <value>true</value>
  35.     </property>
  36.     <!-- 使嵌入式自动故障转移。HA环境启动，与 ZKRMStateStore 配合 处理fencing -->
  37.     <property>
  38.         <name>yarn.resourcemanager.ha.automatic-failover.embedded</name>
  39.         <value>true</value>
  40.     </property>
  41.     <!-- 集群名称，确保HA选举时对应的集群 -->
  42.     <property>
  43.         <name>yarn.resourcemanager.cluster-id</name>
  44.         <value>yarn-cluster</value>
  45.     </property>
  46.     <property>
  47.         <name>yarn.resourcemanager.ha.rm-ids</name>
  48.         <value>rm1,rm2</value>
  49.     </property>
  50.     <!--这里RM主备结点需要单独指定,（可选）
  51.     <property>
  52.         <name>yarn.resourcemanager.ha.id</name>
  53.         <value>rm2</value>
  54. </property>
  55.  -->
  56.     <property>
  57.         <name>yarn.resourcemanager.scheduler.class</name>
  58.         <value>org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler</value>
  59.     </property>
  60.     <property>
  61.         <name>yarn.resourcemanager.recovery.enabled</name>
  62.         <value>true</value>
  63.     </property>
  64.     <property>
  65.         <name>yarn.app.mapreduce.am.scheduler.connection.wait.interval-ms</name>
  66.         <value>5000</value>
  67.     </property>
  68.     <!-- ZKRMStateStore 配置 -->
  69.     <property>
  70.         <name>yarn.resourcemanager.store.class</name>
  71.         <value>org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore</value>
  72.     </property>
  73.     <property>
  74.         <name>yarn.resourcemanager.zk-address</name>
  75.         <value>sht-sgmhadoopdn-01:2181,sht-sgmhadoopdn-02:2181,sht-sgmhadoopdn-03:2181</value>
  76.     </property>
  77.     <property>
  78.         <name>yarn.resourcemanager.zk.state-store.address</name>
  79.         <value>sht-sgmhadoopdn-01:2181,sht-sgmhadoopdn-02:2181,sht-sgmhadoopdn-03:2181</value>
  80.     </property>
  81.     <!-- Client访问RM的RPC地址 (applications manager interface) -->
  82.     <property>
  83.         <name>yarn.resourcemanager.address.rm1</name>
  84.         <value>sht-sgmhadoopnn-01:23140</value>
  85.     </property>
  86.     <property>
  87.         <name>yarn.resourcemanager.address.rm2</name>
  88.         <value>sht-sgmhadoopnn-02:23140</value>
  89.     </property>
  90.     <!-- AM访问RM的RPC地址(scheduler interface) -->
  91.     <property>
  92.         <name>yarn.resourcemanager.scheduler.address.rm1</name>
  93.         <value>sht-sgmhadoopnn-01:23130</value>
  94.     </property>
  95.     <property>
  96.         <name>yarn.resourcemanager.scheduler.address.rm2</name>
  97.         <value>sht-sgmhadoopnn-02:23130</value>
  98.     </property>
  99.     <!-- RM admin interface -->
  100.     <property>
  101.         <name>yarn.resourcemanager.admin.address.rm1</name>
  102.         <value>sht-sgmhadoopnn-01:23141</value>
  103.     </property>
  104.     <property>
  105.         <name>yarn.resourcemanager.admin.address.rm2</name>
  106.         <value>sht-sgmhadoopnn-02:23141</value>
  107.     </property>
  108.     <!--NM访问RM的RPC端口 -->
  109.     <property>
  110.         <name>yarn.resourcemanager.resource-tracker.address.rm1</name>
  111.         <value>sht-sgmhadoopnn-01:23125</value>
  112.     </property>
  113.     <property>
  114.         <name>yarn.resourcemanager.resource-tracker.address.rm2</name>
  115.         <value>sht-sgmhadoopnn-02:23125</value>
  116.     </property>
  117.     <!-- RM web application 地址 -->
  118.     <property>
  119.         <name>yarn.resourcemanager.webapp.address.rm1</name>
  120.         <value>sht-sgmhadoopnn-01:8088</value>
  121.     </property>
  122.     <property>
  123.         <name>yarn.resourcemanager.webapp.address.rm2</name>
  124.         <value>sht-sgmhadoopnn-02:8088</value>
  125.     </property>
  126.     <property>
  127.         <name>yarn.resourcemanager.webapp.https.address.rm1</name>
  128.         <value>sht-sgmhadoopnn-01:23189</value>
  129.     </property>
  130.     <property>
  131.         <name>yarn.resourcemanager.webapp.https.address.rm2</name>
  132.         <value>sht-sgmhadoopnn-02:23189</value>
  133.     </property>
  134. </configuration>

8.修改slaves

[root@sht-sgmhadoopnn-01 hadoop]# vi slaves

sht-sgmhadoopdn-01

sht-sgmhadoopdn-02

sht-sgmhadoopdn-03

9.分发文件夹

[root@sht-sgmhadoopnn-01 hadoop]# scp -r hadoop-2.7.2 root@sht-sgmhadoopnn-02:/hadoop

[root@sht-sgmhadoopnn-01 hadoop]# scp -r hadoop-2.7.2 root@sht-sgmhadoopdn-01:/hadoop

[root@sht-sgmhadoopnn-01 hadoop]# scp -r hadoop-2.7.2 root@sht-sgmhadoopdn-02:/hadoop

[root@sht-sgmhadoopnn-01 hadoop]# scp -r hadoop-2.7.2 root@sht-sgmhadoopdn-03:/hadoop

八.启动集群

另外一种启动方式:http://www.micmiu.com/bigdata/hadoop/hadoop2-cluster-ha-setup/

1.启动zookeeper

点击(此处)折叠或打开

  1. command: ./zkServer.sh start|stop|status
  2. [root@sht-sgmhadoopdn-01 bin]# ./zkServer.sh start
  3. JMX enabled by default
  4. Using config: /hadoop/zookeeper/bin/../conf/zoo.cfg
  5. Starting zookeeper ... STARTED
  6. [root@sht-sgmhadoopdn-01 bin]# jps
  7. 2073 QuorumPeerMain
  8. 2106 Jps
  9. [root@sht-sgmhadoopdn-02 bin]# ./zkServer.sh start
  10. JMX enabled by default
  11. Using config: /hadoop/zookeeper/bin/../conf/zoo.cfg
  12. Starting zookeeper ... STARTED
  13. [root@sht-sgmhadoopdn-02 bin]# jps
  14. 2073 QuorumPeerMain
  15. 2106 Jps
  16. [root@sht-sgmhadoopdn-03 bin]# ./zkServer.sh start
  17. JMX enabled by default
  18. Using config: /hadoop/zookeeper/bin/../conf/zoo.cfg
  19. Starting zookeeper ... STARTED
  20. [root@sht-sgmhadoopdn-03 bin]# jps
  21. 2073 QuorumPeerMain
  22. 2106 Jps

2.启动hadoop(HDFS+YARN)

a.格式化前,先在journalnode 节点机器上先启动JournalNode进程

点击(此处)折叠或打开

  1. [root@sht-sgmhadoopdn-01 ~]# cd /hadoop/hadoop-2.7.2/sbin
  2. [root@sht-sgmhadoopdn-01 sbin]# hadoop-daemon.sh start journalnode
  3. starting journalnode, logging to /hadoop/hadoop-2.7.2/logs/hadoop-root-journalnode-sht-sgmhadoopdn-03.telenav.cn.out
  4. [root@sht-sgmhadoopdn-03 sbin]# jps
  5. 16722 JournalNode
  6. 16775 Jps
  7. 15519 QuorumPeerMain
  8. [root@sht-sgmhadoopdn-02 ~]# cd /hadoop/hadoop-2.7.2/sbin
  9. [root@sht-sgmhadoopdn-02 sbin]# hadoop-daemon.sh start journalnode
  10. starting journalnode, logging to /hadoop/hadoop-2.7.2/logs/hadoop-root-journalnode-sht-sgmhadoopdn-03.telenav.cn.out
  11. [root@sht-sgmhadoopdn-03 sbin]# jps
  12. 16722 JournalNode
  13. 16775 Jps
  14. 15519 QuorumPeerMain
  15. [root@sht-sgmhadoopdn-03 ~]# cd /hadoop/hadoop-2.7.2/sbin
  16. [root@sht-sgmhadoopdn-03 sbin]# hadoop-daemon.sh start journalnode
  17. starting journalnode, logging to /hadoop/hadoop-2.7.2/logs/hadoop-root-journalnode-sht-sgmhadoopdn-03.telenav.cn.out
  18. [root@sht-sgmhadoopdn-03 sbin]# jps
  19. 16722 JournalNode
  20. 16775 Jps
  21. 15519 QuorumPeerMain

b.NameNode格式化

点击(此处)折叠或打开

  1. [root@sht-sgmhadoopnn-01 bin]# hadoop namenode -format
  2. 16/02/25 14:05:04 INFO namenode.NameNode: STARTUP_MSG:
  3. /************************************************************
  4. STARTUP_MSG: Starting NameNode
  5. STARTUP_MSG: host = sht-sgmhadoopnn-01.telenav.cn/172.16.101.55
  6. STARTUP_MSG: args = [-format]
  7. STARTUP_MSG: version = 2.7.2
  8. STARTUP_MSG: classpath =
  9. ……………..
  10. ………………
  11. 16/02/25 14:05:07 INFO namenode.FSNamesystem: dfs.namenode.safemode.threshold-pct = 0.9990000128746033
  12. 16/02/25 14:05:07 INFO namenode.FSNamesystem: dfs.namenode.safemode.min.datanodes = 0
  13. 16/02/25 14:05:07 INFO namenode.FSNamesystem: dfs.namenode.safemode.extension = 30000
  14. 16/02/25 14:05:07 INFO metrics.TopMetrics: NNTop conf: dfs.namenode.top.window.num.buckets = 10
  15. 16/02/25 14:05:07 INFO metrics.TopMetrics: NNTop conf: dfs.namenode.top.num.users = 10
  16. 16/02/25 14:05:07 INFO metrics.TopMetrics: NNTop conf: dfs.namenode.top.windows.minutes = 1,5,25
  17. 16/02/25 14:05:07 INFO namenode.FSNamesystem: Retry cache on namenode is enabled
  18. 16/02/25 14:05:07 INFO namenode.FSNamesystem: Retry cache will use 0.03 of total heap and retry cache entry expiry time is 600000 millis
  19. 16/02/25 14:05:07 INFO util.GSet: Computing capacity for map NameNodeRetryCache
  20. 16/02/25 14:05:07 INFO util.GSet: VM type = 64-bit
  21. 16/02/25 14:05:07 INFO util.GSet: 0.029999999329447746% max memory 889 MB = 273.1 KB
  22. 16/02/25 14:05:07 INFO util.GSet: capacity = 2^15 = 32768 entries
  23. 16/02/25 14:05:08 INFO namenode.FSImage: Allocated new BlockPoolId: BP-1182930464-172.16.101.55-1456380308394
  24. 16/02/25 14:05:08 INFO common.Storage: Storage directory /hadoop/hadoop-2.7.2/data/dfs/name has been successfully formatted.
  25. 16/02/25 14:05:08 INFO namenode.NNStorageRetentionManager: Going to retain 1 images with txid >= 0
  26. 16/02/25 14:05:08 INFO util.ExitUtil: Exiting with status 0
  27. 16/02/25 14:05:08 INFO namenode.NameNode: SHUTDOWN_MSG:
  28. /************************************************************
  29. SHUTDOWN_MSG: Shutting down NameNode at sht-sgmhadoopnn-01.telenav.cn/172.16.101.55
  30. ************************************************************/

c.同步NameNode元数据

点击(此处)折叠或打开

  1. 同步sht-sgmhadoopnn-01 元数据到sht-sgmhadoopnn-02
  2. 主要是：dfs.namenode.name.dir，dfs.namenode.edits.dir还应该确保共享存储目录下(dfs.namenode.shared.edits.dir ) 包含NameNode 所有的元数据。
  3. [root@sht-sgmhadoopnn-01 hadoop-2.7.2]# pwd
  4. /hadoop/hadoop-2.7.2
  5. [root@sht-sgmhadoopnn-01 hadoop-2.7.2]# scp -r data/ root@sht-sgmhadoopnn-02:/hadoop/hadoop-2.7.2
  6. seen_txid 100% 2 0.0KB/s 00:00
  7. fsimage_0000000000000000000 100% 351 0.3KB/s 00:00
  8. fsimage_0000000000000000000.md5 100% 62 0.1KB/s 00:00
  9. VERSION 100% 205 0.2KB/s 00:00

d.初始化ZFCK

点击(此处)折叠或打开

  1. [root@sht-sgmhadoopnn-01 bin]# hdfs zkfc -formatZK
  2. ……………..
  3. ……………..
  4. 16/02/25 14:14:41 INFO zookeeper.ZooKeeper: Client environment:user.home=/root
  5. 16/02/25 14:14:41 INFO zookeeper.ZooKeeper: Client environment:user.dir=/hadoop/hadoop-2.7.2/bin
  6. 16/02/25 14:14:41 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=sht-sgmhadoopdn-01:2181,sht-sgmhadoopdn-02:2181,sht-sgmhadoopdn-03:2181 sessionTimeout=2000 watcher=org.apache.hadoop.ha.ActiveStandbyElector$WatcherWithClientRef@5f4298a5
  7. 16/02/25 14:14:41 INFO zookeeper.ClientCnxn: Opening socket connection to server sht-sgmhadoopdn-01.telenav.cn/172.16.101.58:2181. Will not attempt to authenticate using SASL (unknown error)
  8. 16/02/25 14:14:41 INFO zookeeper.ClientCnxn: Socket connection established to sht-sgmhadoopdn-01.telenav.cn/172.16.101.58:2181, initiating session
  9. 16/02/25 14:14:42 INFO zookeeper.ClientCnxn: Session establishment complete on server sht-sgmhadoopdn-01.telenav.cn/172.16.101.58:2181, sessionid = 0x15316c965750000, negotiated timeout = 4000
  10. 16/02/25 14:14:42 INFO ha.ActiveStandbyElector: Session connected.
  11. 16/02/25 14:14:42 INFO ha.ActiveStandbyElector: Successfully created /hadoop-ha/mycluster in ZK.
  12. 16/02/25 14:14:42 INFO zookeeper.ClientCnxn: EventThread shut down
  13. 16/02/25 14:14:42 INFO zookeeper.ZooKeeper: Session: 0x15316c965750000 closed

e.启动HDFS 系统

集群启动,在sht-sgmhadoopnn-01执行start-dfs.sh

集群关闭,在sht-sgmhadoopnn-01执行stop-dfs.sh

#####集群启动############

点击(此处)折叠或打开

  1. [root@sht-sgmhadoopnn-01 sbin]# start-dfs.sh
  2. 16/02/25 14:21:42 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
  3. Starting namenodes on [sht-sgmhadoopnn-01 sht-sgmhadoopnn-02]
  4. sht-sgmhadoopnn-01: starting namenode, logging to /hadoop/hadoop-2.7.2/logs/hadoop-root-namenode-sht-sgmhadoopnn-01.telenav.cn.out
  5. sht-sgmhadoopnn-02: starting namenode, logging to /hadoop/hadoop-2.7.2/logs/hadoop-root-namenode-sht-sgmhadoopnn-02.telenav.cn.out
  6. sht-sgmhadoopdn-01: starting datanode, logging to /hadoop/hadoop-2.7.2/logs/hadoop-root-datanode-sht-sgmhadoopdn-01.telenav.cn.out
  7. sht-sgmhadoopdn-02: starting datanode, logging to /hadoop/hadoop-2.7.2/logs/hadoop-root-datanode-sht-sgmhadoopdn-02.telenav.cn.out
  8. sht-sgmhadoopdn-03: starting datanode, logging to /hadoop/hadoop-2.7.2/logs/hadoop-root-datanode-sht-sgmhadoopdn-03.telenav.cn.out
  9. Starting journal nodes [sht-sgmhadoopdn-01 sht-sgmhadoopdn-02 sht-sgmhadoopdn-03]
  10. sht-sgmhadoopdn-01: journalnode running as process 6348. Stop it first.
  11. sht-sgmhadoopdn-03: journalnode running as process 16722. Stop it first.
  12. sht-sgmhadoopdn-02: journalnode running as process 7197. Stop it first.
  13. 16/02/25 14:21:56 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
  14. Starting ZK Failover Controllers on NN hosts [sht-sgmhadoopnn-01 sht-sgmhadoopnn-02]
  15. sht-sgmhadoopnn-01: starting zkfc, logging to /hadoop/hadoop-2.7.2/logs/hadoop-root-zkfc-sht-sgmhadoopnn-01.telenav.cn.out
  16. sht-sgmhadoopnn-02: starting zkfc, logging to /hadoop/hadoop-2.7.2/logs/hadoop-root-zkfc-sht-sgmhadoopnn-02.telenav.cn.out
  17. You have mail in /var/spool/mail/root

####单进程启动###########

NameNode(sht-sgmhadoopnn-01, sht-sgmhadoopnn-02):

hadoop-daemon.sh start namenode

DataNode(sht-sgmhadoopdn-01, sht-sgmhadoopdn-02, sht-sgmhadoopdn-03):

hadoop-daemon.sh start datanode

JournamNode(sht-sgmhadoopdn-01, sht-sgmhadoopdn-02, sht-sgmhadoopdn-03):

hadoop-daemon.sh start journalnode

ZKFC(sht-sgmhadoopnn-01, sht-sgmhadoopnn-02):

hadoop-daemon.sh start zkfc

f.验证namenode,datanode,zkfc

1) 进程

点击(此处)折叠或打开

  1. [root@sht-sgmhadoopnn-01 sbin]# jps
  2. 12712 Jps
  3. 12593 DFSZKFailoverController
  4. 12278 NameNode
  5. [root@sht-sgmhadoopnn-02 ~]# jps
  6. 29714 NameNode
  7. 29849 DFSZKFailoverController
  8. 30229 Jps
  9. [root@sht-sgmhadoopdn-01 ~]# jps
  10. 6348 JournalNode
  11. 8775 Jps
  12. 559 QuorumPeerMain
  13. 8509 DataNode
  14. [root@sht-sgmhadoopdn-02 ~]# jps
  15. 9430 Jps
  16. 9160 DataNode
  17. 7197 JournalNode
  18. 2073 QuorumPeerMain
  19. [root@sht-sgmhadoopdn-03 ~]# jps
  20. 16722 JournalNode
  21. 17369 Jps
  22. 15519 QuorumPeerMain
  23. 17214 DataNode

2) 页面

sht-sgmhadoopnn-01:

http://172.16.101.55:50070/

sht-sgmhadoopnn-02:

http://172.16.101.56:50070/

g.启动YARN运算框架

#####集群启动############

1) sht-sgmhadoopnn-01启动Yarn，命令所在目录：$HADOOP_HOME/sbin

点击(此处)折叠或打开

  1. [root@sht-sgmhadoopnn-01 sbin]# start-yarn.sh
  2. starting yarn daemons
  3. starting resourcemanager, logging to /hadoop/hadoop-2.7.2/logs/yarn-root-resourcemanager-sht-sgmhadoopnn-01.telenav.cn.out
  4. sht-sgmhadoopdn-03: starting nodemanager, logging to /hadoop/hadoop-2.7.2/logs/yarn-root-nodemanager-sht-sgmhadoopdn-03.telenav.cn.out
  5. sht-sgmhadoopdn-02: starting nodemanager, logging to /hadoop/hadoop-2.7.2/logs/yarn-root-nodemanager-sht-sgmhadoopdn-02.telenav.cn.out
  6. sht-sgmhadoopdn-01: starting nodemanager, logging to /hadoop/hadoop-2.7.2/logs/yarn-root-nodemanager-sht-sgmhadoopdn-01.telenav.cn.out

2) sht-sgmhadoopnn-02备机启动RM

点击(此处)折叠或打开

  1. [root@sht-sgmhadoopnn-02 sbin]# yarn-daemon.sh start resourcemanager
  2. starting resourcemanager, logging to /hadoop/hadoop-2.7.2/logs/yarn-root-resourcemanager-sht-sgmhadoopnn-02.telenav.cn.out

####单进程启动###########

1) ResourceManager(sht-sgmhadoopnn-01, sht-sgmhadoopnn-02)

yarn-daemon.sh start resourcemanager

2) NodeManager(sht-sgmhadoopdn-01, sht-sgmhadoopdn-02, sht-sgmhadoopdn-03)

yarn-daemon.sh start nodemanager

######关闭#############

[root@sht-sgmhadoopnn-01 sbin]# stop-yarn.sh

#包含namenode的resourcemanager进程，datanode的nodemanager进程

[root@sht-sgmhadoopnn-02 sbin]# yarn-daemon.sh stop resourcemanager

h.验证resourcemanager,nodemanager

1) 进程

点击(此处)折叠或打开

  1. [root@sht-sgmhadoopnn-01 sbin]# jps
  2. 13611 Jps
  3. 12593 DFSZKFailoverController
  4. 12278 NameNode
  5. 13384 ResourceManager
  6. [root@sht-sgmhadoopnn-02 sbin]# jps
  7. 32265 ResourceManager
  8. 32304 Jps
  9. 29714 NameNode
  10. 29849 DFSZKFailoverController
  11. [root@sht-sgmhadoopdn-01 ~]# jps
  12. 6348 JournalNode
  13. 559 QuorumPeerMain
  14. 8509 DataNode
  15. 10286 NodeManager
  16. 10423 Jps
  17. [root@sht-sgmhadoopdn-02 ~]# jps
  18. 9160 DataNode
  19. 10909 NodeManager
  20. 11937 Jps
  21. 7197 JournalNode
  22. 2073 QuorumPeerMain
  23. [root@sht-sgmhadoopdn-03 ~]# jps
  24. 18031 Jps
  25. 16722 JournalNode
  26. 17710 NodeManager
  27. 15519 QuorumPeerMain
  28. 17214 DataNode

2) 页面

ResourceManger（Active）：http://172.16.101.55:8088

ResourceManger（Standby）：http://172.16.101.56:8088/cluster/cluster

九.监控集群

[root@sht-sgmhadoopnn-01 ~]# hdfs dfsadmin -report

十.附件及参考

#http://archive-primary.cloudera.com/cdh5/cdh/5/hadoop-2.6.0-cdh5.5.2.tar.gz

#http://archive-primary.cloudera.com/cdh5/cdh/5/zookeeper-3.4.5-cdh5.5.2.tar.gz

hadoop : http://mirror.bit.edu.cn/apache/hadoop/common/hadoop-2.7.2/hadoop-2.7.2.tar.gz

zookeeper :http://mirror.bit.edu.cn/apache/zookeeper/zookeeper-3.4.6/zookeeper-3.4.6.tar.gz

参考:

Hadoop-2.3.0-cdh5.0.1完全分布式环境搭建(NameNode,ResourceManager HA):

http://blog.itpub.net/30089851/viewspace-1987620/

如何解决这类问题：The string "--" is not permitted within comments:

http://blog.csdn.net/free4294/article/details/38681095

SecureCRT连接linux终端中文显示乱码解决办法:

http://www.cnblogs.com/qi09/archive/2013/02/05/2892922.html


参照:http://blog.itpub.net/30089851/viewspace-1987620/





来自 “ ITPUB博客 ” ，链接：https://blog.itpub.net/30089851/viewspace-1994585/ ，如需转载，请注明出处，否则将追究法律责任。

0
0
上一篇： jps报27342 -- process information unavailable
下一篇： Hadoop资料整理导向
请登录后发表评论 登录
全部评论
hackeruncle
专家姓名：武基鹏
主要从事大数据平台产品的技术工作； 负责基于Spark、Flink生产开发及性能调优； 负责对公司的集群的部署、监控、调优和运营； 主导我司的数据平台0到1及实时中间件项目开发； 此外,精通Java、Shell、Python编程； 且荣获2016年IBM的Spark大赛全球第四名，中国区域第一名。

注册时间：2015-01-03

  • 博文量
    187
  • 访问量
    396846

最新文章

支持我们 作者招募 用户协议 FAQ Contact Us 编辑准则

北京盛拓优讯信息技术有限公司. 版权所有  京ICP备16024965号-8 北京公安局海淀分局网监中心备案编号：11010802021510 niuxiaotong@pcpop.com 17352615567 未成年人举报专区

广播电视节目制作经营许可证(京) 字第1234号 中国互联网协会会员

违法和不良信息举报电话:18101376593