0.集群规划
主机名 ip地址 安装的软件 运行的进程
hadoop1 10.1.245.72 hadoop,zookeeper namenode,zkfc,resourcemanager,journalnode
hadoop2 10.1.245.73 hadoop,zookeeper namenode,zkfc,resourcemanager,datanode,nodemanager,journalnode
hadoop3 10.1.245.74 hadoop,zookeeper datanode,nodemanager,journalnode
1.安装jdk
mkdir -p /opt/freeware/
cd /opt/freeware/
tar xvf jdk1.7.0_79.tgz
vi /etc/profile
export JAVA_HOME=/opt/freeware/jdk1.7.0_79
export PATH=$JAVA_HOME/bin:$PATH
source /etc/profile
java -version
java version "1.7.0_79"
2.创建用户和配置hosts文件
groupadd -g 511 hadoop
useradd -u 511 -g hadoop hdpusr01
echo oracle|passwd --stdin hdpusr01
vi /etc/hosts
10.1.245.72 hadoop1
10.1.245.73 hadoop2
10.1.245.74 hadoop3
3.配置hadoop用户ssh互信
3.1 生产rsa公钥对
su - hdpusr01
ssh-keygen -t rsa
cat ~/.ssh/id_rsa.pub > ~/.ssh/authorized_keys
chmod 600 .ssh/authorized_keys
3.2 配置授权文件--在主机hadoop1操作
[hdpusr01@hadoop1 ~]$ ssh hadoop2 cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
[hdpusr01@hadoop1 ~]$ ssh hadoop3 cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
[hdpusr01@hadoop1 ~]$ scp ~/.ssh/authorized_keys hadoop2:/home/hdpusr01/.ssh/authorized_keys
[hdpusr01@hadoop1 ~]$ scp ~/.ssh/authorized_keys hadoop3:/home/hdpusr01/.ssh/authorized_keys
3.3 验证ssh互信
ssh hadoop1 date
ssh hadoop2 date
ssh hadoop3 date
4.安装Hadoop集群
4.1 安装ZooKeeper集群
具体步骤请参考《小丸子学ZooKeeper系列之——部署ZooKeeper集群 》
http://blog.itpub.net/20801486/viewspace-1866925/
4.2 安装hadoop集群
4.2.1 解压软件--在主机hadoop1操作
[hdpusr01@hadoop1 ~]$ tar xvf /opt/freeware/hadoop-2.6.0.tgz
[hdpusr01@hadoop1 ~]$ chown -R hdpusr01:hadoop hadoop-2.6.0/
[hdpusr01@hadoop1 ~]$ mv hadoop-2.6.0/ hadoop
4.2.2 配置hadoop--在主机hadoop1操作
4.2.2.1 修改core-site.xml
[hdpusr01@hadoop1 ~]$ vi core-site.xml
01:
02:
03: fs.defaultFS
04: hdfs://hdfscls1
05:
06:
07: io.file.buffer.size
08: 131072
09:
10:
11: hadoop.tmp.dir
12: /home/hdpusr01/hadoop/tmp
13:
14:
15: ha.zookeeper.quorum
16: hadoop1:29181,hadoop2:29181,hadoop3:29181
17:
18:
4.2.2.2 修改hdfs-stie.xml
[hdpusr01@hadoop1 ~]$ vi hadoop/etc/hadoop/hdfs-site.xml
01:
02:
03: dfs.nameservices
04: hdfscls1
05:
06:
07: dfs.ha.namenodes.hdfscls1
08: nn1,nn2
09:
10:
11: dfs.namenode.rpc-address.hdfscls1.nn1
12: hadoop1:8920
13:
14:
15: dfs.namenode.http-address.hdfscls1.nn1
16: hadoop1:59070
17:
18:
19: dfs.namenode.rpc-address.hdfscls1.nn2
20: hadoop2:8920
21:
22:
23: dfs.namenode.http-address.hdfscls1.nn2
24: hadoop2:59070
25:
26:
27: dfs.namenode.shared.edits.dir
28: qjournal://hadoop1:8485;hadoop2:8485;hadoop3:8485/hdfscls1
29:
30:
31: dfs.journalnode.edits.dir
32: /home/hdpusr01/hadoop/dfs/jn
33:
34:
35: dfs.journalnode.rpc-address
36: 0.0.0.0:8485
37:
38:
39: dfs.journalnode.http-address
40: 0.0.0.0:8480
41:
42:
43: dfs.datanode.address
44: 0.0.0.0:59010
45:
46:
47: dfs.datanode.http.address
48: 0.0.0.0:59075
49:
50:
51: dfs.datanode.ipc.address
52: 0.0.0.0:59020
53:
54:
55: dfs.ha.automatic-failover.enabled
56: true
57:
58:
59: dfs.client.failover.proxy.provider.hdfscls1
60: org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider
61:
62:
63: dfs.ha.fencing.methods
64: sshfence
65:
66:
67: dfs.ha.fencing.ssh.private-key-files
68: /home/hdpusr01/.ssh/id_rsa
69:
70:
71: dfs.ha.fencing.ssh.connect-timeout
72: 30000
73:
74:
75: dfs.datanode.data.dir
76: file:/home/hdpusr01/hadoop/dfs/dn/data01
77:
78:
4.2.2.3 修改mapred-site.xml
[hdpusr01@hadoop1 ~]$ vi hadoop/etc/hadoop/mapred-site.xml
01:
02:
03: mapreduce.framework.name
04: yarn
05:
06:
4.2.2.4 修改yarn-site.xml
[hdpusr01@hadoop1 ~]$ vi hadoop/etc/hadoop/yarn-site.xml
01:
02:
03: yarn.resourcemanager.ha.enabled
04: true
05:
06:
07: yarn.resourcemanager.cluster-id
08: rmcls1
09:
10:
11: yarn.resourcemanager.ha.rm-ids
12: rm1,rm2
13:
14:
15: yarn.resourcemanager.hostname.rm1
16: hadoop1
17:
18:
19: yarn.resourcemanager.hostname.rm2
20: hadoop2
21:
22:
23: yarn.resourcemanager.recovery.enabled
24: true
25:
26:
27: yarn.resourcemanager.store.class
28: org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore
29:
30:
31: yarn.resourcemanager.zk-address
32: hadoop1:29181,hadoop2:29181,hadoop3:29181
33:
34:
35: yarn.nodemanager.aux-services
36: mapreduce_shuffle
37:
38:
4.2.2.5 修改slaves
[hdpusr01@hadoop1 ~]$ vi hadoop/etc/hadoop/slaves
hadoop2
hadoop3
4.2.2.6 创建hadoop相关目录
[hdpusr01@hadoop1 ~]$ mkdir -p hadoop/{tmp,dfs/jn,dfs/dn/data01}
4.2.3 拷贝hadoop目录到其他节点并解压
[hdpusr01@hadoop1 ~]$ tar cvf hdp.tar hadoop/
[hdpusr01@hadoop1 ~]$ scp hdp.tar hadoop2:/home/hdpusr01
[hdpusr01@hadoop1 ~]$ scp hdp.tar hadoop3:/home/hdpusr01
[hdpusr01@hadoop2 ~]$ tar xvf hdp.tar
[hdpusr01@hadoop3 ~]$ tar xvf hdp.tar
4.2.4 检查zookeeper集群状态
[hdpusr01@hadoop1 ~]$ zookeeper/bin/zkServer.sh status
Mode: follower
[hdpusr01@hadoop2 ~]$ zookeeper/bin/zkServer.sh status
Mode: leader
[hdpusr01@hadoop3 ~]$ zookeeper/bin/zkServer.sh status
Mode: follower
4.2.5 启动journalnode——在主机hadoop1~hadoop3操作
[hdpusr01@hadoop1 ~]$ hadoop/sbin/hadoop-daemon.sh start journalnode
starting journalnode, logging to /home/hdpusr01/hadoop/logs/hadoop-hdpusr01-journalnode-hadoop1.out
[hdpusr01@hadoop1 ~]$ jps
1766 JournalNode
注：启动正常的话用jps命令可以看到JournalNode进程
4.2.6 格式化HDFS——在主机hadoop1操作
[hdpusr01@hadoop1 ~]$ hadoop/bin/hdfs namenode -format
15/12/21 17:55:54 INFO namenode.FSImage: Allocated new BlockPoolId: BP-1012038946-10.1.245.72-1450691754279
15/12/21 17:55:54 INFO common.Storage: Storage directory /home/hdpusr01/hadoop/tmp/dfs/name has been successfully formatted.
15/12/21 17:55:54 INFO namenode.NNStorageRetentionManager: Going to retain 1 images with txid >= 0
注：从日志信息看出hdfs格式化成功，生产的文件在目录/home/hdpusr01/hadoop/tmp/dfs/name中
4.2.7 拷贝namenode临时目录到主机hadoop2——在主机hadoop1操作
[hdpusr01@hadoop1 ~]$ scp -r hadoop/tmp/* hadoop2:/home/hdpusr01/hadoop/tmp
4.2.8 格式化ZK——在主机hadoop1操作
[hdpusr01@hadoop1 ~]$ hadoop/bin/hdfs zkfc -formatZK
15/12/21 18:00:56 INFO zookeeper.ClientCnxn: Session establishment complete on server hadoop1/10.1.245.72:29181, sessionid = 0x151c3bc8d780000, negotiated timeout = 80000
15/12/21 18:00:56 INFO ha.ActiveStandbyElector: Successfully created /hadoop-ha/hdfscls1 in ZK.
注：从日志信息看出hadoop已经与zookeeper建立连接，并且在zk中创建了/hadoop-ha/hdfscls1的znode节点，下面登录zk验证一下。
[hdpusr01@hadoop1 ~]$ zookeeper/bin/zkCli.sh -server hadoop1:29181
WATCHER::
WatchedEvent state:SyncConnected type:None path:null
[zk: hadoop1:29181(CONNECTED) 2] get /hadoop-ha/hdfscls1
cZxid = 0x100000003
ctime = Mon Dec 21 18:00:56 CST 2015
mZxid = 0x100000003
mtime = Mon Dec 21 18:00:56 CST 2015
pZxid = 0x100000003
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 0
numChildren = 0
4.2.9 启动HDFS——在主机hadoop1操作
[hdpusr01@hadoop1 ~]$ hadoop/sbin/start-dfs.sh
2015-12-21 18:26:03,341 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2015-12-21 18:26:03,997 WARN org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Only one image storage directory (dfs.namenode.name.dir) configured. Beware of data loss due to lack of redundant storage directories!
注：启动hdfs过程中遇到一个告警，提示无法加载native-hadoop library，尝试设置环境变量后再次启动hdfs，警告仍然存在
[hdpusr01@hadoop1 ~]$ jps
16966 NameNode
17161 JournalNode
17333 DFSZKFailoverController
[hdpusr01@hadoop2 ~]$ jps
8907 DFSZKFailoverController
8784 JournalNode
8697 DataNode
8626 NameNode
[hdpusr01@hadoop3 ~]$ jps
3339 DataNode
3426 JournalNode
注：从上面jps命令输出可以看到hadoop1~2主机上的NameNode进程，hadoop2~3主机上的DataNode进程以及hadoop1~3主机上的JournalNode进程都启动了
4.2.10 启动YARN——在主机hadoop1操作
[hdpusr01@hadoop1 ~]$ hadoop/sbin/start-yarn.sh
[hdpusr01@hadoop1 ~]$ jps
21031 ResourceManager
[hdpusr01@hadoop2 ~]$ jps
10634 NodeManager
[hdpusr01@hadoop3 ~]$ jps
3760 NodeManager
注：从上面jps命令输出可以看到hadoop1主机上的ResourceManager进程，hadoop2~3主机上的NodeManager进程都启动了，hadoop2主机上的ResourceManager进程需要手动启动
4.2.11 启动ResourceManager——在主机hadoop2操作
[hdpusr01@hadoop2 ~]$ hadoop/sbin/yarn-daemon.sh start resourcemanager
starting resourcemanager, logging to /home/hdpusr01/hadoop/logs/yarn-hdpusr01-resourcemanager-hadoop2.out
[hdpusr01@hadoop2 ~]$ jps
10860 ResourceManager
注：至此hadoop集群部署完毕
4.3 查看集群状态——通过web页面查看
4.3.1 查看hadoop1上的NameNode
http://10.1.245.72:59070
4.3.2 查看hadoop2上的NameNode和DataNode
http://10.1.245.73:59070
注：从hadoop提供的web页面可以看到NameNode和DataNode的进程信息，一切正常,至此Hadoop集群搭建完毕。
主机名 ip地址 安装的软件 运行的进程
hadoop1 10.1.245.72 hadoop,zookeeper namenode,zkfc,resourcemanager,journalnode
hadoop2 10.1.245.73 hadoop,zookeeper namenode,zkfc,resourcemanager,datanode,nodemanager,journalnode
hadoop3 10.1.245.74 hadoop,zookeeper datanode,nodemanager,journalnode
1.安装jdk
mkdir -p /opt/freeware/
cd /opt/freeware/
tar xvf jdk1.7.0_79.tgz
vi /etc/profile
export JAVA_HOME=/opt/freeware/jdk1.7.0_79
export PATH=$JAVA_HOME/bin:$PATH
source /etc/profile
java -version
java version "1.7.0_79"
2.创建用户和配置hosts文件
groupadd -g 511 hadoop
useradd -u 511 -g hadoop hdpusr01
echo oracle|passwd --stdin hdpusr01
vi /etc/hosts
10.1.245.72 hadoop1
10.1.245.73 hadoop2
10.1.245.74 hadoop3
3.配置hadoop用户ssh互信
3.1 生产rsa公钥对
su - hdpusr01
ssh-keygen -t rsa
cat ~/.ssh/id_rsa.pub > ~/.ssh/authorized_keys
chmod 600 .ssh/authorized_keys
3.2 配置授权文件--在主机hadoop1操作
[hdpusr01@hadoop1 ~]$ ssh hadoop2 cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
[hdpusr01@hadoop1 ~]$ ssh hadoop3 cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
[hdpusr01@hadoop1 ~]$ scp ~/.ssh/authorized_keys hadoop2:/home/hdpusr01/.ssh/authorized_keys
[hdpusr01@hadoop1 ~]$ scp ~/.ssh/authorized_keys hadoop3:/home/hdpusr01/.ssh/authorized_keys
3.3 验证ssh互信
ssh hadoop1 date
ssh hadoop2 date
ssh hadoop3 date
4.安装Hadoop集群
4.1 安装ZooKeeper集群
具体步骤请参考《小丸子学ZooKeeper系列之——部署ZooKeeper集群 》
http://blog.itpub.net/20801486/viewspace-1866925/
4.2 安装hadoop集群
4.2.1 解压软件--在主机hadoop1操作
[hdpusr01@hadoop1 ~]$ tar xvf /opt/freeware/hadoop-2.6.0.tgz
[hdpusr01@hadoop1 ~]$ chown -R hdpusr01:hadoop hadoop-2.6.0/
[hdpusr01@hadoop1 ~]$ mv hadoop-2.6.0/ hadoop
4.2.2 配置hadoop--在主机hadoop1操作
4.2.2.1 修改core-site.xml
[hdpusr01@hadoop1 ~]$ vi core-site.xml
01:
02:
03: fs.defaultFS
04: hdfs://hdfscls1
05:
06:
07: io.file.buffer.size
08: 131072
09:
10:
11: hadoop.tmp.dir
12: /home/hdpusr01/hadoop/tmp
13:
14:
15: ha.zookeeper.quorum
16: hadoop1:29181,hadoop2:29181,hadoop3:29181
17:
18:
4.2.2.2 修改hdfs-stie.xml
[hdpusr01@hadoop1 ~]$ vi hadoop/etc/hadoop/hdfs-site.xml
01:
02:
03: dfs.nameservices
04: hdfscls1
05:
06:
07: dfs.ha.namenodes.hdfscls1
08: nn1,nn2
09:
10:
11: dfs.namenode.rpc-address.hdfscls1.nn1
12: hadoop1:8920
13:
14:
15: dfs.namenode.http-address.hdfscls1.nn1
16: hadoop1:59070
17:
18:
19: dfs.namenode.rpc-address.hdfscls1.nn2
20: hadoop2:8920
21:
22:
23: dfs.namenode.http-address.hdfscls1.nn2
24: hadoop2:59070
25:
26:
27: dfs.namenode.shared.edits.dir
28: qjournal://hadoop1:8485;hadoop2:8485;hadoop3:8485/hdfscls1
29:
30:
31: dfs.journalnode.edits.dir
32: /home/hdpusr01/hadoop/dfs/jn
33:
34:
35: dfs.journalnode.rpc-address
36: 0.0.0.0:8485
37:
38:
39: dfs.journalnode.http-address
40: 0.0.0.0:8480
41:
42:
43: dfs.datanode.address
44: 0.0.0.0:59010
45:
46:
47: dfs.datanode.http.address
48: 0.0.0.0:59075
49:
50:
51: dfs.datanode.ipc.address
52: 0.0.0.0:59020
53:
54:
55: dfs.ha.automatic-failover.enabled
56: true
57:
58:
59: dfs.client.failover.proxy.provider.hdfscls1
60: org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider
61:
62:
63: dfs.ha.fencing.methods
64: sshfence
65:
66:
67: dfs.ha.fencing.ssh.private-key-files
68: /home/hdpusr01/.ssh/id_rsa
69:
70:
71: dfs.ha.fencing.ssh.connect-timeout
72: 30000
73:
74:
75: dfs.datanode.data.dir
76: file:/home/hdpusr01/hadoop/dfs/dn/data01
77:
78:
4.2.2.3 修改mapred-site.xml
[hdpusr01@hadoop1 ~]$ vi hadoop/etc/hadoop/mapred-site.xml
01:
02:
03: mapreduce.framework.name
04: yarn
05:
06:
4.2.2.4 修改yarn-site.xml
[hdpusr01@hadoop1 ~]$ vi hadoop/etc/hadoop/yarn-site.xml
01:
02:
03: yarn.resourcemanager.ha.enabled
04: true
05:
06:
07: yarn.resourcemanager.cluster-id
08: rmcls1
09:
10:
11: yarn.resourcemanager.ha.rm-ids
12: rm1,rm2
13:
14:
15: yarn.resourcemanager.hostname.rm1
16: hadoop1
17:
18:
19: yarn.resourcemanager.hostname.rm2
20: hadoop2
21:
22:
23: yarn.resourcemanager.recovery.enabled
24: true
25:
26:
27: yarn.resourcemanager.store.class
28: org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore
29:
30:
31: yarn.resourcemanager.zk-address
32: hadoop1:29181,hadoop2:29181,hadoop3:29181
33:
34:
35: yarn.nodemanager.aux-services
36: mapreduce_shuffle
37:
38:
4.2.2.5 修改slaves
[hdpusr01@hadoop1 ~]$ vi hadoop/etc/hadoop/slaves
hadoop2
hadoop3
4.2.2.6 创建hadoop相关目录
[hdpusr01@hadoop1 ~]$ mkdir -p hadoop/{tmp,dfs/jn,dfs/dn/data01}
4.2.3 拷贝hadoop目录到其他节点并解压
[hdpusr01@hadoop1 ~]$ tar cvf hdp.tar hadoop/
[hdpusr01@hadoop1 ~]$ scp hdp.tar hadoop2:/home/hdpusr01
[hdpusr01@hadoop1 ~]$ scp hdp.tar hadoop3:/home/hdpusr01
[hdpusr01@hadoop2 ~]$ tar xvf hdp.tar
[hdpusr01@hadoop3 ~]$ tar xvf hdp.tar
4.2.4 检查zookeeper集群状态
[hdpusr01@hadoop1 ~]$ zookeeper/bin/zkServer.sh status
Mode: follower
[hdpusr01@hadoop2 ~]$ zookeeper/bin/zkServer.sh status
Mode: leader
[hdpusr01@hadoop3 ~]$ zookeeper/bin/zkServer.sh status
Mode: follower
4.2.5 启动journalnode——在主机hadoop1~hadoop3操作
[hdpusr01@hadoop1 ~]$ hadoop/sbin/hadoop-daemon.sh start journalnode
starting journalnode, logging to /home/hdpusr01/hadoop/logs/hadoop-hdpusr01-journalnode-hadoop1.out
[hdpusr01@hadoop1 ~]$ jps
1766 JournalNode
注：启动正常的话用jps命令可以看到JournalNode进程
4.2.6 格式化HDFS——在主机hadoop1操作
[hdpusr01@hadoop1 ~]$ hadoop/bin/hdfs namenode -format
15/12/21 17:55:54 INFO namenode.FSImage: Allocated new BlockPoolId: BP-1012038946-10.1.245.72-1450691754279
15/12/21 17:55:54 INFO common.Storage: Storage directory /home/hdpusr01/hadoop/tmp/dfs/name has been successfully formatted.
15/12/21 17:55:54 INFO namenode.NNStorageRetentionManager: Going to retain 1 images with txid >= 0
注：从日志信息看出hdfs格式化成功，生产的文件在目录/home/hdpusr01/hadoop/tmp/dfs/name中
4.2.7 拷贝namenode临时目录到主机hadoop2——在主机hadoop1操作
[hdpusr01@hadoop1 ~]$ scp -r hadoop/tmp/* hadoop2:/home/hdpusr01/hadoop/tmp
4.2.8 格式化ZK——在主机hadoop1操作
[hdpusr01@hadoop1 ~]$ hadoop/bin/hdfs zkfc -formatZK
15/12/21 18:00:56 INFO zookeeper.ClientCnxn: Session establishment complete on server hadoop1/10.1.245.72:29181, sessionid = 0x151c3bc8d780000, negotiated timeout = 80000
15/12/21 18:00:56 INFO ha.ActiveStandbyElector: Successfully created /hadoop-ha/hdfscls1 in ZK.
注：从日志信息看出hadoop已经与zookeeper建立连接，并且在zk中创建了/hadoop-ha/hdfscls1的znode节点，下面登录zk验证一下。
[hdpusr01@hadoop1 ~]$ zookeeper/bin/zkCli.sh -server hadoop1:29181
WATCHER::
WatchedEvent state:SyncConnected type:None path:null
[zk: hadoop1:29181(CONNECTED) 2] get /hadoop-ha/hdfscls1
cZxid = 0x100000003
ctime = Mon Dec 21 18:00:56 CST 2015
mZxid = 0x100000003
mtime = Mon Dec 21 18:00:56 CST 2015
pZxid = 0x100000003
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 0
numChildren = 0
4.2.9 启动HDFS——在主机hadoop1操作
[hdpusr01@hadoop1 ~]$ hadoop/sbin/start-dfs.sh
2015-12-21 18:26:03,341 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2015-12-21 18:26:03,997 WARN org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Only one image storage directory (dfs.namenode.name.dir) configured. Beware of data loss due to lack of redundant storage directories!
注：启动hdfs过程中遇到一个告警，提示无法加载native-hadoop library，尝试设置环境变量后再次启动hdfs，警告仍然存在
[hdpusr01@hadoop1 ~]$ jps
16966 NameNode
17161 JournalNode
17333 DFSZKFailoverController
[hdpusr01@hadoop2 ~]$ jps
8907 DFSZKFailoverController
8784 JournalNode
8697 DataNode
8626 NameNode
[hdpusr01@hadoop3 ~]$ jps
3339 DataNode
3426 JournalNode
注：从上面jps命令输出可以看到hadoop1~2主机上的NameNode进程，hadoop2~3主机上的DataNode进程以及hadoop1~3主机上的JournalNode进程都启动了
4.2.10 启动YARN——在主机hadoop1操作
[hdpusr01@hadoop1 ~]$ hadoop/sbin/start-yarn.sh
[hdpusr01@hadoop1 ~]$ jps
21031 ResourceManager
[hdpusr01@hadoop2 ~]$ jps
10634 NodeManager
[hdpusr01@hadoop3 ~]$ jps
3760 NodeManager
注：从上面jps命令输出可以看到hadoop1主机上的ResourceManager进程，hadoop2~3主机上的NodeManager进程都启动了，hadoop2主机上的ResourceManager进程需要手动启动
4.2.11 启动ResourceManager——在主机hadoop2操作
[hdpusr01@hadoop2 ~]$ hadoop/sbin/yarn-daemon.sh start resourcemanager
starting resourcemanager, logging to /home/hdpusr01/hadoop/logs/yarn-hdpusr01-resourcemanager-hadoop2.out
[hdpusr01@hadoop2 ~]$ jps
10860 ResourceManager
注：至此hadoop集群部署完毕
4.3 查看集群状态——通过web页面查看
4.3.1 查看hadoop1上的NameNode
http://10.1.245.72:59070
4.3.2 查看hadoop2上的NameNode和DataNode
http://10.1.245.73:59070
注：从hadoop提供的web页面可以看到NameNode和DataNode的进程信息，一切正常,至此Hadoop集群搭建完毕。
来自 “ ITPUB博客 ” ，链接：https://blog.itpub.net/20801486/viewspace-1877048/ ，如需转载，请注明出处，否则将追究法律责任。
请登录后发表评论
登录
全部评论
