ITPub博客

首页 > 大数据 > Hadoop > 【转】RedHat6.3搭建Hadoop2.2.0单机版

【转】RedHat6.3搭建Hadoop2.2.0单机版

Hadoop 作者:ashana 时间:2014-03-06 10:23:06 0 删除 编辑

1:下载Hadoop安装包

          wget http://mirror.esocc.com/apache/hadoop/common/hadoop-2.2.0/hadoop-2.2.0.tar.gz

2:下载Jdk

          wget https://edelivery.oracle.com/otn-pub/java/jdk/7u45-b18/jdk-7u45-linux-i586.tar.gz

3redhat6.3配置JDK

          步骤一:tar -zxvf jdk-7u45-linux-i586.tar.gz

          步骤二:mv jdk-xxxs jdk1.7

          步骤三:配置环境变量

                    vim /etc/profile

                    添加内容如下:

                    # set java environment

                    export JAVA_HOME=/opt/software/jdk1.7

                    export JRE_HOME=$JAVA_HOME/jre

                    export CLASSPATH=.:$JAVA_HOME/lib:$JAVA_HOME/jre/lib

                    export PATH=$JAVA_HOME/bin:$JAVA_HOME/jre/bin:$PATH

          步骤四:让profile文件生效

                    source /etc/profile

          步骤五:配置默认JDK

                    update-alternatives --install /usr/bin/java java /opt/software/jdk1.7/bin/java 300

                    update-alternatives --install /usr/bin/javac javac /opt/software/jdk1.7/bin/javac 300

                    update-alternatives --install /usr/bin/jar jar /opt/software/jdk1.7/bin/jar 300

                    update-alternatives --install /usr/bin/javah javah /opt/software/jdk1.7/bin/javah 300

                    update-alternatives --install /usr/bin/javap javap /opt/software/jdk1.7/bin/javap 300

          步骤六:验证

                    java -version

                               java version "1.7.0_45"

                               Java(TM) SE Runtime Environment (build 1.7.0_45-b18)

                               Java HotSpot(TM) Server VM (build 24.45-b08, mixed mode)

4:解压hadoop-2.2.0安装包

          tar -zxvf hadoop-2.2.0.tar.gz

          mv hadoopxxxxx hadoop2.0

5:修改hadoop配置文件,配置文件所在位置/opt/software/hadoop2.0/etc/hadoop

          步骤一:修改hadoop-env.sh,添加java安装目录

                    export JAVA_HOME=/opt/software/jdk1.7

          步骤二:修改mapred-site.xml,修改此文件之前记得要重命名。添加内容如下:

                   

                     

                                    mapreduce.framework.name</name>

                                    yarnvalue>

                            </property>

                    configuration>

                    hadoop1.0比较,用户无须配置mapred.job.tarcker。因为在2.0中统一由资源调度框架yarn管理。所以只需要指定运行时框架yarn即可。

          步骤三:修改core-site.xml 。添加内容如下:

                     

                              fs.default.name</name>

                            hdfs://hadoop:8020</value>

                    property>

                   

                               hadoop.tmp.dir</name>

                            /opt/software/hadoop2.0/tmp</value>

                    property>

                    这里简单的配置下hdfs的工作模式,并没配置hdfs federation等特性。

          步骤四:修改yarn-site.xml。添加内容如下:

                   

                          yarn.nodemanager.aux-services</name>

                          mapreduce_shufflevalue>

                    </property>

                    为了能够运行MapReduce工作,需要让各个nodemanager在启动时加载shuffle server

          步骤五:修改hdfs-site.xml。添加内容如下:

                   

                    dfs.replicationname>

                    1</value>

                            property>

           

                    dfs.namenode.name.dir</name>

                    /opt/software/hadoop2.0/data/dfs/name</value>

            property>

           

                    dfs.datanode.data.dir</name>

                    /opt/software/hadoop2.0/data/dfs/data</value>

            property>

                    配置下文件块的备份策略(默认是3),以及hdfs格式换之后的目录。

6:初始化Hdfs,命令在/opt/software/hadoop2.0/bin目录下

          ./hadoop namenode -format

7:启动hadoop2.0,命令在/opt/software/hadoop2.0/sbin

          ./start-all.sh

8:启动后的jps进程

          8084 ResourceManager

          7944 SecondaryNameNode

          7776 DataNode

          7683 NameNode

 

 

问题

1nodemanager无法启动。

          报错信息如下:The ServiceName: mapreduce-shuffle set in yarn.nodemanager.aux-services is invalid.The valid service name should only contain a-zA-Z0-9_ and can not start with numbers

          解决方案:

          修改yarn-site.xml文件:

           

            yarn.nodemanager.aux-services</name> 

            mapreduce-shufflevalue> 

          </property>

          改成

           

            yarn.nodemanager.aux-servicesname> 

            mapreduce_shuffle</value> 

          property>

 

          另外关于maprduce.shuffle,也有一个解决方案:The correct value is now mapreduce_shuffle instead of mapreduce.shuffle

                                        

 

<!-- 正文结束 -->

来自 “ ITPUB博客 ” ,链接:http://blog.itpub.net/22869292/viewspace-1114281/,如需转载,请注明出处,否则将追究法律责任。

下一篇: 没有了~
请登录后发表评论 登录
全部评论

注册时间:2009-11-13

最新文章