官网上下载的稳定版本

hadoop-1.0.4.tar.gz

解压到D盘

把hadoop解压到了d盘

修改修改hadoop-env.sh

 export JAVA_HOME=/cygdrive/d/java/jdk1.7.0_09



(2) 修改core-site.xml：（指定namenode）

[html] view plaincopy

<configuration>     

<property>  

  <name>fs.default.namename>  

  <value>hdfs://localhost:9000value>  

property>  

configuration>



(3)修改hdfs-site.xml（指定副本为1）

[html] view plaincopy

<configuration>     

<property>  

  <name>dfs.replicationname>  

  <value>1value>  

property>  

configuration>



(4) 修改mapred-site.xml （指定jobtracker）

[html] view plaincopy

<configuration>     

<property>  

  <name>mapred.job.trackername>  

  <value>localhost:9001value>  

property>  

configuration>

icecutie@ice /cygdrive/d/hadoop-1.0.4

$ bin/hadoop version

Hadoop 1.0.4

Subversion https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.0 -r 1393290

Compiled by hortonfo on Wed Oct  3 05:13:58 UTC 2012

From source with checksum fe2baea87c4c81a2c505767f3f9b71f4

bin/hadoop namenode –format

bin/start-all.sh

jps

namenode启动了

其他没启动

jobtracker日志

Error starting tracker: java.io.IOException: Failed to set permissions of path: D:hadoop-1.0.4logshistory to 0755

datanode的日志

ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: All directories in dfs.data.dir are invalid.

ERROR org.apache.hadoop.mapred.TaskTracker: Can not start task tracker because java.io.IOException: Failed to set permissions of path: mphadoop-icecutiemapredlocal askTracker to 0755

这么复杂

干脆改一个安装路径算了

改到/home路径下面

core-site.xml

fs.default.name

hdfs://localhost:9000

hadoop.tmp.dir

/home/hadoop/hadoop-1.0.4/temp

hdfs-site.xml

dfs.name.dir

/home/hadoop/hadoop-1.0.4/hdfs/name

dfs.data.dir

/home/hadoop/hadoop-1.0.4/hdfs/data

dfs.replication

1

mapred-site.xml

mapred.job.tracker

localhost:9001

mapred.local.dir

/home/hadoop/hadoop-1.0.4/mapred/local

mapred.system.dir

/home/hadoop/hadoop-1.0.4/mapred/system

还是只启动了namenode

看网址localhost:50070

2013-05-15 22:04:40,963 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

2013-05-15 22:04:40,963 ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:icecutie cause:java.io.IOException: Failed to set permissions of path: D:cygwinhomehadoop-1.0.4logshistory to 0755

2013-05-15 22:04:40,973 WARN org.apache.hadoop.mapred.JobTracker: Error starting tracker: java.io.IOException: Failed to set permissions of path: D:cygwinhomehadoop-1.0.4logshistory to 0755

2013-05-15 22:04:41,998 FATAL org.apache.hadoop.mapred.JobTracker: java.net.BindException: Problem binding to localhost/127.0.0.1:9001 : Address already in use: bind

2013-05-15 22:04:44,838 WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

2013-05-15 22:04:44,838 ERROR org.apache.hadoop.mapred.TaskTracker: Can not start task tracker because java.io.IOException: Failed to set permissions of path: homehadoophadoop-1.0.4mapredlocal askTracker to 0755

修改配置文件中的路径为cygwin/home/hadoop-1.0.4

就是物理机上的地址

重新format 然后start-all

报错

NameNode: java.net.BindException: Problem binding to localhost/127.0.0.1:9000 : Address already in use: bind

端口已经被占用了点解？

root@ice /home/hadoop-1.0.4

$ netstat -ano|grep 9000

TCP 127.0.0.1:8015 127.0.0.1:9000 ESTABLISHED 36488

TCP 127.0.0.1:9000 0.0.0.0:0 LISTENING 20192

TCP 127.0.0.1:9000 127.0.0.1:8015 ESTABLISHED 20192

root@ice /home/hadoop-1.0.4

$ kill 20192

-bash: kill: (20192) - No such process

手动关掉再来
datanode报错

ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: java.io.IOException: Incompatible namespaceIDs

123838122; datanode namespaceID = 441398030

尝试

edit the value of namespaceID in /current/VERSION to match the value of the current namenode

重新启动

dadatanode启动了

原因:每次namenode format会重新创建一个namenodeId,而tmp/dfs/data下包含了上次format下的id,namenode format清空了namenode下的数据,但是没有晴空datanode下的数据,导致启动时失败,所要做的就是每次fotmat前,清空tmp一下的所有目录.

但但是jobtracker没启动

ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:icecutie cause:java.io.IOException: Failed to set permissions of path: D:cygwinhomehadoop-1.0.4logshistorydone to 0755

ITPub博客

win8下安堓hadoop