一、集群环境：
HostnameIPHadoop版本Hadoop功能系统
node1
192.168.1.1510.20.0namenodehive+sqooprhel5.4X86
node2
192.168.1.1520.20.0datanodemysqlrhel5.4X86
node3
192.168.1.1530.20.0datanode rhel5.4X86
二、安装sqoop
1、下载sqoop压缩包，并解压
压缩包分别是：sqoop-1.2.0-CDH3B4.tar.gz，hadoop-0.20.2-CDH3B4.tar.gz， Mysql JDBC驱动包mysql-connector-java-5.1.10-bin.jar
[root@node1 ~]# ll
drwxr-xr-x15root root4096Feb222011hadoop-0.20.2-CDH3B4-rw-r--r-- 1 root root 724225 Sep 15 06:46 mysql-connector-java-5.1.10-bin.jardrwxr-xr-x11root root4096Feb222011sqoop-1.2.0-CDH3B4
2、将sqoop-1.2.0-CDH3B4拷贝到/home/hadoop目录下，并将Mysql JDBC驱动包和hadoop-0.20.2-CDH3B4下的hadoop-core-0.20.2-CDH3B4.jar至sqoop-1.2.0-CDH3B4/lib下，最后修改一下属主。
[root@node1 ~]# cp mysql-connector-java-5.1.10-bin.jar sqoop-1.2.0-CDH3B4/lib[root@node1 ~]# cp hadoop-0.20.2-CDH3B4/hadoop-core-0.20.2-CDH3B4.jar sqoop-1.2.0-CDH3B4/lib[root@node1 ~]# chown-R hadoop:hadoop sqoop-1.2.0-CDH3B4[root@node1 ~]# mv sqoop-1.2.0-CDH3B4/home/hadoop[root@node1 ~]# ll/home/hadoop
total 35748-rw-rw-r-- 1 hadoop hadoop 343 Sep 15 05:13 derby.logdrwxr-xr-x13hadoop hadoop4096Sep1416:16hadoop-0.20.2drwxr-xr-x9hadoop hadoop4096Sep1420:21hive-0.10.0-rw-r--r-- 1 hadoop hadoop 36524032 Sep 14 20:20 hive-0.10.0.tar.gzdrwxr-xr-x8hadoop hadoop4096Sep252012jdk1.7drwxr-xr-x12hadoop hadoop4096Sep1500:25mahout-distribution-0.7drwxrwxr-x5hadoop hadoop4096Sep1505:13 metastore_db-rw-rw-r-- 1 hadoop hadoop 406 Sep 14 16:02 scp.shdrwxr-xr-x11hadoop hadoop4096Feb222011sqoop-1.2.0-CDH3B4
drwxrwxr-x3hadoop hadoop4096Sep1416:17tempdrwxrwxr-x3hadoop hadoop4096Sep1415:59user
3、配置configure-sqoop，注释掉对于HBase和ZooKeeper的检查
[root@node1 bin]# pwd/home/hadoop/sqoop-1.2.0-CDH3B4/bin[root@node1 bin]# vi configure-sqoop
#!/bin/bash
#
# Licensed toCloudera, Inc. under oneor more
# contributor license agreements. See the NOTICE filedistributedwith# this workfor additional information regarding copyright ownership.
.
.
.
# Check:Ifwe can't find our dependencies, give up here.
if [ ! -d "${HADOOP_HOME}" ]; then
echo "Error: $HADOOP_HOME does not exist!"
echo 'Pleaseset$HADOOP_HOMEtothe rootofyour Hadoop installation.' exit 1
fi
#if [ ! -d "${HBASE_HOME}" ]; then
# echo "Error: $HBASE_HOME does not exist!"
# echo 'Pleaseset$HBASE_HOMEtothe rootofyour HBase installation.'# exit 1
#fi
#if [ ! -d "${ZOOKEEPER_HOME}" ]; then
# echo "Error: $ZOOKEEPER_HOME does not exist!"
# echo 'Pleaseset$ZOOKEEPER_HOMEtothe rootofyour ZooKeeper installation.'# exit 1
#fi
4、修改/etc/profile和.bash_profile文件，添加Hadoop_Home,调整PATH
[hadoop@node1 ~]$ vi .bash_profile
# .bash_profile
# Get the aliases and functionsif[ -f ~/.bashrc ];then . ~/.bashrc
fi
# Userspecific environmentand startup programs
HADOOP_HOME=/home/hadoop/hadoop-0.20.2PATH=$HADOOP_HOME/bin:$PATH:$HOME/bin
export HIVE_HOME=/home/hadoop/hive-0.10.0export MAHOUT_HOME=/home/hadoop/mahout-distribution-0.7export PATH HADOOP_HOME
三、测试Sqoop
1、查看mysql中的数据库：
[hadoop@node1 bin]$ ./sqoop list-databases--connect jdbc:mysql://192.168.1.152:3306/ --username sqoop --password sqoop13/09/1507:17:16WARN tool.BaseSqoopTool: Setting your passwordonthe command-lineisinsecure. Consider using-P instead.13/09/1507:17:17 INFO manager.MySQLManager: Executing SQL statement: SHOW DATABASES
information_schema
mysql
performance_schema
sqoop
test
2、将mysql的表导入到hive中：
[hadoop@node1 bin]$ ./sqoop import--connect jdbc:mysql://192.168.1.152:3306/sqoop --username sqoop --password sqoop --table test --hive-import -m 113/09/1508:15:01WARN tool.BaseSqoopTool: Setting your passwordonthe command-lineisinsecure. Consider using-P instead.13/09/1508:15:01INFO tool.BaseSqoopTool: Using Hive-specific delimitersfor output. You can override13/09/1508:15:01INFO tool.BaseSqoopTool: delimiterswith--fields-terminated-by, etc.13/09/1508:15:01 INFO tool.CodeGenTool: Beginning code generation13/09/1508:15:01INFO manager.MySQLManager: Executing SQL statement:SELECTt.*FROM`test`ASt LIMIT113/09/1508:15:02INFO manager.MySQLManager: Executing SQL statement:SELECTt.*FROM`test`ASt LIMIT113/09/1508:15:02INFO orm.CompilationManager: HADOOP_HOMEis/home/hadoop/hadoop-0.20.2/bin/..13/09/1508:15:02INFO orm.CompilationManager: Found hadoop core jar at:/home/hadoop/hadoop-0.20.2/bin/../hadoop-0.20.2-core.jar13/09/1508:15:03INFO orm.CompilationManager: Writing jarfile:/tmp/sqoop-hadoop/compile/a71936fd2bb45ea6757df22751a320e3/test.jar13/09/1508:15:03WARN manager.MySQLManager: It lookslikeyou are importingfrom mysql.13/09/1508:15:03WARN manager.MySQLManager: This transfer can be faster!Usethe--direct13/09/1508:15:03WARN manager.MySQLManager:optiontoexercise a MySQL-specific fast path.13/09/1508:15:03INFO manager.MySQLManager: Setting zeroDATETIMEbehaviorto convertToNull (mysql)13/09/1508:15:03INFO mapreduce.ImportJobBase: Beginning importof test13/09/1508:15:04INFO manager.MySQLManager: Executing SQL statement:SELECTt.*FROM`test`ASt LIMIT113/09/1508:15:05 INFO mapred.JobClient: Running job: job_201309150505_000913/09/1508:15:06INFO mapred.JobClient: map0%reduce0%13/09/1508:15:34INFO mapred.JobClient: map100%reduce0%13/09/1508:15:36 INFO mapred.JobClient: Job complete: job_201309150505_000913/09/1508:15:36INFO mapred.JobClient: Counters:513/09/1508:15:36 INFO mapred.JobClient: Job Counters 13/09/1508:15:36INFO mapred.JobClient: Launched map tasks=113/09/1508:15:36 INFO mapred.JobClient: FileSystemCounters13/09/1508:15:36INFO mapred.JobClient: HDFS_BYTES_WRITTEN=58332313/09/1508:15:36INFO mapred.JobClient: Map-Reduce Framework13/09/1508:15:36INFO mapred.JobClient: Map input records=6553613/09/1508:15:36INFO mapred.JobClient: Spilled Records=013/09/1508:15:36INFO mapred.JobClient: Map output records=6553613/09/1508:15:36INFO mapreduce.ImportJobBase: Transferred569.6514KBin32.0312seconds (17.7842KB/sec)13/09/1508:15:36INFO mapreduce.ImportJobBase: Retrieved65536 records.13/09/1508:15:36INFO hive.HiveImport: Removingtemporaryfilesfromimport process: test/_logs13/09/1508:15:36INFO hive.HiveImport: Loading uploaded datainto Hive13/09/1508:15:36INFO manager.MySQLManager: Executing SQL statement:SELECTt.*FROM`test`ASt LIMIT113/09/1508:15:36INFO manager.MySQLManager: Executing SQL statement:SELECTt.*FROM`test`ASt LIMIT113/09/1508:15:41INFO hive.HiveImport: Logging initialized using configurationinjar:file:/home/hadoop/hive-0.10.0/lib/hive-common-0.10.0.jar!/hive-log4j.properties13/09/1508:15:41INFO hive.HiveImport: Hive historyfile=/tmp/hadoop/hive_job_log_hadoop_201309150815_1877092059.txt13/09/1508:16:10 INFO hive.HiveImport: OK13/09/1508:16:10INFO hive.HiveImport: Time taken:28.791 seconds13/09/1508:16:11INFO hive.HiveImport: Loading datatotabledefault.test13/09/1508:16:12INFO hive.HiveImport:Tabledefault.test stats:[num_partitions: 0, num_files: 1, num_rows: 0, total_size: 583323, raw_data_size: 0]13/09/1508:16:12 INFO hive.HiveImport: OK13/09/1508:16:12INFO hive.HiveImport: Time taken:1.704 seconds13/09/1508:16:12INFO hive.HiveImport: Hive import complete.
来自 “ ITPUB博客 ” ，链接：https://blog.itpub.net/2730/viewspace-3122603/ ，如需转载，请注明出处，否则将追究法律责任。
