hive 数据通过sqoop 将增量数据导入mysql
查看hive表结构
-
hive> desc hbase_visit;
-
OK
-
key string from deserializer
-
insert_time string from deserializer
-
label string from deserializer
-
sessionid string from deserializer
-
stay_time int from deserializer
-
visit_time bigint from deserializer
-
leave_time bigint from deserializer
-
page_url string from deserializer
-
referrer_url string from deserializer
-
visit_number int from deserializer
-
site string from deserializer
- ip string from deserializer
hive新建内部表
“\001” 为分隔符-
create table hbase_visit_cp(
-
key_table varchar(100),
-
insert_time varchar(20),
-
label varchar(50),
-
sessionid varchar(50),
-
stay_time int,
-
visit_time bigint,
-
leave_time bigint,
-
page_url varchar(2000),
-
referrer_url varchar(2000),
-
visit_number int,
-
site varchar(20),
-
ip varchar(20)
- ) row format delimited fields terminated by '\001'
mysql建表
-
create table hbase_visit(
-
key_table varchar(100),
-
insert_time varchar(20),
-
label varchar(50),
-
sessionid varchar(50),
-
stay_time int,
-
visit_time bigint,
-
leave_time bigint,
-
page_url varchar(2000),
-
referrer_url varchar(2000),
-
visit_number int,
-
site varchar(20),
-
ip varchar(20)
- ) ENGINE=InnoDB DEFAULT CHARSET=utf8;
hive 将增量数据导入hbase_visit_cp
- insert overwrite table hbase_visit_cp SELECT * from hbase_visit where insert_time>=20150818000000000 and insert_time<20150819000000000
sqoop将数据导入mysql
- /usr/bin/sqoop export --connect jdbc:mysql://10.10.10.10:3306/test --username ** --password ******** --table hbase_visit --export-dir /user/hive/warehouse/hbase_visit_cp --input-fields-terminated-by '\001' --input-null-string '\\N' --input-null-non-string '\\N'
来自 “ ITPUB博客 ” ，链接：https://blog.itpub.net/30150152/viewspace-1778764/ ，如需转载，请注明出处，否则将追究法律责任。
上一篇：
python连接mysql语句
下一篇：
mysql常见错误
请登录后发表评论
登录
全部评论
最新文章
- ORA-01111,ORA-01110,ORA-01157,ORA-01111,ORA-01110
- tar:implausibly old time stamp 1970-01-01 08:00:00
- ORA-27090: Unable to reserve kernel resources for asynchronous disk I/O
- ORA-16139: media recovery required
- SSD 下的 MySQL IO 优化
- dataguard的switchover和failover
- ORACLE ddl触发器创建同义词
- pt-table-checksum使用实践
- MySQL主库添加从库
- 修改innodb_file_per_table
