手动将jar包加在所有计算节点/opt/cloudera/parcels/CDH/lib/hadoop-yarn目录下
例：scp /opt/jar/mysql-connector-java-5.1.39.jar root@CDH23:/opt/cloudera/parcels/CDH-5.7.0-1.cdh5.7.0.p0.45/lib/hadoop-yarn/
spark-shell --driver-class-path /opt/jar/mysql-connector-java-5.1.39.jar
代码：
-
val aa =sqlContext.read.option("mergeSchema", "true").parquet("/monitor/parquet/a7/year=2016/month=05/day=11")
-
aa.registerTempTable("aa")
-
val url="jdbc:mysql://ip:port/dbName"
-
val prop = new java.util.Properties
-
prop.setProperty("user","myUserName")
-
prop.setProperty("password","myPassword")
-
import org.apache.spark.sql.SaveMode
-
- sqlContext.sql("select appId,count(appId) as number from aa group by appId").write.mode(SaveMode.Append).jdbc(url,"tableName",prop)
点击(此处)折叠或打开
-
prop.setProperty("useUnicode","true")
- prop.setProperty("characterEncoding","utf8")
来自 “ ITPUB博客 ” ，链接：https://blog.itpub.net/29754888/viewspace-2108092/ ，如需转载，请注明出处，否则将追究法律责任。
请登录后发表评论
登录
全部评论
