使用纱线群集模式从Spark在EMR上运行Phoenix时出错

时间:2018-05-08 13:09:51

标签: apache-spark phoenix

我正在尝试使用spark-submit运行一个在本地模式下正常运行的简单Phoenix程序。但是要在集群模式下运行以下异常。

  

l.cj.core.exceptions.WrongArgumentException:格式错误的数据库URL,无法解析主URL部分。   java.lang.RuntimeException:java.sql.SQLNonTransientConnectionException:由于基础异常,无法加载连接类:com.mysql.cj.core.exceptions.WrongArgumentException:格式错误的数据库URL,无法解析主URL部分。       在org.apache.phoenix.mapreduce.PhoenixInputFormat.getQueryPlan(PhoenixInputFormat.java:201)       在org.apache.phoenix.mapreduce.PhoenixInputFormat.getSplits(PhoenixInputFormat.java:87)       在org.apache.spark.rdd.NewHadoopRDD.getPartitions(NewHadoopRDD.scala:125)       在org.apache.spark.rdd.RDD $$ anonfun $ partitions $ 2.apply(RDD.scala:252)       在org.apache.spark.rdd.RDD $$ anonfun $ partitions $ 2.apply(RDD.scala:250)       在scala.Option.getOrElse(Option.scala:121)       在org.apache.spark.rdd.RDD.partitions(RDD.scala:250)       在org.apache.spark.rdd.MapPartitionsRDD.getPartitions(MapPartitionsRDD.scala:35)       在org.apache.spark.rdd.RDD $$ anonfun $ partitions $ 2.apply(RDD.scala:252)       在org.apache.spark.rdd.RDD $$ anonfun $ partitions $ 2.apply(RDD.scala:250)       在scala.Option.getOrElse(Option.scala:121)       在org.apache.spark.rdd.RDD.partitions(RDD.scala:250)       在org.elasticsearch.spark.rdd.EsSpark $ .doSaveToEs(EsSpark.scala:94)       在org.elasticsearch.spark.rdd.EsSpark $ .saveToEs(EsSpark.scala:79)       在org.elasticsearch.spark.rdd.EsSpark $ .saveToEs(EsSpark.scala:76)       在org.elasticsearch.spark.rdd.EsSpark $ .saveJsonToEs(EsSpark.scala:111)       在org.elasticsearch.spark.rdd.api.java.JavaEsSpark $ .saveJsonToEs(JavaEsSpark.scala:63)       在org.elasticsearch.spark.rdd.api.java.JavaEsSpark.saveJsonToEs(JavaEsSpark.scala)       在com.voicebase.etl.PhoenixToElasticSearch.main(PhoenixToElasticSearch.java:99)       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)       at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)       at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)       at java.lang.reflect.Method.invoke(Method.java:498)       在org.apache.spark.deploy.SparkSubmit $ .org $ apache $ spark $ deploy $ SparkSubmit $$ runMain(SparkSubmit.scala:775)       在org.apache.spark.deploy.SparkSubmit $ .doRunMain $ 1(SparkSubmit.scala:180)       在org.apache.spark.deploy.SparkSubmit $ .submit(SparkSubmit.scala:205)       在org.apache.spark.deploy.SparkSubmit $ .main(SparkSubmit.scala:119)       在org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)   引发者:java.sql.SQLNonTransientConnectionException:由于基础异常,无法加载连接类:com.mysql.cj.core.exceptions.WrongArgumentException:格式错误的数据库URL,无法解析主URL部分。       在com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:526)       在com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:513)       在com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:505)       在com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:479)       在com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:489)       at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:72)       at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:124)       在com.mysql.cj.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:224)       在java.sql.DriverManager.getConnection(DriverManager.java:664)       在java.sql.DriverManager.getConnection(DriverManager.java:208)       在org.apache.phoenix.mapreduce.util.ConnectionUtil.getConnection(ConnectionUtil.java:98)       at org.apache.phoenix.mapreduce.util.ConnectionUtil.getInputConnection(ConnectionUtil.java:57)       在org.apache.phoenix.mapreduce.PhoenixInputFormat.getQueryPlan(PhoenixInputFormat.java:176)       ......还有27个   由以下原因引起:com.mysql.cj.core.exceptions.UnableToConnectException:由于基础异常,无法加载连接类:com.mysql.cj.core.exceptions.WrongArgumentException:格式错误的数据库URL,无法解析主URL部分。       at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)       at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)       at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)       at java.lang.reflect.Constructor.newInstance(Constructor.java:423)       at com.mysql.cj.core.exceptions.ExceptionFactory.createException(ExceptionFactory.java:54)       at com.mysql.cj.core.exceptions.ExceptionFactory.createException(ExceptionFactory.java:93)       ......还有33个   原因:com.mysql.cj.core.exceptions.WrongArgumentException:格式错误的数据库URL,无法解析主URL部分。       at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)       at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)       at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)       at java.lang.reflect.Constructor.newInstance(Constructor.java:423)       at com.mysql.cj.core.exceptions.ExceptionFactory.createException(ExceptionFactory.java:54)       在com.mysql.cj.core.conf.url.ConnectionUrlParser.parseConnectionString(ConnectionUrlParser.java:134)       在com.mysql.cj.core.conf.url.ConnectionUrlParser。(ConnectionUrlParser.java:124)       在com.mysql.cj.core.conf.url.ConnectionUrlParser.parseConnectionString(ConnectionUrlParser.java:113)       在com.mysql.cj.core.conf.url.ConnectionUrl.getConnectionUrlInstance(ConnectionUrl.java:192)       在com.mysql.cj.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:194)       ......还有32个   18/05/08 13:05:03错误mapreduce.PhoenixInputFormat:无法获取错误的查询计划[由于基础异常而无法加载连接类:com.mysql.cj.core.exceptions.WrongArgumentException:格式错误的数据库URL,失败解析主要URL部分。]   线程" main"中的例外情况java.lang.RuntimeException:java.sql.SQLNonTransientConnectionException:由于基础异常,无法加载连接类:com.mysql.cj.core.exceptions.WrongArgumentException:格式错误的数据库URL,无法解析主URL部分。       在org.apache.phoenix.mapreduce.PhoenixInputFormat.getQueryPlan(PhoenixInputFormat.java:201)       在org.apache.phoenix.mapreduce.PhoenixInputFormat.getSplits(PhoenixInputFormat.java:87)       在org.apache.spark.rdd.NewHadoopRDD.getPartitions(NewHadoopRDD.scala:125)       在org.apache.spark.rdd.RDD $$ anonfun $ partitions $ 2.apply(RDD.scala:252)       在org.apache.spark.rdd.RDD $$ anonfun $ partitions $ 2.apply(RDD.scala:250)       在scala.Option.getOrElse(Option.scala:121)       在org.apache.spark.rdd.RDD.partitions(RDD.scala:250)       在org.apache.spark.rdd.MapPartitionsRDD.getPartitions(MapPartitionsRDD.scala:35)       在org.apache.spark.rdd.RDD $$ anonfun $ partitions $ 2.apply(RDD.scala:252)       在org.apache.spark.rdd.RDD $$ anonfun $ partitions $ 2.apply(RDD.scala:250)       在scala.Option.getOrElse(Option.scala:121)       在org.apache.spark.rdd.RDD.partitions(RDD.scala:250)       在org.apache.spark.SparkContext.runJob(SparkContext.scala:2094)       在org.apache.spark.rdd.RDD.count(RDD.scala:1158)       在org.apache.spark.api.java.JavaRDDLike $ class.count(JavaRDDLike.scala:455)       在org.apache.spark.api.java.AbstractJavaRDDLike.count(JavaRDDLike.scala:45)       在com.voicebase.etl.PhoenixToElasticSearch.main(PhoenixToElasticSearch.java:104)       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)       at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)       at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)       at java.lang.reflect.Method.invoke(Method.java:498)       在org.apache.spark.deploy.SparkSubmit $ .org $ apache $ spark $ deploy $ SparkSubmit $$ runMain(SparkSubmit.scala:775)       在org.apache.spark.deploy.SparkSubmit $ .doRunMain $ 1(SparkSubmit.scala:180)       在org.apache.spark.deploy.SparkSubmit $ .submit(SparkSubmit.scala:205)       在org.apache.spark.deploy.SparkSubmit $ .main(SparkSubmit.scala:119)       在org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)   引发者:java.sql.SQLNonTransientConnectionException:由于基础异常,无法加载连接类:com.mysql.cj.core.exceptions.WrongArgumentException:格式错误的数据库URL,无法解析主URL部分。       在com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:526)       在com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:513)       在com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:505)       在com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:479)       在com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:489)       at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:72)       at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:124)       在com.mysql.cj.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:224)       在java.sql.DriverManager.getConnection(DriverManager.java:664)       在java.sql.DriverManager.getConnection(DriverManager.java:208)       在org.apache.phoenix.mapreduce.util.ConnectionUtil.getConnection(ConnectionUtil.java:98)       at org.apache.phoenix.mapreduce.util.ConnectionUtil.getInputConnection(ConnectionUtil.java:57)       在org.apache.phoenix.mapreduce.PhoenixInputFormat.getQueryPlan(PhoenixInputFormat.java:176)       ......还有25个   由以下原因引起:com.mysql.cj.core.exceptions.UnableToConnectException:由于基础异常,无法加载连接类:com.mysql.cj.core.exceptions.WrongArgumentException:格式错误的数据库URL,无法解析主URL部分。       at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)       at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)       at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)       at java.lang.reflect.Constructor.newInstance(Constructor.java:423)       at com.mysql.cj.core.exceptions.ExceptionFactory.createException(ExceptionFactory.java:54)       at com.mysql.cj.core.exceptions.ExceptionFactory.createException(ExceptionFactory.java:93)       ......还有31个   原因:com.mysql.cj.core.exceptions.WrongArgumentException:格式错误的数据库URL,无法解析主URL部分。       at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)       at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)       at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)       at java.lang.reflect.Constructor.newInstance(Constructor.java:423)       at com.mysql.cj.core.exceptions.ExceptionFactory.createException(ExceptionFactory.java:54)       在com.mysql.cj.core.conf.url.ConnectionUrlParser.parseConnectionString(ConnectionUrlParser.java:134)       在com.mysql.cj.core.conf.url.ConnectionUrlParser。(ConnectionUrlParser.java:124)       在com.mysql.cj.core.conf.url.ConnectionUrlParser.parseConnectionString(ConnectionUrlParser.java:113)       在com.mysql.cj.core.conf.url.ConnectionUrl.getConnectionUrlInstance(ConnectionUrl.java:192)       在com.mysql.cj.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:194)       ......还有30多个   18/05/08 13:05:03 INFO spark.SparkContext:从关机钩子调用stop()   18/05/08 13:05:03 INFO server.AbstractConnector:停止Spark @ 1a28be00 {HTTP / 1.1,[http / 1.1]} {0.0.0.0:4040}   18/05/08 13:05:03 INFO ui.SparkUI:停在http://10.16.129.152:4040的Spark Web UI   18/05/08 13:05:03 INFO cluster.YarnClientSchedulerBackend:中断监控线程   18/05/08 13:05:03 INFO cluster.YarnClientSchedulerBackend:关闭所有执行者

0 个答案:

没有答案