DSE 4.8.8升级抛出spark cassandra连接器配置变量错误

时间:2016-06-23 21:37:10

标签: cassandra-2.0 spark-cassandra-connector

我最近将我的DSE版本从4.8.4升级到4.8.8,我看到了以下意外结果 我正在使用:spark-cassandra-connector-java_2.10:1.4.2
spark-streaming_2.10:1.4.1
spark-core_2.10 1.4.1

WARN  2016-06-23 21:28:50,132 org.apache.spark.scheduler.TaskSetManager: Lost task 3.0 in stage 3365.0 (TID 6731, 172.31.17.116): java.lang.ExceptionInInitializerError
    at com.dynosense.dynospark.SparkApp$2.call(SparkApp.java:107)
    at com.dynosense.dynospark.SparkApp$2.call(SparkApp.java:103)
    at org.apache.spark.api.java.JavaPairRDD$$anonfun$toScalaFunction$1.apply(JavaPairRDD.scala:1027)
    at scala.collection.Iterator$$anon$11.next(Iterator.scala:328)
    at org.apache.spark.util.Utils$.getIteratorSize(Utils.scala:1508)
    at org.apache.spark.rdd.RDD$$anonfun$count$1.apply(RDD.scala:1099)
    at org.apache.spark.rdd.RDD$$anonfun$count$1.apply(RDD.scala:1099)
    at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1792)
    at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1792)
    at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:63)
    at org.apache.spark.scheduler.Task.run(Task.scala:70)
    at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:213)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)
Caused by: com.datastax.spark.connector.util.ConfigCheck$ConnectorConfigurationException: Invalid Config Variables
Only known spark.cassandra.* variables are allowed when using the Spark Cassandra Connector.
spark.cassandra.connection.host.autoUpdate is not a valid Spark Cassandra Connector variable.
No likely matches found.
    at com.datastax.spark.connector.util.ConfigCheck$.checkConfig(ConfigCheck.scala:46)
    at com.datastax.spark.connector.cql.CassandraConnectorConf$.apply(CassandraConnectorConf.scala:171)
    at com.datastax.spark.connector.cql.CassandraConnector$.apply(CassandraConnector.scala:192)
    at com.datastax.spark.connector.cql.CassandraConnector.apply(CassandraConnector.scala)
    at com.dynosense.dynospark.SparkApp.<clinit>(SparkApp.java:62)
    ... 15 more

0 个答案:

没有答案