我在Scala中使用Cassandra Spark连接器。 这是我的示例代码,用于将数据保存到Cassandra。
val data = rdd.map
(f => new CassandraRow(IndexedSeq("pk", "count"), IndexedSeq(f._1, f._2.toString())))
data.saveToCassandra("keyspace", "table")
然后,我尝试使用TTL。这是我添加TTL的示例代码。
添加导入
import com.datastax.spark.connector.writer.{TTLOption, WriteConf}
并添加TTL以保存ToCassandra
data.saveToCassandra
("keyspace", "table", writeConf = WriteConf(ttl = TTLOption.constant(604800))
但是,当我编译它时,会出现错误。
错误消息
[error] bad symbolic reference. A signature in TTLOption.class refers to term streaming
[error] in package org.apache.spark which is not available.
[error] It may be completely missing from the current classpath, or the version on
[error] the classpath might be incompatible with the version used when compiling TTLOption.class.
[error] one error found
[error] (compile:compile) Compilation failed
[error] Total time: 4 s, completed Mar 31, 2015 11:55:14 AM
build.sbt
name := "Cassandra"
version := "1.0"
scalaVersion := "2.10.4"
libraryDependencies += "org.apache.spark" % "spark-core_2.10" % "1.2.1"
libraryDependencies += "org.apache.spark" % "spark-sql_2.10" % "1.2.1"
libraryDependencies += "com.datastax.spark" % "spark-cassandra-connector_2.10" % "1.2.0-rc2"
我的代码有问题吗?
答案 0 :(得分:1)
这是当前连接器中与已知依赖关系有关的已知错误 流式火花库。
https://datastax-oss.atlassian.net/browse/SPARKC-113
如果这是阻止的,您需要一个解决方法,您可以将spark-streaming库作为依赖项包含在构建文件中。
libraryDependencies += "org.apache.spark" % "spark-streaming_2.10" % "1.2.1"