通过Spark查询Cassandra表

Question

我试图通过spark-1.6.0和scala-2.11.7从Cassandra 2.0.17表中获取值，并执行以下步骤

1. 启动cassandra - service cassandra start
2. 启动spark-- sbin / start-all.sh
3. 陈述火花scala - bin / spark-shell --jars spark-cassandra-connector_2.10-1.5.0-M1.jar

在scala中执行这些命令

import org.apache.spark.SparkContext
import org.apache.spark.SparkConf
import org.apache.spark.SparkContext._

sc.stop

val conf = new SparkConf(true).set("spark.cassandra.connection.host","127.0.0.1")

val sc=new SparkContext("local[2]","test",conf)

import com.datastax.spark.connector._

一切正常，直到这里，但当我执行 -

val rdd=sc.cassandraTable("tutorialspoint","emp")

它给我以下错误

error: bad symbolic reference. A signature in CassandraTableScanRDD.class refers to term driver
in package com.datastax which is not available.
It may be completely missing from the current classpath, or the version on
the classpath might be incompatible with the version used when compiling CassandraTableScanRDD.class.
error: bad symbolic reference. A signature in CassandraTableScanRDD.class refers to term core
in value com.datastax.driver which is not available.
It may be completely missing from the current classpath, or the version on
the classpath might be incompatible with the version used when compiling CassandraTableScanRDD.class.
error: bad symbolic reference. A signature in CassandraTableScanRDD.class refers to term core
in value com.datastax.driver which is not available.
It may be completely missing from the current classpath, or the version on
the classpath might be incompatible with the version used when compiling CassandraTableScanRDD.class.

已经从cassandra lib添加了罐子来点燃并反思它。  我使用的是java版本1.8.0_72

我错过了什么？

Answer 1

您使用的驱动程序与您的scala和spark版本不兼容。您正在使用scala-2.11.7，但此驱动程序适用于scala 2.10。此外，对于spark，此驱动程序支持spark 1.5.x。