用火花连接Cassandra节点

时间:2017-11-29 16:53:22

标签: java apache-spark cassandra

我有一个节点,我正在运行我的火花作业,它与我尝试连接的cassandra节点处于相同的环境中。 使用ssl auth,我为SparkConf设置了以下属性

SparkConf sparkConf = new SparkConf()
    .setAppName("AppName")
    .set("spark.cassandra.connection.ssl.trusStore.path","/path/trustStore")
    .set("spark.cassandra.connection.ssl.trustStore.password","password")
    .set("spark.cassandra.connection.ssl.keyStore.path","/path/keyStore")
    .set("spark.cassandra.connection.ssl.keyStore.password","password")
    .set("spark.cassandra.connection.host",args[0])
    .set("spark.executor.JAVA_HOME","/path/jdk64/jdk1.8.0_112")
    .set("spark.cassandra.auth.username","username")
    .set("spark.cassandra.auth.password","password")
    .set("spark.cassandra.connection.timeout_ms","50000")
    .set("spark.cassandra.connection.ssl.enabled","true")
    .set("spark.cassandra.sql.cluster","Cluster")
    .set("spark.cassandra.connection.local_dc","local_dc");

当我尝试使用openSession()方法初始化连接时,它会抛出异常

JavaSparkContext jssc = new JavaSparkContext(sparkConf);
CassandraConnector connector = CassandraConnector.apply(jssc.getConf());
Session session = connector.openSession();

抛出的异常如下,还包括当我得到异常的原因并打印错误消息时会发生什么

java.io.IOException: Failed to open native connection to Cassandra at {HOST}:PORT
[NODE_NAME/HOST:PORT] Channel has been closed

我已经验证了所有凭据都是正确的,从用户名/密码到keyStore trustStore文件&密码。在cassandra节点方面,它甚至没有在日志中显示尝试连接。

我已经检查过,这不是防火墙问题

编辑:这是我的spark-submit代码

spark-submit --master local[*] --class 
com.sparkCassandra.Main file:///path/SparkCass.jar XXX.X.X.XXX

0 个答案:

没有答案