Question

我正在尝试为Cassandra Spark连接器配置SSL，但我找不到如何操作的示例。

我试图像这样配置它：

SparkConf conf = new SparkConf().setAppName("someApp")
        .set("spark.cassandra.connection.host", "111.111.111.111")
        .set("spark.cassandra.connection.ssl.enabled", "true")
        .set("spark.cassandra.connection.ssl.trustStore.path", "/some/tfile.jks")
        .set("spark.cassandra.connection.ssl.trustStore.password", "apassword")
        .set("spark.cassandra.connection.ssl.trustStore.type", "JKS")
        .set("spark.cassandra.connection.ssl.enabledAlgorithms", "TLS_RSA_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_256_CBC_SHA")
        .set("spark.cassandra.connection.ssl.keyStore.path", "/some/kfile.jks")
        .set("spark.cassandra.connection.ssl.keyStore.password", "anotherpassword")
        .set("spark.cassandra.connection.ssl.keyStore.type", "JKS")
        .set("spark.cassandra.connection.ssl.protocol", "TLS");

当我尝试提交spark作业时，我收到了以下错误：

Exception in thread "main" com.datastax.spark.connector.util.ConfigCheck$ConnectorConfigurationException: Invalid Config Variables
Only known spark.cassandra.* variables are allowed when using the Spark Cassandra Connector.
spark.cassandra.connection.ssl.keyStore.password is not a valid Spark Cassandra Connector variable.
No likely matches found.
spark.cassandra.connection.ssl.enabled is not a valid Spark Cassandra Connector variable.
No likely matches found.
spark.cassandra.connection.ssl.protocol is not a valid Spark Cassandra Connector variable.
No likely matches found.
spark.cassandra.connection.ssl.keyStore.type is not a valid Spark Cassandra Connector variable.
No likely matches found.
spark.cassandra.connection.ssl.trustStore.path is not a valid Spark Cassandra Connector variable.
No likely matches found.
spark.cassandra.connection.ssl.enabledAlgorithms is not a valid Spark Cassandra Connector variable.
No likely matches found.
spark.cassandra.connection.ssl.keyStore.path is not a valid Spark Cassandra Connector variable.
No likely matches found.
spark.cassandra.connection.ssl.trustStore.password is not a valid Spark Cassandra Connector variable.
No likely matches found.
spark.cassandra.connection.ssl.trustStore.type is not a valid Spark Cassandra Connector variable.
No likely matches found.

所以我不确定这是否支持，或者我只是使用了错误的属性名称。

我在连接器的1.2.3版本中看到了这个ticket，但我找不到如何使用它的示例，听起来它可能不支持密钥库。我使用的是连接器版本1.4.0-M1。

有人能告诉我一个如何为Spark Cassandra连接器配置SSL的示例吗？感谢。

Answer 1

虽然我没有看到任何密钥库配置，但我可以看到下面的配置变量，它们对我来说运行正常。

注意：我使用的是1.5.0-M1版本。不确定您使用的版本中是否还有其他错误。

sparkConf.set("spark.cassandra.connection.ssl.enabled", "true");
sparkConf.set("spark.cassandra.connection.ssl.trustStore.password", "password");
sparkConf.set("spark.cassandra.connection.ssl.trustStore.path", "jks file path");

如何在Spark和Cassandra之间配置SSL？

1 个答案: