Question

我正在使用SparkSession，SQLContext和DataFrame。所以，这段代码有效。

val sparkSession: SparkSession = SparkSession.builder()
  .master("local")
  .appName("MongoSparkConnectorIntro")
  .config("spark.mongodb.input.uri", "mongodb://127.0.0.1/nasa.eva2")
  .config("spark.mongodb.output.uri", "mongodb://127.0.0.1/nasa.astronautTotals")
  .getOrCreate()
val sqlContext: SQLContext = sparkSession.sqlContext

val evadf: DataFrame = sqlContext.read.option("collection", "eva2").mongo()
evadf.printSchema()

它将打印架构。

但是，当我指向需要身份验证的MongoDB设置时，具有不同URI的相同代码会失败。

val sparkSession: SparkSession = SparkSession.builder()
  .master("local")
  .appName("MongoSparkConnectorIntro")
  .config("spark.mongodb.input.uri", "mongodb://m103-admin:m103-pass@192.168.103.100:27000/nasa.eva2?authSource=admin")
  .config("spark.mongodb.output.uri", "mongodb://m103-admin:m103-pass@192.168.103.100:27000/nasa.astronautTotals?authSource=admin")
.getOrCreate()
val sqlContext: SQLContext = sparkSession.sqlContext

val evadf: DataFrame = sqlContext.read.option("collection", "eva2").mongo()
evadf.printSchema()

为什么？

我正在使用以下

<dependency>
    <groupId>org.apache.spark</groupId>
    <artifactId>spark-core_2.11</artifactId>
    <version>2.2.1</version>
    <scope>provided</scope>
</dependency>
<dependency>
    <groupId>org.mongodb</groupId>
    <artifactId>mongodb-driver</artifactId>
    <version>3.6.3</version>
</dependency>
<dependency>
    <groupId>org.mongodb.spark</groupId>
    <artifactId>mongo-spark-connector_2.11</artifactId>
    <version>2.2.1</version>
</dependency>
<dependency>
    <groupId>org.apache.spark</groupId>
    <artifactId>spark-sql_2.11</artifactId>
    <version>2.2.1</version>
</dependency>

sqlContext.read.option（...）。mongo（）在需要身份验证时失败

0 个答案: