Question

我尝试使用MongoDB-Spark连接器文档示例，但是，它们不起作用。结果是这样的例外：

MongoTimeoutException: Timed out after 30000 ms while waiting for a server that matches WritableServerSelector. Client view of cluster state is {type=UNKNOWN, servers=[{address=localhost:8081, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketReadException: Prematurely reached end of stream}}]

我的Scala对象中的代码是：

object SparkTest {

  def main(args: Array[String]): Unit = {

    val spark = SparkSession.builder()
      .master("local")
      .appName("MongoSparkConnectorIntro")
      .config("spark.mongodb.input.uri", "mongodb://localhost:8081/admin.new_col")
      .config("spark.mongodb.output.uri", "mongodb://localhost:8081/admin.new_col")
      .getOrCreate()

    val sparkContext = spark.sparkContext
    sparkContext.setLogLevel("ERROR")

    val writeConfig = WriteConfig(Map("uri" -> "mongodb://localhost:8081/admin.new_col"))

    val documents = sparkContext.parallelize((1 to 10).map(i => Document.parse(s"{test: $i}")))

MongoSpark.save(documents, writeConfig)
  }
}

我没有安装Kafka和MongoDB，但我在docker-compose中使用了它们：

version: '3.3'
services:
  kafka:
      image: spotify/kafka
      ports:
        - "9092:9092"
      environment:
      - ADVERTISED_HOST=localhost
  mongo:
      image: mongo
      restart: always
      environment:
        MONGO_INITDB_ROOT_USERNAME: user
        MONGO_INITDB_ROOT_PASSWORD: password

  mongo-express:
      image: mongo-express
      restart: always
      ports:
        - 8081:8081
      environment:
        ME_CONFIG_MONGODB_ADMINUSERNAME: user
        ME_CONFIG_MONGODB_ADMINPASSWORD: password

什么可能导致这种例外？

MongoTimeoutException Spark-MongoDB连接器Scala

0 个答案: