Question

当我运行spark流示例org.apache.spark.examples.streaming.JavaDirectKafkaWordCount时，我发现EOFException关注，我该如何解决？

Exception in thread "main" org.apache.spark.SparkException: java.io.EOFException: Received -1 when reading from channel, socket has likely been closed.
java.io.EOFException: Received -1 when reading from channel, socket has likely been closed.
java.io.EOFException: Received -1 when reading from channel, socket has likely been closed.
    at org.apache.spark.streaming.kafka.KafkaUtils$$anonfun$createDirectStream$2.apply(KafkaUtils.scala:413)
    at org.apache.spark.streaming.kafka.KafkaUtils$$anonfun$createDirectStream$2.apply(KafkaUtils.scala:413)
    at scala.util.Either.fold(Either.scala:97)
    at org.apache.spark.streaming.kafka.KafkaUtils$.createDirectStream(KafkaUtils.scala:412)
    at org.apache.spark.streaming.kafka.KafkaUtils$.createDirectStream(KafkaUtils.scala:528)
    at org.apache.spark.streaming.kafka.KafkaUtils.createDirectStream(KafkaUtils.scala)

Answer 1

直接流使用底层的低级Kafka消费者，因此需要提供经纪人列表。您很可能没有这样做，您可能希望以metadata.broker.list形式设置bhost1:9092,bhost2:9092,...,bhostN:9092属性。

另见Kafka SimpleConsumer cannot connect to zookeeper : Received -1 when reading from channel。

Spark Streaming Kafka Integration直接方法EOFException

1 个答案: