为什么使用Kafka的Spark Streaming应用程序失败并且＃34;要求失败：numRecords不得为负数＆＃34;？

Question

我有Spark Streaming应用程序，我试图在5节点集群（包括master）上运行。我有2个zookeeper和3个kafka节点。我正在尝试运行HiBench Streaming Benchmarks作为示例应用程序。但是，每当我运行Spark Streaming应用程序时，我都会遇到以下错误：

java.lang.IllegalArgumentException: requirement failed: numRecords must not be negative
        at scala.Predef$.require(Predef.scala:224)
        at org.apache.spark.streaming.scheduler.StreamInputInfo.<init>(InputInfoTracker.scala:38)
        at org.apache.spark.streaming.kafka.DirectKafkaInputDStream.compute(DirectKafkaInputDStream.scala:165)
        at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1$$anonfun$apply$7.apply(DStream.scala:341)
        at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1$$anonfun$apply$7.apply(DStream.scala:341)
        at scala.util.DynamicVariable.withValue(DynamicVariable.scala:58)
        at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1.apply(DStream.scala:340)
        at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1$$anonfun$1.apply(DStream.scala:340)
        at org.apache.spark.streaming.dstream.DStream.createRDDWithLocalProperties(DStream.scala:415)
        at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1.apply(DStream.scala:335)
        at org.apache.spark.streaming.dstream.DStream$$anonfun$getOrCompute$1.apply(DStream.scala:333)
        at scala.Option.orElse(Option.scala:289)

我已尝试删除此类似question中建议的Spark Streaming检查点文件。但是，即使我第一次启动Kafka主题及其相应的消费者Spark Streaming应用程序，问题仍然存在。当我第一次开始这个话题时，问题也无法抵消。