Question

为什么我在spark-submit上收到java.lang.NoClassDefFoundError错误？ stacktrace如下所示

[appadm@elk01 spark]$ bin/spark-submit --class "com.ipponusa.SparkStringConsumer" --master localhost:9092 samples/my-app-1.0-SNAPSHORT.jar 1
Hello
Exception in thread "main" java.lang.NoClassDefFoundError: scala/collection/GenTraversableOnce$class
        at kafka.utils.Pool.<init>(Pool.scala:26)
        at kafka.consumer.FetchRequestAndResponseStatsRegistry$.<init>(FetchRequestAndResponseStats.scala:61)
        at kafka.consumer.FetchRequestAndResponseStatsRegistry$.<clinit>(FetchRequestAndResponseStats.scala)
        at kafka.consumer.SimpleConsumer.<init>(SimpleConsumer.scala:44)
        at org.apache.spark.streaming.kafka.KafkaCluster.connect(KafkaCluster.scala:52)
        at org.apache.spark.streaming.kafka.KafkaCluster$$anonfun$org$apache$spark$streaming$kafka$KafkaCluster$$withBrokers$1.apply(KafkaCluster.scala:345)
        at org.apache.spark.streaming.kafka.KafkaCluster$$anonfun$org$apache$spark$streaming$kafka$KafkaCluster$$withBrokers$1.apply(KafkaCluster.scala:342)

Answer 1

您必须使用与您编译Spark代码相同的Scala版本。检查它们，您可能需要降级Scala版本。

Spark连接到Kafka获取错误

1 个答案: