Kafka嵌入Spark。依赖性问题

时间:2018-03-20 21:50:58

标签: apache-spark apache-kafka

我尝试将Spark Streaming 2.0.0与Kafka 0.10一起使用。我正在使用我的集成测试https://github.com/manub/scalatest-embedded-kafka但是我在启动服务器时遇到了一些问题。当我尝试使用Spark 2.2.0时,它可以工作。

   <dependency>
        <groupId>net.manub</groupId>
        <artifactId>scalatest-embedded-kafka_2.11</artifactId>
        <version>${embedded-kafka.version}</version> -->I tried many versions.
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>org.apache.spark</groupId>
        <artifactId>spark-streaming-kafka-0-10_2.11</artifactId>
        <version>2.0.2</version>
    </dependency>

An exception or error caused a run to abort: kafka.server.KafkaServer$.$lessinit$greater$default$2()Lorg/apache/kafka/common/utils/Time; 
java.lang.NoSuchMethodError: kafka.server.KafkaServer$.$lessinit$greater$default$2()Lorg/apache/kafka/common/utils/Time;
    at net.manub.embeddedkafka.EmbeddedKafkaSupport$class.startKafka(EmbeddedKafka.scala:467)
    at net.manub.embeddedkafka.EmbeddedKafka$.startKafka(EmbeddedKafka.scala:38)
    at net.manub.embeddedkafka.EmbeddedKafka$.start(EmbeddedKafka.scala:55)
    at iris.orange.ScalaTest$$anonfun$1.apply$mcV$sp(ScalaTest.scala:10)

这似乎是依赖关系的问题,但我没有&#39;开始工作。我选择了一个使用相同kafka版本的嵌入式kafka。

1 个答案:

答案 0 :(得分:1)

您需要使用public class TestFragment : AppCompatDialogFragment { Button btnClick; public override void OnCreate(Bundle savedInstanceState) { base.OnCreate(savedInstanceState); // Create your fragment here } public override View OnCreateView(LayoutInflater inflater, ViewGroup container, Bundle savedInstanceState) { // Use this to return your custom view for this Fragment // return inflater.Inflate(Resource.Layout.YourFragment, container, false); View view=inflater.Inflate(Resource.Layout.test_fragment, container, false); btnClick = view.FindViewById<Button>(Resource.Id.btnClick); btnClick.Click += BtnClick_Click; return view; } private void BtnClick_Click(object sender, EventArgs e) { this.View.Rotation += 5f; } } 的正确版本 https://mvnrepository.com/artifact/org.apache.spark/spark-streaming-kafka-0-10_2.10/2.0.0

spark-streaming-kafka