Question

我正试图在Flink流媒体中读取kafka主题的数据。我正在尝试运行以下示例代码，该代码在页面APACHE Flink 1.1.3文档中作为示例：Apache kafka连接器，

import java.util.Properties;
import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;
import org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer09;
import org.apache.flink.streaming.api.datastream.DataStream;
import org.apache.flink.streaming.util.serialization.DeserializationSchema;
import org.apache.flink.streaming.util.serialization.SerializationSchema;
import org.apache.flink.streaming.util.serialization.SimpleStringSchema;

public class stock_streaming_kafka {

    public static void main(String[] args) throws Exception
    {
        StreamExecutionEnvironment env=StreamExecutionEnvironment.getExecutionEnvironment();
        Properties properties = new Properties();
        properties.setProperty("bootstrap.servers", "localhost:9092");
FlinkKafkaConsumer09<String> myConsumer = new FlinkKafkaConsumer09<>("nsestocks4k", new SimpleStringSchema(), properties);

    DataStream<String> stream = env
        .addSource(myConsumer)
        .print();
}

}

我有以下错误：

Exception in thread "main" java.lang.Error: Unresolved compilation problems: 
The type org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumerBase cannot be resolved. It is indirectly referenced from required .class files
The method addSource(SourceFunction<OUT>) in the type StreamExecutionEnvironment is not applicable for the arguments (FlinkKafkaConsumer09<String>)

at stock_streaming_kafka.main(stock_streaming_kafka.java:25)

你能指导我解决这个问题吗？ Kafka连接器是否存在任何依赖性问题。我的版本是：

Flink 1.1.3
Kafka 2.10
弗林克连接器-卡夫卡0.9_2.11-1.0.0.jar

Answer 1

请使用以下版本。它适用于您的Kafka版本。

   <dependency>
            <groupId>org.apache.flink</groupId>
            <artifactId>flink-streaming-java_2.11</artifactId>
            <version>1.1.4</version>
            <scope>provided</scope>
        </dependency>

        <dependency>
            <groupId>org.apache.flink</groupId>
            <artifactId>flink-connector-kafka-0.9_2.10</artifactId>
            <version>1.1.3</version>
        </dependency>

我在代码中看到了编译问题。

改变这个：

DataStream<String> stream = env
        .addSource(myConsumer)
        .print();

为：

DataStream<String> stream = env
        .addSource(myConsumer);
stream.print();

如果它仍然不适合您，请告诉我，我将分享工作代码。

Answer 2

Flink和Flink连接器的版本必须匹配。将<div class="w3-content w3-section"> <img class="mySlides" src="https://placehold.it/100x100"> <img class="mySlides" src="https://placehold.it/150x150"> <img class="mySlides" src="https://placehold.it/200x200"> </div>依赖关系更新为1.1.3。

Answer 3

由于尚未接受答案，因此使用Flink从Kafka读取数据complete Maven code example。

您可能需要调整pom.xml以匹配您的Kafka和Scala版本的设置。

希望这会有所帮助。

Flink与Kafka作为来源

3 个答案: