Question

当我从flinkkafkaconsumer09更改为flinkkafkaconsumer时出现了新错误链接代码：

import org.apache.flink.api.common.functions.MapFunction;
import org.apache.flink.streaming.api.datastream.DataStream;
import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;
import org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer;
import org.apache.flink.streaming.util.serialization.SimpleStringSchema;
import java.util.Properties;

@SuppressWarnings("deprecation")
public class ReadFromKafka {


  public static void main(String[] args) throws Exception {
    // create execution environment
    StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();

    Properties properties = new Properties();
    properties.setProperty("bootstrap.servers", "localhost:9092");
    properties.setProperty("group.id", "test-consumer-group");


    DataStream<String> stream = env
            .addSource(new FlinkKafkaConsumer<String>("test4", new SimpleStringSchema(), properties));

    stream.map(new MapFunction<String, String>() {
      private static final long serialVersionUID = -6867736771747690202L;

      @Override
      public String map(String value) throws Exception {
        return "Stream Value: " + value;
      }
    }).print();

    env.execute();
  }


}

错误： log4j：WARN找不到记录器的附加程序（org.apache.flink.api.java.ClosureCleaner）。 log4j：WARN请正确初始化log4j系统。 log4j：WARN有关更多信息，请参见http://logging.apache.org/log4j/1.2/faq.html#noconfig。线程“主”中的异常org.apache.flink.runtime.client.JobExecutionException：作业执行失败。在org.apache.flink.runtime.jobmaster.JobResult.toJobExecutionResult（JobResult.java:146）在org.apache.flink.runtime.minicluster.MiniCluster.executeJobBlocking（MiniCluster.java:626）在org.apache.flink.streaming.api.environment.LocalStreamEnvironment.execute（LocalStreamEnvironment.java:117）在org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.execute（StreamExecutionEnvironment.java:1507）在org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.execute（StreamExecutionEnvironment.java:1489）在ReadFromKafka.main（ReadFromKafka.java:33）由以下原因引起：org.apache.kafka.common.errors.TimeoutException：提取主题元数据时超时已过期

pom.xml

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>com.dataartisans</groupId>
  <artifactId>kafka-example</artifactId>
  <version>0.0.1-SNAPSHOT</version>
  <name>kafkaex</name>
  <description>this is flink kafka example</description>
  <dependencies>
    <dependency>
        <groupId>org.apache.flink</groupId>
        <artifactId>flink-java</artifactId>
        <version>1.9.1</version>
    </dependency>
    <dependency>
        <groupId>org.apache.flink</groupId>
        <artifactId>flink-streaming-java_2.12</artifactId>
        <version>1.9.1</version>
    </dependency>
    <dependency>
        <groupId>org.apache.flink</groupId>
        <artifactId>flink-clients_2.12</artifactId>
        <version>1.9.1</version>
    </dependency>
    <dependency>
        <groupId>org.apache.flink</groupId>
        <artifactId>flink-connector-kafka_2.12</artifactId>
        <version>1.9.1</version>
    </dependency>
    <dependency>
    <groupId>org.apache.flink</groupId>
    <artifactId>flink-core</artifactId>
    <version>1.9.1</version>
</dependency>

    <dependency>
        <groupId>com.googlecode.json-simple</groupId>
        <artifactId>json-simple</artifactId>
        <version>1.1</version>
    </dependency>  
</dependencies>
</project>

Answer 1

Setter' (Bool,Char) Char与flink-connector-kafka_2.12不兼容。

FlinkKafkaConsumer09是一个“通用” kafka连接器，已编译用于Scala 2.12。从0.11.0开始，此通用连接器可与任何版本的Kafka一起使用。

flink-connector-kafka_2.12用于Kafka0.9.x。如果您的Kafka经纪人运行的是Kafka 0.9.x，则需要FlinkKafkaConsumer09或flink-connector-kafka-0.9_2.11，具体取决于所需的Scala版本。

另一方面，如果您的Kafka经纪人正在运行Kafka的最新版本（0.11.0或更高版本），则坚持使用flink-connector-kafka-0.9_2.12并使用flink-connector-kafka_2.12而不是FlinkKafkaConsumer。

有关更多信息，请参见the documentation。

卡夫卡flink连接错误显示NoSuchMethodError

1 个答案: