Question

我正在尝试使用Apache Flink消耗事件。该代码非常基本，它试图按主题连接主题拆分词并将其打印到控制台。 Kafka版本是0.9。

import org.apache.flink.api.common.functions.FlatMapFunction;
import org.apache.flink.api.common.serialization.SimpleStringSchema;

import org.apache.flink.streaming.api.datastream.DataStream;
import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;
import org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer09;
import org.apache.flink.util.Collector;
import java.util.Properties;

public class KafkaStreaming {

public static void main(String[] args) throws Exception {
    final StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();

    Properties props = new Properties();
    props.setProperty("bootstrap.servers", "kafka servers:9092...");
    props.setProperty("zookeeper.connect", "kafka servers:2181...");
    props.setProperty("group.id", "flinkPOC");
    FlinkKafkaConsumer09<String> consumer = new FlinkKafkaConsumer09<>("topic", new SimpleStringSchema(), props);

    DataStream<String> dataStream = env.addSource(consumer);

    DataStream<String> wordDataStream = dataStream.flatMap(new Splitter());
    wordDataStream.print();
    env.execute("Word Split");

}

public static class Splitter implements FlatMapFunction<String, String> {

    public void flatMap(String sentence, Collector<String> out) throws Exception {

        for (String word : sentence.split(" ")) {
            out.collect(word);
        }
    }

}
}

该应用程序不会在屏幕上打印任何内容（尽管我向Kafka生成了事件）。我试图跳过Splitter FlatMap函数，但仍然没有任何反应。 Kafka不需要SSL。当我将作业提交到集群时，在日志超时异常中发现了

2019-08-20 14:36:17,654 INFO  org.apache.flink.runtime.executiongraph.ExecutionGraph        - Source: Custom Source -> Flat Map -> Sink: Print to Std. Out (1/1) (02258a2cafab83afbc0f5650c088da2b) switched from RUNNING to FAILED.
org.apache.kafka.common.errors.TimeoutException: Timeout expired while fetching topic metadata

我真的不确定我在做什么错：(

Answer 1

问题显然是在Kafka 9的Flink版本中，因为使用Kafka 2时，相同的代码可以正常运行。因此，恐怕Kafka 09无法解决。

Answer 2

这是一个常见错误，仅告诉您客户端无法正确连接到Kafka群集。您知道集群是否启用了身份验证吗？您可以使用相同的if (result.error) { // Show error in payment form } else { // The card action has been handled // The PaymentIntent can be confirmed again on the server return fetch(`http://127.0.0.1:8000/api/create-charge/`, { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify({ payment_intent_id: result.paymentIntent.id }) }) .then(function(confirmResult) { return confirmResult.json(); }) .then(data => this.handleServerResponse(data)); } }.bind(this)````属性通过kafka-topics脚本连接到群集吗？

我还将使用Flink主机上Kafka随附的zookeeper和kafka-console-producer脚本来确保基本功能。

由于“获取主题元数据时超时已过期”，因此使用Kafka失败

2 个答案: