Question

我在Java中有这个MqttConsumer，可以使用Flink使用消息。我以字符串3|TEMPERATURE|1|1|null|25.0的形式接收mqtt消息，并将其拆分以提取每个值。然后，我创建一个由键（MqttSensor）和主题（Tuple3<Integer, String, Tuple2<Integer, Integer>>）和值（String）组成的Double。当我调用ctx.collect(mqttMessage);的方法SourceContext<MqttSensor> ctx时，出现了一个异常，说我无法将Integer转换为String。但是，我想问题是要序列化Tuple3和Tuple2（来源：https://ci.apache.org/projects/flink/flink-docs-stable/dev/types_serialization.html#creating-a-typeinformation-or-typeserializer）。谢谢

Job execution failed.
org.apache.flink.runtime.client.JobExecutionException: Job execution failed.
    at org.apache.flink.runtime.jobmaster.JobResult.toJobExecutionResult(JobResult.java:146)
    at org.apache.flink.runtime.minicluster.MiniCluster.executeJobBlocking(MiniCluster.java:647)
    at org.apache.flink.streaming.api.environment.LocalStreamEnvironment.execute(LocalStreamEnvironment.java:123)
    at org.sense.flink.examples.stream.MultiSensorMultiStationsReadingMqtt.<init>(MultiSensorMultiStationsReadingMqtt.java:48)
    at org.sense.flink.App.main(App.java:130)
Caused by: java.lang.RuntimeException: java.lang.ClassCastException: java.lang.Integer cannot be cast to java.lang.String
Serialization trace:
f1 (org.apache.flink.api.java.tuple.Tuple2)
f2 (org.apache.flink.api.java.tuple.Tuple3)
key (org.sense.flink.mqtt.MqttSensor)
    at org.apache.flink.streaming.runtime.io.RecordWriterOutput.pushToRecordWriter(RecordWriterOutput.java:110)
    at org.apache.flink.streaming.runtime.io.RecordWriterOutput.collect(RecordWriterOutput.java:89)
    at org.apache.flink.streaming.runtime.io.RecordWriterOutput.collect(RecordWriterOutput.java:45)
    at org.apache.flink.streaming.api.operators.AbstractStreamOperator$CountingOutput.collect(AbstractStreamOperator.java:718)
    at org.apache.flink.streaming.api.operators.AbstractStreamOperator$CountingOutput.collect(AbstractStreamOperator.java:696)
    at org.apache.flink.streaming.api.operators.StreamSourceContexts$AutomaticWatermarkContext.processAndCollect(StreamSourceContexts.java:176)
    at org.apache.flink.streaming.api.operators.StreamSourceContexts$WatermarkContext.collect(StreamSourceContexts.java:394)
    at org.sense.flink.mqtt.MqttSensorConsumer.run(MqttSensorConsumer.java:75)
    at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:94)
    at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:58)
    at org.apache.flink.streaming.runtime.tasks.SourceStreamTask.run(SourceStreamTask.java:99)
    at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:300)
    at org.apache.flink.runtime.taskmanager.Task.run(Task.java:704)
    at java.lang.Thread.run(Thread.java:748)

如何序列化Tuple3和Tuple2？这是我的代码：

public class MqttSensorConsumer extends RichSourceFunction<MqttSensor> {

    private static final long serialVersionUID = -1384636057411239133L;
    final private static String DEFAUL_HOST = "127.0.0.1";
    final private static int DEFAUL_PORT = 1883;

    private String host;
    private int port;
    private String topic;
    private QoS qos;

    public MqttSensorConsumer(String topic) {
        this(DEFAUL_HOST, DEFAUL_PORT, topic, QoS.AT_LEAST_ONCE);
    }

    public MqttSensorConsumer(String host, String topic) {
        this(host, DEFAUL_PORT, topic, QoS.AT_LEAST_ONCE);
    }

    public MqttSensorConsumer(String host, int port, String topic) {
        this(host, port, topic, QoS.AT_LEAST_ONCE);
    }

    public MqttSensorConsumer(String host, int port, String topic, QoS qos) {
        this.host = host;
        this.port = port;
        this.topic = topic;
        this.qos = qos;
    }

    @Override
    public void run(SourceContext<MqttSensor> ctx) throws Exception {
        MQTT mqtt = new MQTT();
        mqtt.setHost(host, port);
        BlockingConnection blockingConnection = mqtt.blockingConnection();
        blockingConnection.connect();

        byte[] qoses = blockingConnection.subscribe(new Topic[] { new Topic(topic, qos) });

        while (blockingConnection.isConnected()) {
            Message message = blockingConnection.receive();
            String payload = new String(message.getPayload());
            String[] arr = payload.split("\\|");

            // @formatter:off
            // 2|TEMPERATURE|1|1|null|25.0
            // @formatter:on
            System.out.println("0: " + arr[0]);
            System.out.println("1: " + arr[1]);
            System.out.println("2: " + arr[2]);
            System.out.println("3: " + arr[3]);
            System.out.println("4: " + arr[4]);
            System.out.println("5: " + arr[5]);
            Tuple3<Integer, String, Tuple2<Integer, Integer>> key = Tuple3.of(Integer.parseInt(arr[0]), arr[1],
                    Tuple2.of(Integer.parseInt(arr[2]), Integer.parseInt(arr[3])));
            MqttSensor mqttMessage = new MqttSensor(message.getTopic(), key, Double.valueOf(arr[5]));
            message.ack();
            ctx.collect(mqttMessage);
        }
        blockingConnection.disconnect();
    }

    @Override
    public void cancel() {
        // TODO Auto-generated method stub
    }
}

Answer 1

我用TypeInformation来解决。

TypeInformation<Tuple3<Integer, String, Tuple2<Integer, Integer>>> key = TypeInformation.of(new TypeHint<Tuple3<Integer, String, Tuple2<Integer, Integer>>>() { });

如何在Flink（Java）中序列化Tuple3？

1 个答案: