卡夫卡消费者延迟

时间：2020-08-31 18:30:47

标签： performance apache-kafka kafka-consumer-api spring-kafka latency

在接收主题消息时，我面临着相当大的延迟，并且想到编写使用Spring Kafka和apache kafka的测试程序。我正在使用测试程序将数据发送到主题，并使用以下使用者记录使用者中接收消息的延迟。记录时间戳和使用者实际收到的时间之间的差异。

春季消费者 

@Component
public class SpringKafkaConsumer {
    @KafkaListener(topics = "websocket",
        containerFactory = "dynamicKafkaListenerContainerFactory")
    public void receiveDeliveryTopic(ConsumerRecord<String, String> record, @Headers   MessageHeaders headers) {
    Log.info("0", "Kafka latency in spring receiver: Topic: {} latency: {}", record.topic(),
            System.currentTimeMillis() - record.timestamp());
}

Apache Consumer 

@Component
public class ApacheKafkaConsumer {
    @PostConstruct
    public void start() {
        Properties props = new Properties();
        props.put("bootstrap.servers", "localhost:9092");
        props.put("group.id", "test");
        props.put("enable.auto.commit", "true");
        props.put("auto.commit.interval.ms", "1000");
        props.put("key.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");
        props.put("value.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");

        KafkaConsumer<String, String> consumer = new KafkaConsumer<>(props);
        consumer.subscribe(Arrays.asList("websocket"));

        while (true) {
            ConsumerRecords<String, String> records = consumer.poll(Duration.ofMillis(100));
            for (ConsumerRecord<String, String> record : records)
                Log.info("0", "Latency in apache kafka consumer: {}", System.currentTimeMillis() - record.timestamp());
        }
    }
}

结果

春季消费者 

Kafka latency in spring receiver: Topic: websocket latency: 246
Kafka latency in spring receiver: Topic: websocket latency: 246
Kafka latency in spring receiver: Topic: websocket latency: 247
Kafka latency in spring receiver: Topic: websocket latency: 267
Kafka latency in spring receiver: Topic: websocket latency: 260
Kafka latency in spring receiver: Topic: websocket latency: 261
Kafka latency in spring receiver: Topic: websocket latency: 260
Kafka latency in spring receiver: Topic: websocket latency: 306
Kafka latency in spring receiver: Topic: websocket latency: 300
Kafka latency in spring receiver: Topic: websocket latency: 300
Kafka latency in spring receiver: Topic: websocket latency: 299
Kafka latency in spring receiver: Topic: websocket latency: 340
Kafka latency in spring receiver: Topic: websocket latency: 337
Kafka latency in spring receiver: Topic: websocket latency: 337
Kafka latency in spring receiver: Topic: websocket latency: 337
Kafka latency in spring receiver: Topic: websocket latency: 345
Kafka latency in spring receiver: Topic: websocket latency: 345
Kafka latency in spring receiver: Topic: websocket latency: 345
Kafka latency in spring receiver: Topic: websocket latency: 346
Kafka latency in spring receiver: Topic: websocket latency: 356
Kafka latency in spring receiver: Topic: websocket latency: 352
Kafka latency in spring receiver: Topic: websocket latency: 350
Kafka latency in spring receiver: Topic: websocket latency: 349
Kafka latency in spring receiver: Topic: websocket latency: 359
Kafka latency in spring receiver: Topic: websocket latency: 355
Kafka latency in spring receiver: Topic: websocket latency: 354
Kafka latency in spring receiver: Topic: websocket latency: 354
Kafka latency in spring receiver: Topic: websocket latency: 361
Kafka latency in spring receiver: Topic: websocket latency: 360
Kafka latency in spring receiver: Topic: websocket latency: 360
Kafka latency in spring receiver: Topic: websocket latency: 360
Kafka latency in spring receiver: Topic: websocket latency: 354
Kafka latency in spring receiver: Topic: websocket latency: 351
Kafka latency in spring receiver: Topic: websocket latency: 351
Kafka latency in spring receiver: Topic: websocket latency: 350
Kafka latency in spring receiver: Topic: websocket latency: 358
Kafka latency in spring receiver: Topic: websocket latency: 356
Kafka latency in spring receiver: Topic: websocket latency: 356
Kafka latency in spring receiver: Topic: websocket latency: 356
Kafka latency in spring receiver: Topic: websocket latency: 362
Kafka latency in spring receiver: Topic: websocket latency: 362
Kafka latency in spring receiver: Topic: websocket latency: 356
Kafka latency in spring receiver: Topic: websocket latency: 356
Kafka latency in spring receiver: Topic: websocket latency: 363
Kafka latency in spring receiver: Topic: websocket latency: 361
Kafka latency in spring receiver: Topic: websocket latency: 359
Kafka latency in spring receiver: Topic: websocket latency: 359
Kafka latency in spring receiver: Topic: websocket latency: 358
Kafka latency in spring receiver: Topic: websocket latency: 351
Kafka latency in spring receiver: Topic: websocket latency: 352
Kafka latency in spring receiver: Topic: websocket latency: 351
Kafka latency in spring receiver: Topic: websocket latency: 370
Kafka latency in spring receiver: Topic: websocket latency: 370
Kafka latency in spring receiver: Topic: websocket latency: 369
Kafka latency in spring receiver: Topic: websocket latency: 368
Kafka latency in spring receiver: Topic: websocket latency: 368
Kafka latency in spring receiver: Topic: websocket latency: 366
Kafka latency in spring receiver: Topic: websocket latency: 366
Kafka latency in spring receiver: Topic: websocket latency: 364
Kafka latency in spring receiver: Topic: websocket latency: 366
Kafka latency in spring receiver: Topic: websocket latency: 365
Kafka latency in spring receiver: Topic: websocket latency: 360
Kafka latency in spring receiver: Topic: websocket latency: 360
Kafka latency in spring receiver: Topic: websocket latency: 360
Kafka latency in spring receiver: Topic: websocket latency: 360
Kafka latency in spring receiver: Topic: websocket latency: 358
Kafka latency in spring receiver: Topic: websocket latency: 358
Kafka latency in spring receiver: Topic: websocket latency: 358
Kafka latency in spring receiver: Topic: websocket latency: 359
Kafka latency in spring receiver: Topic: websocket latency: 356
Kafka latency in spring receiver: Topic: websocket latency: 353
Kafka latency in spring receiver: Topic: websocket latency: 370
Kafka latency in spring receiver: Topic: websocket latency: 363
Kafka latency in spring receiver: Topic: websocket latency: 362
Kafka latency in spring receiver: Topic: websocket latency: 361
Kafka latency in spring receiver: Topic: websocket latency: 379
Kafka latency in spring receiver: Topic: websocket latency: 378
Kafka latency in spring receiver: Topic: websocket latency: 379
Kafka latency in spring receiver: Topic: websocket latency: 373
Kafka latency in spring receiver: Topic: websocket latency: 371
Kafka latency in spring receiver: Topic: websocket latency: 371
Kafka latency in spring receiver: Topic: websocket latency: 366
Kafka latency in spring receiver: Topic: websocket latency: 372
Kafka latency in spring receiver: Topic: websocket latency: 366
Kafka latency in spring receiver: Topic: websocket latency: 366
Kafka latency in spring receiver: Topic: websocket latency: 366
Kafka latency in spring receiver: Topic: websocket latency: 364
Kafka latency in spring receiver: Topic: websocket latency: 374
Kafka latency in spring receiver: Topic: websocket latency: 368
Kafka latency in spring receiver: Topic: websocket latency: 367
Kafka latency in spring receiver: Topic: websocket latency: 366
Kafka latency in spring receiver: Topic: websocket latency: 367
Kafka latency in spring receiver: Topic: websocket latency: 362
Kafka latency in spring receiver: Topic: websocket latency: 363
Kafka latency in spring receiver: Topic: websocket latency: 362
Kafka latency in spring receiver: Topic: websocket latency: 362
Kafka latency in spring receiver: Topic: websocket latency: 361
Kafka latency in spring receiver: Topic: websocket latency: 364
Kafka latency in spring receiver: Topic: websocket latency: 363

Apache Consumer 

Latency in apache kafka consumer: 134  
Latency in apache kafka consumer: 130
Latency in apache kafka consumer: 130
Latency in apache kafka consumer: 130
Latency in apache kafka consumer: 130
Latency in apache kafka consumer: 138
Latency in apache kafka consumer: 133
Latency in apache kafka consumer: 133
Latency in apache kafka consumer: 131
Latency in apache kafka consumer: 143
Latency in apache kafka consumer: 141
Latency in apache kafka consumer: 141
Latency in apache kafka consumer: 142
Latency in apache kafka consumer: 140
Latency in apache kafka consumer: 143
Latency in apache kafka consumer: 137
Latency in apache kafka consumer: 225
Latency in apache kafka consumer: 224
Latency in apache kafka consumer: 225
Latency in apache kafka consumer: 223
Latency in apache kafka consumer: 225
Latency in apache kafka consumer: 222
Latency in apache kafka consumer: 217
Latency in apache kafka consumer: 217
Latency in apache kafka consumer: 217
Latency in apache kafka consumer: 230
Latency in apache kafka consumer: 234
Latency in apache kafka consumer: 233
Latency in apache kafka consumer: 233
Latency in apache kafka consumer: 233
Latency in apache kafka consumer: 231
Latency in apache kafka consumer: 231
Latency in apache kafka consumer: 230
Latency in apache kafka consumer: 228
Latency in apache kafka consumer: 226
Latency in apache kafka consumer: 225
Latency in apache kafka consumer: 229
Latency in apache kafka consumer: 226
Latency in apache kafka consumer: 226
Latency in apache kafka consumer: 227
Latency in apache kafka consumer: 227
Latency in apache kafka consumer: 258
Latency in apache kafka consumer: 253
Latency in apache kafka consumer: 252
Latency in apache kafka consumer: 252
Latency in apache kafka consumer: 254
Latency in apache kafka consumer: 252
Latency in apache kafka consumer: 263
Latency in apache kafka consumer: 262
Latency in apache kafka consumer: 261
Latency in apache kafka consumer: 261
Latency in apache kafka consumer: 261
Latency in apache kafka consumer: 257
Latency in apache kafka consumer: 256
Latency in apache kafka consumer: 255
Latency in apache kafka consumer: 270
Latency in apache kafka consumer: 269
Latency in apache kafka consumer: 266
Latency in apache kafka consumer: 263
Latency in apache kafka consumer: 263
Latency in apache kafka consumer: 263
Latency in apache kafka consumer: 262
Latency in apache kafka consumer: 263
Latency in apache kafka consumer: 260
Latency in apache kafka consumer: 257
Latency in apache kafka consumer: 254
Latency in apache kafka consumer: 253
Latency in apache kafka consumer: 253
Latency in apache kafka consumer: 271
Latency in apache kafka consumer: 270
Latency in apache kafka consumer: 267
Latency in apache kafka consumer: 270
Latency in apache kafka consumer: 266
Latency in apache kafka consumer: 266
Latency in apache kafka consumer: 266
Latency in apache kafka consumer: 266
Latency in apache kafka consumer: 264
Latency in apache kafka consumer: 296
Latency in apache kafka consumer: 295
Latency in apache kafka consumer: 297
Latency in apache kafka consumer: 293
Latency in apache kafka consumer: 294
Latency in apache kafka consumer: 298
Latency in apache kafka consumer: 297
Latency in apache kafka consumer: 295
Latency in apache kafka consumer: 295
Latency in apache kafka consumer: 298
Latency in apache kafka consumer: 298
Latency in apache kafka consumer: 302
Latency in apache kafka consumer: 302
Latency in apache kafka consumer: 302
Latency in apache kafka consumer: 300
Latency in apache kafka consumer: 303
Latency in apache kafka consumer: 302
Latency in apache kafka consumer: 303
Latency in apache kafka consumer: 301
Latency in apache kafka consumer: 300
Latency in apache kafka consumer: 299
Latency in apache kafka consumer: 300
Latency in apache kafka consumer: 299

为什么我在kafka消费者中看到如此高的延迟？

0 个答案:

没有答案
相关问题
最新问题