无法将消息插入Kafka

时间:2018-06-23 16:57:46

标签: apache-kafka kafka-producer-api

当我尝试通过Producer API将消息插入Kafka时遇到以下错误

function filterDates(){
     var list = document.getElementById("myList"); //wanted substring from first of the string to the first space
     var listItems = Array.from(list.children); //list.children is not actually an array
     list.innerHTML = "" //make the list empty
     var now = new Date(); //current time

     var filteredList = listItems.filter(function(li){
        var dateString = li.innerHTML.split(' ')[0];
        var splited = dateString.split('/') //it is an array: [MM, DD, YYYY]
        var dateToCheck = new Date(splited[2], splited[0], splited[1]); // new Date(year, month, day)
        return dateToCheck.getTime() > now.getTime();
     })
     filteredList.forEach(function(el){
        list.appendChild(el);
     })
}

这可能是什么问题?

更多代码如下:

 org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for metadata_notification-0: 30036 ms has passed since batch creation plus linger time

此时,它失败了,并给出了上面的错误-TimeoutException

 protected void produceMessage(String topic, String orgId, Map<String, byte[]> header, String payload) throws Exception {
        System.out.println("In Produce Message Method");
        final Producer<String, String> producer = createMetaProducer();
        long time = System.currentTimeMillis();
        try {
            List<Header> messageHeaders = header.entrySet().stream().map(e -> new RecordHeader(e.getKey(), e.getValue())).collect(Collectors.toList());
            System.out.println("Send header");
            LOGGER.info("Sending headers:");
            for(Header mh: messageHeaders) {
                LOGGER.info("{}: {}",mh.key(),new String(mh.value()));
            }
            final ProducerRecord<String, String> record = new ProducerRecord<>(topic, null, orgId, payload, messageHeaders);
            RecordMetadata metadata = producer.send(record).get();
            long elapsedTime = System.currentTimeMillis() - time;
            System.out.printf("sent record(key=%s value=%s) " +
                            "meta(partition=%d, offset=%d) time=%d\n",
                    record.key(), record.value(), metadata.partition(),
                    metadata.offset(), elapsedTime);
        } finally {
            producer.flush();
            producer.close();
        }
    }

在我的情况下,kafka是码头工人形象

1 个答案:

答案 0 :(得分:0)

您可能会遇到错误KAFKA-5886。有关更多详细信息,请参见KIP-91。为避免此问题,请增加request.timeout.ms