Question

我和卡夫卡合作，我做了这样的制作人：

synchronized (obj) {

        while (true){

            long start = Instant.now().toEpochMilli();
            for (int i=0; i< NUM_MSG_SEC  ; i++)  
            {

                PriceStreamingData data = PriceStreamingData.newBuilder()
                        .setUser(getRequest().getUser())
                        .setSecurity(getRequest().getSecurity())
                        .setTimestamp(Instant.now().toEpochMilli())
                        .setPrice(new Random().nextDouble()*200)
                        .build();


                record = new ProducerRecord<>(topic, keyBuilder.build(data), 
                        data);



                producer.send(record,new Callback(){
                    @Override
                    public void onCompletion(RecordMetadata arg0, Exception arg1) {
                        counter.incrementAndGet();
                        if(arg1 != null){
                            arg1.printStackTrace();
                        }


                    } 
                });

            }
            long diffCiclo = Instant.now().toEpochMilli() - start;
            long diff = Instant.now().toEpochMilli() - startTime;


            System.out.println("Number of sent: " + counter.get() +  
                    " Millisecond:" + (diff) + " - NumberOfSent/Diff(K): " + counter.get()/diff );

            try {
                if(diffCiclo >= 1000){
                    System.out.println("over 1 second: "  + diffCiclo);

                }
                else {
                    obj.wait( 1000 - diffCiclo );

                }
            } catch (InterruptedException e) {
                // TODO Auto-generated catch block
                e.printStackTrace();
            }




        }
    }

你可以看到它非常简单，它只是发一条新消息并发送它。如果我看到日志：

    NumberOfSent/Diff(K)

在前10秒内表现非常糟糕

  30k per second

60秒后我

  180k  per second

为什么？我怎么能把这个过程开始到180k呢？

我的kafka制作人配置是Follwing

  Async producer ( but also with sync producer the situation dose not change)
  ACKS_CONFIG = 0
  BATCH_SIZE_CONFIG = 20000 
  COMPRESSION_TYPE_CONFIG = none
  LINGER_MS_CONFIG = 0

最后一个细节：

  NUM_MSG_SEC is set to 200000 or bigger number

Kafka在第一秒钟生成消息的速度很慢

0 个答案: