Question

我正在阅读Hadoop IPC实施。 https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java

/**
 * When the read or write buffer size is larger than this limit, i/o will be 
 * done in chunks of this size. Most RPC requests and responses would be
 * be smaller.
 */
private static int NIO_BUFFER_LIMIT = 8*1024; //should not be more than 64KB.

/**
 * This is a wrapper around {@link WritableByteChannel#write(ByteBuffer)}.
 * If the amount of data is large, it writes to channel in smaller chunks. 
 * This is to avoid jdk from creating many direct buffers as the size of 
 * buffer increases. This also minimizes extra copies in NIO layer
 * as a result of multiple write operations required to write a large 
 * buffer.  
 *
 * @see WritableByteChannel#write(ByteBuffer)
 */
private int channelWrite(WritableByteChannel channel, 
                         ByteBuffer buffer) throws IOException {

  int count =  (buffer.remaining() <= NIO_BUFFER_LIMIT) ?
               channel.write(buffer) : channelIO(null, channel, buffer);
  if (count > 0) {
    rpcMetrics.incrSentBytes(count);
  }
  return count;
}


/**
 * This is a wrapper around {@link ReadableByteChannel#read(ByteBuffer)}.
 * If the amount of data is large, it writes to channel in smaller chunks. 
 * This is to avoid jdk from creating many direct buffers as the size of 
 * ByteBuffer increases. There should not be any performance degredation.
 * 
 * @see ReadableByteChannel#read(ByteBuffer)
 */
private int channelRead(ReadableByteChannel channel, 
                        ByteBuffer buffer) throws IOException {

  int count = (buffer.remaining() <= NIO_BUFFER_LIMIT) ?
              channel.read(buffer) : channelIO(channel, null, buffer);
  if (count > 0) {
    rpcMetrics.incrReceivedBytes(count);
  }
  return count;
}

逻辑是，如果缓冲区很小，它将一次读/写通道。如果缓冲区很大，它会多次执行，每次读/写8kb。

我不明白javadocs以及为什么会这样做。为什么“这是为了避免jdk在缓冲区大小增加时创建许多直接缓冲区。”？大缓冲区大小是否也会影响读取性能？

我理解缓冲区大小如何影响FileInputStream性能（link）。但这里是SocketChannel。所以它是无关的。

Answer 1

好问题。在通道中写入时使用sun.nio.ch.IOUtil，它的write(..)函数中有以下几行

int var7 = var5 <= var6?var6 - var5:0;
ByteBuffer var8 = Util.getTemporaryDirectBuffer(var7);

这是 Util.getTemporaryDirectBuffer

static ByteBuffer getTemporaryDirectBuffer(int var0) {
    Util.BufferCache var1 = (Util.BufferCache)bufferCache.get();
    ByteBuffer var2 = var1.get(var0);
    if(var2 != null) {
        return var2;
    } else {
        if(!var1.isEmpty()) {
            var2 = var1.removeFirst();
            free(var2);
        }

        return ByteBuffer.allocateDirect(var0);
    }
}

在负载很重的情况下，当int var0处于较大范围内时，它会产生大量新缓冲区，并free(..)旧缓冲区。bufferCache。因为IOUtil.IOV_MAX具有有限的长度（等于系统配置中定义的On modern Linux systems, the limit is 1024。This is to avoid jdk from creating many direct buffers as the size of buffer increases.）并且不会存储每个长度的缓冲区。我认为这是<script> // jQuery will bind a click function for your A tag with #menu-link-one id. $("#menu-link-one").click(function(){ // Here you have to inform the target of your ajax request (your server). var uri = "<%=request.getContextPath()%>/path/for/your/desired.jsp"; var uriForServlets = "<%=request.getContextPath()%>/some-mapped-servlet/" var args= {}; $.post(uri, args, function(data){ // This will put the ajax response inside your div with #main-area id. $("#main-area).html(data); }); }); </script>。

缓冲区大小如何影响NIO通道性能？

1 个答案: