Question

我使用httpasyncclient 4.1，我发现直接内存不断增长。最后抛出java.lang.OutOfMemoryError。

Exception in thread "I/O dispatcher 2" java.lang.OutOfMemoryError: Direct buffer memory
    at java.nio.Bits.reserveMemory(Bits.java:658)
    at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:123)
    at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:306)
    at sun.nio.ch.Util.getTemporaryDirectBuffer(Util.java:174)
    at sun.nio.ch.IOUtil.read(IOUtil.java:195)
    at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:379)
    at org.apache.http.impl.nio.codecs.AbstractContentDecoder.readFromChannel(AbstractContentDecoder.java:129)
    at org.apache.http.impl.nio.codecs.LengthDelimitedDecoder.read(LengthDelimitedDecoder.java:86)
    at org.apache.http.nio.util.SimpleInputBuffer.consumeContent(SimpleInputBuffer.java:68)
    at org.apache.http.nio.protocol.BasicAsyncResponseConsumer.onContentReceived(BasicAsyncResponseConsumer.java:82)
    at org.apache.http.nio.protocol.AbstractAsyncResponseConsumer.consumeContent(AbstractAsyncResponseConsumer.java:141)
    at org.apache.http.impl.nio.client.MainClientExec.consumeContent(MainClientExec.java:329)
    at org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl.consumeContent(DefaultClientExchangeHandlerImpl.java:153)
    at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.inputReady(HttpAsyncRequestExecutor.java:303)
    at org.apache.http.impl.nio.DefaultNHttpClientConnection.consumeInput(DefaultNHttpClientConnection.java:267)
    at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:81)
    at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:39)
    at org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:116)
    at org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:164)
    at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:339)
    at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:317)
    at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:278)
    at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:106)
    at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:590)
    at java.lang.Thread.run(Thread.java:744)

我已经设置了

MaxDirectMemorySize =百米

，我已经设置了

-XX：-DisableExplicitGC

并且poolingNhttpClientManager的主堆栈正在跟随，似乎select（）卡住了。

sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269)
sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:79)
sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:87)
   - locked sun.nio.ch.Util$2@3920a91
   - locked java.util.Collections$UnmodifiableSet@68f3b76b
   - locked sun.nio.ch.EPollSelectorImpl@588f07a1
sun.nio.ch.SelectorImpl.select(SelectorImpl.java:98)
org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor.execute(AbstractMultiworkerIOReactor.java:342)
org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager.execute(PoolingNHttpClientConnectionManager.java:191)
org.apache.http.impl.nio.client.CloseableHttpAsyncClientBase$1.run(CloseableHttpAsyncClientBase.java:64)
java.lang.Thread.run(Thread.java:744)

Answer 1

你可能会得到这个，因为你达到了-XX：MaxDirectMemorySize设置的直接内存的最大限制。应用程序中的所有NIO DirectByteBuffer对象（如上面的堆栈中的对象）都保留在直接内存中，直到它们被垃圾回收才会释放。正如您在OpenJDK代码here中看到的，一旦这些对象持有的直接内存量超过MaxDirectMemorySize，就会抛出OutOfMemoryError。这是相关的方法：

static void reserveMemory(long size, int cap) {
    synchronized (Bits.class) {
        if (!memoryLimitSet && VM.isBooted()) {
            maxMemory = VM.maxDirectMemory();
            memoryLimitSet = true;
        }
        // -XX:MaxDirectMemorySize limits the total capacity rather than the
        // actual memory usage, which will differ when buffers are page
        // aligned.
        if (cap <= maxMemory - totalCapacity) {
            reservedMemory += size;
            totalCapacity += cap;
            count++;
            return;
        }
    }

    System.gc();
    try {
        Thread.sleep(100);
    } catch (InterruptedException x) {
        // Restore interrupt status
        Thread.currentThread().interrupt();
    }
    synchronized (Bits.class) {
        if (totalCapacity + cap > maxMemory)
            throw new OutOfMemoryError("Direct buffer memory");
        reservedMemory += size;
        totalCapacity += cap;
        count++;
    }

}

正如您在该方法调用中也可以看到的，如果max下的直接内存不足，作为最后的努力，jvm将显式调用System.gc（）以尝试获取一些直接缓冲区对象GCed因此也释放他们的相关记忆。所以你可以在这里做一些事情：

增加-XX：MaxDirectMemorySize
删除-XX：-DisableExplicitGC设置，该设置可以允许在最后一次调用System.gc（）时释放其中一些对象（尽管出于性能原因通常希望避免这种情况）。
调整GC调整，以便在达到最大直接内存上限之前，通过正常的GC过程收集DirectByteBuffer对象

您可以在此处从相关信息中获取更多信息：Impact of setting -XX:+DisableExplicitGC when NIO direct buffers are used。

httpasyncclient OutOfMemoryError直接缓冲内存

1 个答案: