致命异常：使用okhttp3 okio下载文件的java.lang.OutOfMemoryError

Question

我有一个应用程序可以从网络上下载内容。 音乐，视频，pdf ....就像下载经理一样。

但现在每次下载内容时都会崩溃：

$("#link").attr("href",finalLink);

和

E/LVN/advanced_memory_manager.c: ---------------------------------- AMM report ------------------------------
-> Memory Currently Allocated: 0 bytes <=> 0 components
-> Max Memory Need: 512000 bytes
-> Overall Memory Allocation: 515652 bytes (l:423)
E/art: Throwing OutOfMemoryError "Failed to allocate a 2060 byte allocation with 16777232 free bytes and 308MB until OOM; failed due to fragmentation (required continguous free 4096 bytes where largest contiguous free 0 bytes)"
E/AndroidRuntime: FATAL EXCEPTION: OkHttp Dispatcher
    Process: com.rokki.life2, PID: 32171
    java.lang.OutOfMemoryError: Failed to allocate a 2060 byte allocation with 16777232 free bytes and 308MB until OOM; failed due to fragmentation (required continguous free 4096 bytes where largest contiguous free 0 bytes)
        at okio.Segment.<init>(Segment.java:58)
        at okio.SegmentPool.take(SegmentPool.java:46)
        at okio.Buffer.writableSegment(Buffer.java:1114)
        at okio.Okio$2.read(Okio.java:137)
        at okio.AsyncTimeout$2.read(AsyncTimeout.java:211)
        at okio.RealBufferedSource.read(RealBufferedSource.java:50)
        at okhttp3.internal.http.Http1xStream$FixedLengthSource.read(Http1xStream.java:381)
        at okio.RealBufferedSource.request(RealBufferedSource.java:71)
        at okhttp3.logging.HttpLoggingInterceptor.intercept(HttpLoggingInterceptor.java:225)
        at okhttp3.RealCall$ApplicationInterceptorChain.proceed(RealCall.java:187)
        at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:160)
        at okhttp3.RealCall.access$100(RealCall.java:30)
        at okhttp3.RealCall$AsyncCall.execute(RealCall.java:127)
        at okhttp3.internal.NamedRunnable.run(NamedRunnable.java:32)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1112)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:587)
        at java.lang.Thread.run(Thread.java:818)

我已将此添加到我的清单中：

Fatal Exception: java.lang.OutOfMemoryError: Failed to allocate a 106405148 byte allocation with 16777120 free bytes and 82MB until OOM
       at java.lang.String.<init>(String.java:332)
       at java.lang.String.<init>(String.java:371)
       at okio.Buffer.readString(Buffer.java:579)
       at okio.Buffer.readString(Buffer.java:562)
       at okhttp3.logging.HttpLoggingInterceptor.intercept(HttpLoggingInterceptor.java:244)
       at okhttp3.RealCall$ApplicationInterceptorChain.proceed(RealCall.java:187)
       at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:160)
       at okhttp3.RealCall.access$100(RealCall.java:30)
       at okhttp3.RealCall$AsyncCall.execute(RealCall.java:127)
       at okhttp3.internal.NamedRunnable.run(NamedRunnable.java:32)
       at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1112)
       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:587)
       at java.lang.Thread.run(Thread.java:818)

代码示例：

<application
    android:name=".app.MyApp"
    android:allowBackup="true"
    android:hardwareAccelerated="true"
    android:icon="@mipmap/ic_launcher"
    tools:replace="android:icon"
    android:label="@string/app_name"
    android:largeHeap="true"
    android:supportsRtl="true"
    android:theme="@style/AppTheme.NoActionBar">

我的BuildGradle：

private OkHttpClient client;


onViewCreated...{

    //Initiate OkHttp with interceptor
    HttpLoggingInterceptor logging = new HttpLoggingInterceptor();
    logging.setLevel(HttpLoggingInterceptor.Level.BODY);

    client = new OkHttpClient.Builder()
            .addInterceptor(logging)
            .build();

...
}



private void downloadPdf() {
AsyncTask.execute(new Runnable() {
    @Override
    public void run() {


    client
        .newCall(getRequest(Config._API_PDF))
        .enqueue(new Callback() {
            @Override
            public void onFailure(Call call, IOException e) {
                ...
            }


              @Override
                public void onResponse(Call call, Response response) throws IOException {


              try {

                                    InputStream ins = response.body().byteStream();
                                    BufferedReader in = new BufferedReader(new InputStreamReader(ins));

                                    String lineTotal = "";
                                    while (true) {
                                        String line = in.readLine();
                                        if (line == null)
                                            break;
                                        else
                                            lineTotal += line;
                                    }

                                    ...json parsing ...



...

然后我有另一个类似的下载视频方法......

以前有人遇到过这个吗？ 感谢

Answer 1

使用HttpLoggingInterceptor.Level.BODY时，尝试下载大文件，会将所有正文保存在内存中以备日志。

对OOM来说很容易。

尝试删除日志正文，或者仅记录NONE，basic或header，然后重试。

当您尝试获取大文件时，可以在Retrofit2中使用@Streaming。

@Streaming
@GET
fun downloadFileWithDynamicUrlSync(@Url fileUrl: String): Call<ResponseBody>

尝试在此处删除日志正文。

onViewCreated...{

//Initiate OkHttp with interceptor
HttpLoggingInterceptor logging = new HttpLoggingInterceptor();
logging.setLevel(HttpLoggingInterceptor.Level.BODY);

尝试一下。

onViewCreated...{

//Initiate OkHttp with interceptor
HttpLoggingInterceptor logging = new HttpLoggingInterceptor();
logging.setLevel(HttpLoggingInterceptor.Level.NONE);

Answer 2

如果赶时间，请复制/粘贴此版本的 HttpLoggingInterceptor 并像原始版本一样使用它。它将通过切断太长的日志来解决您的问题。效果非常好。

class CustomHttpLoggingInterceptor @JvmOverloads constructor(
        private val maxLogSize : Long = 5000,
        private val logger: Logger = Logger.DEFAULT
) : Interceptor {

    @Volatile private var headersToRedact = emptySet<String>()

    @set:JvmName("level")
    @Volatile var level = Level.NONE

    enum class Level {
        /** No logs. */
        NONE,

        /**
         * Logs request and response lines.
         *
         * Example:
         * ```
         * --> POST /greeting http/1.1 (3-byte body)
         *
         * <-- 200 OK (22ms, 6-byte body)
         * ```
         */
        BASIC,

        /**
         * Logs request and response lines and their respective headers.
         *
         * Example:
         * ```
         * --> POST /greeting http/1.1
         * Host: example.com
         * Content-Type: plain/text
         * Content-Length: 3
         * --> END POST
         *
         * <-- 200 OK (22ms)
         * Content-Type: plain/text
         * Content-Length: 6
         * <-- END HTTP
         * ```
         */
        HEADERS,

        /**
         * Logs request and response lines and their respective headers and bodies (if present).
         *
         * Example:
         * ```
         * --> POST /greeting http/1.1
         * Host: example.com
         * Content-Type: plain/text
         * Content-Length: 3
         *
         * Hi?
         * --> END POST
         *
         * <-- 200 OK (22ms)
         * Content-Type: plain/text
         * Content-Length: 6
         *
         * Hello!
         * <-- END HTTP
         * ```
         */
        BODY
    }

    interface Logger {
        fun log(message: String)

        companion object {
            /** A [Logger] defaults output appropriate for the current platform. */
            @JvmField
            val DEFAULT: Logger = object : Logger {
                override fun log(message: String) {
                    Platform.get().log(Platform.INFO, message, null)
                }
            }
        }
    }

    fun redactHeader(name: String) {
        val newHeadersToRedact = TreeSet(String.CASE_INSENSITIVE_ORDER)
        newHeadersToRedact += headersToRedact
        newHeadersToRedact += name
        headersToRedact = newHeadersToRedact
    }

    @Deprecated(
            message = "Moved to var. Replace setLevel(...) with level(...) to fix Java",
            replaceWith = ReplaceWith(expression = "apply { this.level = level }"),
            level = DeprecationLevel.WARNING)
    fun setLevel(level: Level) = apply {
        this.level = level
    }

    @JvmName("-deprecated_level")
    @Deprecated(
            message = "moved to var",
            replaceWith = ReplaceWith(expression = "level"),
            level = DeprecationLevel.ERROR)
    fun getLevel(): Level = level

    @Throws(IOException::class)
    override fun intercept(chain: Interceptor.Chain): Response {
        val level = this.level

        val request = chain.request()
        if (level == Level.NONE) {
            return chain.proceed(request)
        }

        val logBody = level == Level.BODY
        val logHeaders = logBody || level == Level.HEADERS

        val requestBody = request.body

        val connection = chain.connection()
        var requestStartMessage =
                ("--> ${request.method} ${request.url}${if (connection != null) " " + connection.protocol() else ""}")
        if (!logHeaders && requestBody != null) {
            requestStartMessage += " (${requestBody.contentLength()}-byte body)"
        }
        logger.log(requestStartMessage)

        if (logHeaders) {
            if (requestBody != null) {
                // Request body headers are only present when installed as a network interceptor. Force
                // them to be included (when available) so there values are known.
                requestBody.contentType()?.let {
                    logger.log("Content-Type: $it")
                }
                if (requestBody.contentLength() != -1L) {
                    logger.log("Content-Length: ${requestBody.contentLength()}")
                }
            }

            val headers = request.headers
            for (i in 0 until headers.size) {
                val name = headers.name(i)
                // Skip headers from the request body as they are explicitly logged above.
                if (!"Content-Type".equals(name, ignoreCase = true) &&
                        !"Content-Length".equals(name, ignoreCase = true)) {
                    logHeader(headers, i)
                }
            }

            if (!logBody || requestBody == null) {
                logger.log("--> END ${request.method}")
            } else if (bodyHasUnknownEncoding(request.headers)) {
                logger.log("--> END ${request.method} (encoded body omitted)")
            } else if (requestBody.isDuplex()) {
                logger.log("--> END ${request.method} (duplex request body omitted)")
            } else {
                val buffer = Buffer()
                requestBody.writeTo(buffer)

                val contentType = requestBody.contentType()
                val charset: Charset = contentType?.charset(StandardCharsets.UTF_8) ?: StandardCharsets.UTF_8

                logger.log("")
                if (buffer.isProbablyUtf8()) {
                    if(requestBody.contentLength()>maxLogSize)logger.log(buffer.readString(maxLogSize,charset))
                    else logger.log(buffer.readString(charset))
                    logger.log("--> END ${request.method} (${requestBody.contentLength()}-byte body)")
                } else {
                    logger.log(
                            "--> END ${request.method} (binary ${requestBody.contentLength()}-byte body omitted)")
                }
            }
        }

        val startNs = System.nanoTime()
        val response: Response
        try {
            response = chain.proceed(request)
        } catch (e: Exception) {
            logger.log("<-- HTTP FAILED: $e")
            throw e
        }

        val tookMs = TimeUnit.NANOSECONDS.toMillis(System.nanoTime() - startNs)

        val responseBody = response.body!!
        val contentLength = responseBody.contentLength()
        val bodySize = if (contentLength != -1L) "$contentLength-byte" else "unknown-length"
        logger.log(
                "<-- ${response.code}${if (response.message.isEmpty()) "" else ' ' + response.message} ${response.request.url} (${tookMs}ms${if (!logHeaders) ", $bodySize body" else ""})")

        if (logHeaders) {
            val headers = response.headers
            for (i in 0 until headers.size) {
                logHeader(headers, i)
            }

            if (!logBody || !response.promisesBody()) {
                logger.log("<-- END HTTP")
            } else if (bodyHasUnknownEncoding(response.headers)) {
                logger.log("<-- END HTTP (encoded body omitted)")
            } else {
                val source = responseBody.source()
                source.request(Long.MAX_VALUE) // Buffer the entire body.
                var buffer = source.buffer

                var gzippedLength: Long? = null
                if ("gzip".equals(headers["Content-Encoding"], ignoreCase = true)) {
                    gzippedLength = buffer.size
                    GzipSource(buffer.clone()).use { gzippedResponseBody ->
                        buffer = Buffer()
                        buffer.writeAll(gzippedResponseBody)
                    }
                }

                val contentType = responseBody.contentType()
                val charset: Charset = contentType?.charset(StandardCharsets.UTF_8) ?: StandardCharsets.UTF_8

                if (!buffer.isProbablyUtf8()) {
                    logger.log("")
                    logger.log("<-- END HTTP (binary ${buffer.size}-byte body omitted)")
                    return response
                }

                if (contentLength != 0L) {
                    logger.log("")
                    logger.log(buffer.clone().readString(charset))
                }

                if (gzippedLength != null) {
                    logger.log("<-- END HTTP (${buffer.size}-byte, $gzippedLength-gzipped-byte body)")
                } else {
                    logger.log("<-- END HTTP (${buffer.size}-byte body)")
                }
            }
        }

        return response
    }

    private fun logHeader(headers: Headers, i: Int) {
        val value = if (headers.name(i) in headersToRedact) "██" else headers.value(i)
        logger.log(headers.name(i) + ": " + value)
    }

    private fun bodyHasUnknownEncoding(headers: Headers): Boolean {
        val contentEncoding = headers["Content-Encoding"] ?: return false
        return !contentEncoding.equals("identity", ignoreCase = true) &&
                !contentEncoding.equals("gzip", ignoreCase = true)
    }

    fun  Buffer.isProbablyUtf8(): Boolean {
        try {
            val prefix = Buffer()
            val byteCount = size.coerceAtMost(64)
            copyTo(prefix, 0, byteCount)
            for (i in 0 until 16) {
                if (prefix.exhausted()) {
                    break
                }
                val codePoint = prefix.readUtf8CodePoint()
                if (Character.isISOControl(codePoint) && !Character.isWhitespace(codePoint)) {
                    return false
                }
            }
            return true
        } catch (_: EOFException) {
            return false // Truncated UTF-8 sequence.
        }
    }

}

然后像往常一样调用它：

 
   val interceptor = CustomHttpLoggingInterceptor( )
            interceptor.apply { interceptor.level = CustomHttpLoggingInterceptor.Level.BODY }
            clientBuilder.addInterceptor(interceptor)
 

更深入的探索：

OkHttp 团队非常了解这个问题，他们认为这不是问题，并建议开发人员 fork 项目并实现适合他们需求的自己的版本。这对他们的项目策略很有意义。 read more here

错误出现在这一行 buffer.readString(charset) 尝试记录您的整个请求正文，通过限制它buffer.readString(maxLogSize,charset)，我们在避免 OOM 错误之后切断任何日志行。简单！

Answer 3

在gradle中的defaultConfig中添加波纹管

multiDexEnabled true

然后在Android功能

dexOptions {
        incremental = true;
        preDexLibraries = false
        javaMaxHeapSize "4g" // 2g should be also OK
    }

添加此依赖

 compile 'com.android.support:multidex:1.0.1'