消耗大量内存的NioBlockingSelector $ BlockPoller类

时间:2015-10-16 08:00:54

标签: java tomcat out-of-memory heap-dump thread-dump

在堆转储中NioBlockingSelector $ BlockPoller类在检查线程转储时消耗了4GB(已分配)的3.2GB堆,发现正在下载文件的那些线程正在使用此类(一个大小为35MB的文件,而其他的是小于5MB)。 RUNNABLE状态下共有五个线程在服务器关闭时下载文件。处于BLOCKED状态的其他一些线程由于记录器没有消耗内存(这些线程使用的类不在jprofiler检查的堆转储的BiggestObject列表中)。

这些是tomcat进程的顶级日志,而在最后一栏中占用时间

9221 tomcat    20   0 7017m 1.8g  26m S 227.3  4.2  89:33.85 java   12:10           
 9221 tomcat    20   0 7017m 2.7g  26m S 361.3  6.3  96:30.57 java   12:15           
 9221 tomcat    20   0 7017m 2.7g  26m S 360.3  6.3  96:30.58 java   12:20           
 9221 tomcat    20   0 7209m 4.5g  26m S 100.1 10.3 111:32.74 java   12:25           
 9221 tomcat    20   0 7209m 4.5g  26m S 98.5 10.3 111:32.75 java    12:30           
 9221 tomcat    20   0 7247m 4.6g  26m S 94.9 10.4 116:40.82 java    12:35           
 9221 tomcat    20   0 7247m 4.6g  26m S 93.3 10.4 116:40.82 java    12:40           
 9221 tomcat    20   0 7224m 4.5g  26m S 98.3 10.3 121:50.08 java    12:45

使用NioBlockingSelector $ BlockPoller类

的其中一个线程
"ajp-nio-13009-exec-33" daemon prio=10 tid=0x00007fac44010800 nid=0x12aa runnable [0x00007fac81b65000]
   java.lang.Thread.State: RUNNABLE
    at org.apache.tomcat.util.net.NioBlockingSelector$BlockPoller.add(NioBlockingSelector.java:232)
    at org.apache.tomcat.util.net.NioBlockingSelector.write(NioBlockingSelector.java:106)
    at org.apache.tomcat.util.net.NioSelectorPool.write(NioSelectorPool.java:155)
    at org.apache.coyote.ajp.AjpNioProcessor.output(AjpNioProcessor.java:292)
    at org.apache.coyote.ajp.AjpNioProcessor$SocketOutputBuffer.doWrite(AjpNioProcessor.java:541)
    at org.apache.coyote.Response.doWrite(Response.java:533)
    at org.apache.catalina.connector.OutputBuffer.realWriteBytes(OutputBuffer.java:368)
    at org.apache.tomcat.util.buf.ByteChunk.append(ByteChunk.java:356)
    at org.apache.catalina.connector.OutputBuffer.writeBytes(OutputBuffer.java:396)
    at org.apache.catalina.connector.OutputBuffer.write(OutputBuffer.java:385)
    at org.apache.catalina.connector.CoyoteOutputStream.write(CoyoteOutputStream.java:93)
    at org.apache.catalina.connector.CoyoteOutputStream.write(CoyoteOutputStream.java:86)
    at com.appnetix.app.util.DownloadFileServlet.doGet(DownloadFileServlet.java:58)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:621)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:722)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:304)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
    at org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:684)
    at org.apache.catalina.core.ApplicationDispatcher.processRequest(ApplicationDispatcher.java:471)
    at org.apache.catalina.core.ApplicationDispatcher.doForward(ApplicationDispatcher.java:402)
    at org.apache.catalina.core.ApplicationDispatcher.forward(ApplicationDispatcher.java:329)
    at org.apache.jasper.runtime.PageContextImpl.doForward(PageContextImpl.java:745)
    at org.apache.jasper.runtime.PageContextImpl.forward(PageContextImpl.java:716)
    at org.apache.jsp.jsp.intranet.library.user.downloadLibraryFile_jsp._jspService(downloadLibraryFile_jsp.java:309)
    at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70)

仅在堆开始从12:10增加到12:45的时候下载文件的apache日志:

28.186 - - [14/Oct/2015:12:10:12 -0400] "GET /massageheight/control/downloadLibFile?downloadFlag=toExecuteInsertDocDownload&mimeType=&documentNo=1943825730 HTTP/1.1" 200 1514903 "https://springfield2.franconnect.net/massageheight/control/libraryUserHomeNew" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.101 Safari/537.36"
76.182.246.134 - - [14/Oct/2015:12:10:36 -0400] "GET /massageheight/control/downloadLibFile?downloadFlag=toExecuteInsertDocDownload&mimeType=&documentNo=485054654 HTTP/1.1" 200 35598879 "https://springfield2.franconnect.net/massageheight/control/main" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.101 Safari/537.36"
68.108.85.82 - - [14/Oct/2015:12:11:55 -0400] "GET /massageheight/control/downloadLibFile?downloadFlag=toExecuteInsertDocDownload&mimeType=&documentNo=875082220 HTTP/1.1" 200 340332 "https://springfield2.franconnect.net/massageheight/control/libraryUserHomeNew#/section/?submenu=xyz&folderNo=1383109718&parentFolderNo=-1&folderType=0&display=no&display=no&folderDetail=true&documentType=G$param$1" "Mozilla/5.0 (Windows NT 6.1; Trident/7.0; rv:11.0) like Gecko"
99.157.61.172 - - [14/Oct/2015:12:12:02 -0400] "GET /massageheight/control/downloadLibFile?downloadFlag=toExecuteInsertDocDownload&mimeType=&documentNo=2045468694 HTTP/1.1" 200 41195 "https://springfield2.franconnect.net/massageheight/control/main" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.101 Safari/537.36"
68.108.85.82 - - [14/Oct/2015:12:13:18 -0400] "GET /massageheight/control/downloadLibFile?downloadFlag=toExecuteInsertDocDownload&mimeType=&documentNo=1720586886 HTTP/1.1" 200 353212 "https://springfield2.franconnect.net/massageheight/control/libraryUserHomeNew#/section/?submenu=xyz&folderNo=871601944&parentFolderNo=-1&folderType=0&display=no&display=no&folderDetail=true&documentType=G$param$1" "Mozilla/5.0 (Windows NT 6.1; Trident/7.0; rv:11.0) like Gecko"
206.75.151.76 - - [14/Oct/2015:12:14:02 -0400] "GET /massageheight/control/downloadLibFile?downloadFlag=toExecuteInsertDocDownload&mimeType=&documentNo=485054654 HTTP/1.1" 200 3850232 "https://springfield2.franconnect.net/massageheight/control/main" "Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko"
104.180.76.228 - - [14/Oct/2015:12:15:15 -0400] "GET /massageheight/control/downloadLibFile?downloadFlag=toExecuteInsertDocDownload&mimeType=&documentNo=1487852819 HTTP/1.1" 200 2953765 "https://springfield2.franconnect.net/massageheight/control/main" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.101 Safari/537.36"
104.180.76.228 - - [14/Oct/2015:12:15:28 -0400] "GET /massageheight/control/downloadLibFile?downloadFlag=toExecuteInsertDocDownload&mimeType=&documentNo=1487852819 HTTP/1.1" 200 2953765 "https://springfield2.franconnect.net/massageheight/control/main" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.101 Safari/537.36"
104.180.76.228 - - [14/Oct/2015:12:16:34 -0400] "GET /massageheight/control/downloadLibFile?downloadFlag=toExecuteInsertDocDownload&mimeType=&documentNo=1487852819 HTTP/1.1" 200 2953765 "https://springfield2.franconnect.net/massageheight/control/libraryUserHomeNew" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.101 Safari/537.36"
107.107.188.183 - - [14/Oct/2015:12:22:52 -0400] "GET /massageheight/control/downloadLibFile?downloadFlag=toExecuteInsertDocDownload&mimeType=&documentNo=485054654 HTTP/1.1" 502 2769 "https://springfield2.franconnect.net/massageheight/control/main" "Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.101 Safari/537.36"
206.75.151.76 - - [14/Oct/2015:12:25:36 -0400] "GET /massageheight/control/downloadLibFile?downloadFlag=toExecuteInsertDocDownload&mimeType=&documentNo=485054654 HTTP/1.1" 502 2769 "https://springfield2.franconnect.net/massageheight/control/main" "Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko"
206.75.151.76 - - [14/Oct/2015:12:27:53 -0400] "GET /massageheight/control/images/franconnect-logo.gif HTTP/1.1" 400 3809 "https://springfield2.franconnect.net/massageheight/control/downloadLibFile?downloadFlag=toExecuteInsertDocDownload&mimeType=&documentNo=485054654" "Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko"
206.75.151.76 - - [14/Oct/2015:12:27:53 -0400] "GET /massageheight/control/images/styleerrorNew.css HTTP/1.1" 400 3809 "https://springfield2.franconnect.net/massageheight/control/downloadLibFile?downloadFlag=toExecuteInsertDocDownload&mimeType=&documentNo=485054654" "Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko"
206.75.151.76 - - [14/Oct/2015:12:28:18 -0400] "GET /massageheight/control/images/404.jpg HTTP/1.1" 400 3809 "https://springfield2.franconnect.net/massageheight/control/downloadLibFile?downloadFlag=toExecuteInsertDocDownload&mimeType=&documentNo=485054654" "Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko"
206.75.151.76 - - [14/Oct/2015:12:13:42 -0400] "GET /massageheight/control/downloadLibFile?downloadFlag=toExecuteInsertDocDownload&mimeType=&documentNo=485054654 HTTP/1.1" 503 - "https://springfield2.franconnect.net/massageheight/control/main" "Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko"
206.75.151.76 - - [14/Oct/2015:12:16:28 -0400] "GET /massageheight/control/downloadLibFile?downloadFlag=toExecuteInsertDocDownload&mimeType=&documentNo=485054654 HTTP/1.1" 503 - "https://springfield2.franconnect.net/massageheight/control/main" "Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko"
107.107.188.183 - - [14/Oct/2015:12:22:37 -0400] "GET /massageheight/control/downloadLibFile?downloadFlag=toExecuteInsertDocDownload&mimeType=&documentNo=485054654 HTTP/1.1" 503 - "https://springfield2.franconnect.net/massageheight/control/main" "Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.101 Safari/537.36"
206.75.151.76 - - [14/Oct/2015:12:14:28 -0400] "GET /massageheight/control/downloadLibFile?downloadFlag=toExecuteInsertDocDownload&mimeType=&documentNo=485054654 HTTP/1.1" 503 - "-" "Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko"
107.107.188.183 - - [14/Oct/2015:12:22:50 -0400] "GET /massageheight/control/downloadLibFile?downloadFlag=toExecuteInsertDocDownload&mimeType=&documentNo=485054654 HTTP/1.1" 503 - "https://springfield2.franconnect.net/massageheight/control/main" "Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.101 Safari/537.36"
108.232.75.214 - - [14/Oct/2015:12:47:05 -0400] "GET /massageheight/control/downloadLibFile?downloadFlag=toExecuteInsertDocDownload&mimeType=null&documentNo=2050688329 HTTP/1.1" 200 1743914 "https://springfield2.franconnect.net/massageheight/control/searchAllDocuments?documents=3&load=false&documents=2&news=1&faq=1&alerts=1&alerts=1&searchAllDocument=massage%20therapist%20new%20hire&corp=1&fran=1&reguser=1&message=1" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2490.71 Safari/537.36"
184.96.214.9 - - [14/Oct/2015:12:47:01 -0400] "GET /massageheight/control/downloadLibFile?downloadFlag=toExecuteInsertDocDownload&mimeType=&documentNo=1003477764 HTTP/1.1" 200 7543138 "https://springfield2.franconnect.net/massageheight/control/libraryUserHomeNew" "Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.1; WOW64; Trident/6.0)"
184.96.214.9 - - [14/Oct/2015:12:48:33 -0400] "GET /massageheight/control/downloadLibFile?downloadFlag=toExecuteInsertDocDownload&mimeType=&documentNo=1699912750 HTTP/1.1" 200 46748 "https://springfield2.franconnect.net/massageheight/control/libraryUserHomeNew" "Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.1; WOW64; Trident/6.0)"

正在下载文件的Servlet的代码片段:

byte fileBytes[] = (byte[]) request.getAttribute("FILEBYTES");
try {
    out = response.getOutputStream();
    out.write(fileBytes);
    out.flush();
    }
  

我的问题是,如果我们不下载大文件,那么为什么呢   NioBlockingSelector $ BlockPoller类消耗了大量的内存和   如何发生导致tomcat失效的OutOfMemoryError。

jvm option given to this application  -Xmx:4096 , -Xms:1024

0 个答案:

没有答案