导出大量数据和内存溢出

时间:2013-05-09 05:52:55

标签: java memory overflow

我在计算机中导入了超过一百个数据使用eclipse并没有问题。但是,当我在应用程序服务器上部署时,它会内存不足,我会复制本地tomcat。

这是什么问题?

在另一个函数中,日期超过数百万,而且只有少于几个字段。它可以正常输出。

int totalSize = behaviorPortraitService.querySiletUserPortraitCountMonth(sietUserPortraitForm.getParams());
int pageNum = 0;
if (totalSize % 1000000 == 0) {
    pageNum = totalSize / 1000000;
}else{
    pageNum = totalSize / 1000000 + 1;
}

write.append("        year/month          IMSI                 phone number     The date of open card"
        + enter);
outSTr = response.getOutputStream(); // establish
buff = new BufferedOutputStream(outSTr);
// Loop total number of pages,Get the number of pages per page
for (int i = 1; i <= pageNum; i++) {
    // According to the number of pages to get PageResult object
    list = behaviorPortraitService.exportSiletUserPortraitMonth(sietUserPortraitForm.getParams(),i,100000);
    // circulate List collection to write
    for (int j = 0; j < list.size(); j++) {
        // Recycling collection,Write data to the TXT
        write.append(list.get(j).getBegin_endtime());
        write.append("  " + list.get(j).getImsi());
        write.append("  " + list.get(j).getMsisdn());
        write.append("  "+ list.get(j).getSilencedays());
        write.append("  " + list.get(j).getStarttime()
                + enter);
    }
    // After writing a page object,empty List,wipe cache 
    list.clear();
    System.gc();
}

buff.write(write.toString().getBytes("UTF-8"));

错误:

org.springframework.web.util.NestedServletException: Handler processing failed; nested exception is java.lang.OutOfMemoryError: Java heap space
    org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:839)
    org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:719)
    org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:644)
    org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:549)
    javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
    javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
    com.trajectory.manager.controller.filter.SessionPrivaligeFilter.doFilter(SessionPrivaligeFilter.java:79)
    org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:88)
    org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76)


root cause 

java.lang.OutOfMemoryError: Java heap space
    java.lang.StringCoding$StringEncoder.encode(StringCoding.java:232)
    java.lang.StringCoding.encode(StringCoding.java:272)
    java.lang.String.getBytes(String.java:946)
    com.trajectory.behaviorportrait.controller.ExportSiletUserPortraitController.writeWeekTxtWeek(ExportSiletUserPortraitController.java:209)
    com.trajectory.behaviorportrait.controller.ExportSiletUserPortraitController.handleRequestInternal(ExportSiletUserPortraitController.java:69)
    org.springframework.web.servlet.mvc.AbstractController.handleRequest(AbstractController.java:153)
    org.springframework.web.servlet.mvc.SimpleControllerHandlerAdapter.handle(SimpleControllerHandlerAdapter.java:48)
    org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:790)
    org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:719)
    org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:644)
    org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:549)
    javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
    javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
    com.trajectory.manager.controller.filter.SessionPrivaligeFilter.doFilter(SessionPrivaligeFilter.java:79)
    org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:88)
    org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:76)

1 个答案:

答案 0 :(得分:1)

您正在创建一个非常大的字符串,您要将其转换为非常大的字节[]。

我建议不要将这些数据逐步写入输出流。即不要使用write.append只使用buf.write作为二进制文件,或者如果你想写文本,请使用

PrintWriter pw = new PrintWriter(new OutputStreamWriter(
                 new BufferedOutputStream(socket.getOuptutStream()), "UTF-8"));

pw.print( some text here );
pw.print( some more ext here );
pw.println( the end of the line );

pw.close(); // when finished.

这根本不会占用太多内存。