Servlet文件下载数百万行

时间:2019-06-26 14:53:37

标签: java

Webapp提供csv报告供用户下载。 当表中有数百万行将其写入响应时,我遇到了问题。

我试图按批次选择(使用rowNum),因为将过多的对象放入arraylist将导致内存不足的问题。 但仍然,在stringbuilder中设置太多数据也会导致内存不足。

以下是用于下载CSV的代码:

    public void downloadCSV(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {

    long totalRecordCnt = getTotalRecords(batchId, ...);
    int range = 10000;


    BufferedOutputStream bos = null;
    try {
        StringBuilder tableStr = new StringBuilder();
        bos = new BufferedOutputStream(response.getOutputStream());
        tableStr.append('\ufeff');
        tableStr.append("\"" + ....);

        tableStr.append("Batch ID" + ","
                +"User Name" + "," + "Acc balance" + "\n");

        // split the records into smaller batch to process
        if (totalRecordCnt > 0) {

            long totalBatchCnt = totalRecordCnt / range;
            if (totalRecordCnt % range > 0) {
                totalBatchCnt += 1;
            }

            long totalRecordCntTemp = totalRecordCnt;
            for (int i = 1; i <= totalBatchCnt; i++) {
                long endRange = 0;
                long startRange = ((i - 1) * range) + 1L;
                if (totalRecordCntTemp < range) {
                    endRange = totalRecordCnt;
                } else {
                    endRange = startRange + range - 1;
                }

                totalRecordCntTemp = totalRecordCntTemp - range;

                // process records
                List<Account> account = null;

                account = getAccountList(batchId, startRange,endRange);


                if (account != null && !account.isEmpty()) {
                    for (Account acc : account) {

                        tableStr.append("\"" + acc.getBatchId + "\",\"" + accnt.getName
                                + "\"" + "\n");

                    }
                }
            }
        }

        // setting some response headers
        response.setHeader("Expires", "0");
        response.setHeader("Cache-Control", "must-revalidate, post-check=0, pre-check=0");
        response.setHeader("Pragma", "public");
        // setting the content type
        String csvFileName = "Acc_report" + ".csv";
        response.setContentType("application/vnd.ms-excel");
        response.setHeader("Content-disposition", "attachment; filename=" + csvFileName);

         bos.write(tableStr.toString().getBytes());// Write individual
        // field
                                                  // followed
        bos.flush();
    } catch (Exception e) {
        throw new IOException(e.getMessage());
    } finally {
        if (bos != null) {
            try {
                bos.close();
            } catch (Exception e) {
            }
        }
    }

}

预期:要能够通过单击下载按钮立即生成包含数百万行的报告。

0 个答案:

没有答案