Question

我们有一个java计划任务在java程序中运行查询并将结果发送到基于Jersey的restful webservice。虽然这似乎对某个案例的某些情况正常工作但它不起作用并最终以Java.lang.OutOfMemoryError: Java heap space

以下程序数据以块的形式进行流式处理，但其中一个查询数据的刷新时间不超过一次。因此，所有块都在内存中累积，导致内存不足。

我们已经看过Java - Upload OutputStream as HTTP File Upload HttpURLConnection timeout question 和 Handling large records in a Java EE application

他们中的许多人建议使用HTTPClient，但我们希望尽可能避免使用它。

 for (int i = 0; i < arr.length(); i++) {
            try {
                JSONObject obj = arr.getJSONObject(i);
                StringBuffer query = new StringBuffer((String) obj.get("query")); // Getting one query from list of queries
                int size =500;
                int count = 0;
                URL url = new URL("http://urltomyserver.com");
                java.security.Security.addProvider(new com.sun.net.ssl.internal.ssl.Provider());
                System.setProperty("java.protocol.handler.pkgs", "com.sun.net.ssl.internal.www.protocol");
                httpcon = (HttpURLConnection) url.openConnection();
                httpcon.setChunkedStreamingMode(1024 * 1024);
                httpcon.setAllowUserInteraction(false);
                httpcon.setDoOutput(true);
                httpcon.setRequestProperty("Content-type", "application/x-www-form-urlencoded");
                httpcon.setRequestMethod("POST");
                OutputStream rawOutStream = httpcon.getOutputStream();
                BufferedWriter w = new BufferedWriter(new OutputStreamWriter(rawOutStream));
                do {
                    Query q = new Query(query + " limit " + count + ", " + size);
                    count += size;
                    Map[] results = q.runQuery();
                    JSONArray resultArr = new JSONArray(results);
                    w.write(resultArr.toString(1));
                    w.flush();
                } while (results != null && results.length > 0);
                InputStream inputStream = httpcon.getInputStream();
                w.close();
            } catch (Exception ex) {
                EcwLog.AppendExceptionToLog(ex);
            }
        }

服务器端接收功能的定义如下

public String saveResultsMP(@QueryParam("apuId") final String apuId, @QueryParam("tableName") String tableName, @QueryParam("queryId") String query_id, InputStream uploadedInputStream) {

Java HttpURLConnection问题与来自MySQL的大量数据流

0 个答案: