如何使此方法使用更少的内存?

时间:2016-06-01 12:56:50

标签: java regex

以下方法用于解析日志文件并在其中查找错误。它工作正常,直到错误数量达到数百万 - 3千万是具体的。然后服务器耗尽内存试图运行应用程序。我得到GC内存错误。我可以将参数更改为java,以便它可以使用更多内存,但我想先优化一下代码。

for (String logFile : fileList) {
        Map<String, Long> messageErrorCount = new HashMap<String, Long>();
        LOGGER.info("Mining data from: " + logFile);
        BufferedReader reader = null;
        try {
            String hostName = getHostName(logFile);
            reader = new BufferedReader(new FileReader(logFile));
            long errorCountPerFile = 0;
            String logLineStr;
            long firstErrorOccurance = 1;
            while ((logLineStr = reader.readLine()) != null) {
                String errorResult = isErrorLine(logLineStr);
                if (errorResult != null) {
                    errorCountPerFile++;
                    if (messageErrorCount.containsKey(errorResult)) {
                        messageErrorCount.put(errorResult, messageErrorCount.get(errorResult) + 1);
                    } else {
                        messageErrorCount.put(errorResult, firstErrorOccurance);
                    }

                    Pattern pattern = Pattern.compile("^(.*?)\\s-");
                    Matcher matcher = pattern.matcher(errorResult);
                    if (matcher.find()) {
                        String formattedError = matcher.group(1);
                        if (totalErrorMessages.containsKey(formattedError)) {
                            totalErrorMessages.put(formattedError, totalErrorMessages.get(formattedError) + 1);
                        } else {
                            totalErrorMessages.put(formattedError, firstErrorOccurance);
                        }
                    }
                }
            }
            // Need to add the file to list only if count is non zero.
            if (errorCountPerFile > 0) {
                String fileName = new File(logFile).getName();

                fileErrorMaps.put(fileName, messageErrorCount);
                putErrorLogRecord(hostName, fileName, errorCountPerFile, messageErrorCount);
            }
        } catch (IOException e) {
            LOGGER.error("File operation failed: " + logFile);
            throw new ParserException("File operation failed: " + logFile, e);
        } finally {
            try {
                if (reader != null) {
                    reader.close();
                }
            } catch (IOException ex) {
                LOGGER.error("File operation failed: " + logFile);
                throw new ParserException("File operation failed: " + logFile, ex);
            }
        }
    }

isErrorLine方法:

public String isErrorLine(String logString) {
    if (isWhitelistedError(logString)) {
           return null;
    } else {
        Matcher matcher = ULOG_LINE_PATTERN_QUOTES.matcher(logString);
        if (matcher.find()) {
            return (matcher.group(2) + " - " + matcher.group(3)).replace("\"", "");
        } 
        matcher = ULOG_LINE_PATTERN_ENDLINE.matcher(logString);
        if (matcher.find()) {
            return (matcher.group(2) + " - " + matcher.group(3)).replace("\"", "");
        }
    }
    return null;
}

这些行在每次使用时都会创建新对象:

 Pattern pattern = Pattern.compile("^(.*?)\\s-");
 Matcher matcher = pattern.matcher(errorResult);

有没有更好的方法来执行此方法,因此它使用更少的内存?

0 个答案:

没有答案