Question

我试图在ANTLR版本4的帮助下解析日志文件。目前该文件由10703行组成，它可以超过数百万行。 Java正在提示堆异常。我将行减少到300.程序运行成功，但当我将它设为400时，它再次开始给我堆异常。我也放大了java的堆内存，但没有运气。我也不知道这个问题是java还是ANTLR。

这是执行语法的程序：

public class parser {
    public static void main(String[] args) {
        System.out.println("Start");
        String fileName = "D:\\folder\\logs.out";
        File file = new File(fileName);
        FileInputStream fis = null;

        try {
        // Open the input file stream

        fis = new FileInputStream(file);

        // Create a CharStream that reads from standard input

        ANTLRInputStream input = new ANTLRInputStream(fis);

        GrammarOSBLexer lexer=new GrammarOSBLexer(input);

        TokenStream tokenStream=new CommonTokenStream(lexer);
        GrammarOSBParser parsr=new GrammarOSBParser(tokenStream);
        fis.close();

        try {
            parsr.logs();

         } catch (RecognitionException e) {

             e.printStackTrace();
         }

        System.out.println("done!");

    }catch (IOException e) {

        e.printStackTrace();
   }            

  } 

}

`

以下是例外：

Exception in thread "main" java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at org.eclipse.jdt.internal.jarinjarloader.JarRsrcLoader.main(JarRsrcLoader.java:58)
Caused by: java.lang.OutOfMemoryError: Java heap space
    at java.util.Arrays.copyOf(Unknown Source)
    at java.util.Arrays.copyOf(Unknown Source)
    at java.util.ArrayList.grow(Unknown Source)
    at java.util.ArrayList.ensureExplicitCapacity(Unknown Source)
    at java.util.ArrayList.ensureCapacityInternal(Unknown Source)
    at java.util.ArrayList.add(Unknown Source)
    at org.antlr.runtime.BufferedTokenStream.fetch(BufferedTokenStream.java:146)
    at org.antlr.runtime.BufferedTokenStream.sync(BufferedTokenStream.java:137)
    at org.antlr.runtime.CommonTokenStream.consume(CommonTokenStream.java:68)
    at org.antlr.runtime.BaseRecognizer.match(BaseRecognizer.java:106)
    at com.javadude.antlr.tutorial.GrammarOSBParser.string(GrammarOSBParser.java:1269)
    at com.javadude.antlr.tutorial.GrammarOSBParser.random_messageText(GrammarOSBParser.java:350)
    at com.javadude.antlr.tutorial.GrammarOSBParser.messageTextTag(GrammarOSBParser.java:219)
    at com.javadude.antlr.tutorial.GrammarOSBParser.log(GrammarOSBParser.java:173)
    at com.javadude.antlr.tutorial.GrammarOSBParser.logs(GrammarOSBParser.java:111)
    at com.javadude.antlr.tutorial.parser.main(parser.java:91)
    ... 5 more

`

Answer 1

看起来缓冲内存中的所有令牌都不会很好。所以你应该寻找一个没有做缓冲的解决方案。 ANTLR4带有一个UnbufferedTokenStream类，适合你。另请参阅this question以了解如何使用此类流以及它具有哪些缺点（Sam Harwell + Terence Parr贡献的地方）。

java.lang.OutOfMemoryError：Java堆空间使用ANTLR解析大文件时出现异常

1 个答案: