Question

我希望能够记录Antlr v3（准确地说是3.0.1）生成语法的解析过程。我试图使用DebugTreeParser，但它什么也没做，看起来它的方法永远不会被调用。

理想情况下，我希望能够输出如下内容，一系列经过尝试/执行的规则：

 parsing: program (token: Foo)
 parsing: statements (token: Foo)
 parsing: statement (token: Foo)
 parsing: block (token: Foo)
 parsed: block -> false (at 0)
 parsing: method call (token: Foo)
 parsing: variable (token: Foo)
 parsed: variable -> true (at 1)
 ...

这是我的解析代码：

        CharStream cs = new ANTLRReaderStream(script);
        MyLexer lex = new MyLexer(cs);
        CommonTokenStream tokens = new CommonTokenStream(lex);
        MyParser parser = new MyParser(tokens);
        return new Program(makeProgram((Tree) parser.program().getTree()));

我尝试了在Antlr Wiki上找到的解决方案：

 ...
 ParseTreeBuilder builder = new ParseTreeBuilder("prog");

 MyParser parser = new MyParser(tokens);
 parser.setTreeAdaptor(new DebugTreeAdaptor(builder, parser.getTreeAdaptor()));

但是建造者没有输出任何有趣的东西。

也许有一个选项可以在源语法中激活以生成与调试兼容的解析器？

Answer 1

首先，使用-debug命令行选项生成语法。完成此操作后，您的令牌解析器将具有额外的，以调试为中心的构造函数，允许您使用自定义DebugEventListener或内置的DebugEventListener。由于您要进行自定义日志记录，因此以下是使用自定义grammar DebugMe; compilationUnit : statements EOF; statements : statement+; statement : block | call | decl; block : LCUR statements RCUR; call : ID LPAR arglist? RPAR SEMI; arglist : ID (COMMA ID)*; decl : VAR ID EQ expr SEMI; expr : add_expr; add_expr : primary_expr ((PLUS|MINUS) primary_expr)*; primary_expr : STRING | ID | INT | LPAR expr RPAR; VAR: 'var'; ID: ('a'..'z'|'A'..'Z')+; INT: ('0'..'9')+; STRING: '"' ~('\r'|'\n'|'"')* '"'; SEMI: ';'; LPAR: '('; RPAR: ')'; LCUR: '{'; RCUR: '}'; PLUS: '+'; MINUS: '-'; COMMA: ','; EQ: '='; WS: (' '|'\t'|'\f'|'\r'|'\n') {skip();};启动的示例解决方案。

这是我用于测试的语法。它可能包含问题。

DebugMe.g

newEventListener

这是我将要使用的测试程序。请注意，我省略了public class TestDebugMeGrammar { public static void main(String[] args) throws Exception { CharStream input = new ANTLRStringStream("var x = 3; print(x);"); DebugMeLexer lexer = new DebugMeLexer(input); CommonTokenStream tokens = new CommonTokenStream(lexer); DebugMeParser parser = new DebugMeParser(tokens, newEventListener()); parser.compilationUnit(); } //... }的实现。

TestDebugMeGrammar.java

DebugEventListener

我并不过分熟悉解析器调用Proxy的方式，因此我将从一个简单的//TestDebugMeGrammar.java private static DebugEventListener newEventListener() { return (DebugEventListener) Proxy.newProxyInstance(TestDebugMeGrammar.class.getClassLoader(), new Class[] { DebugEventListener.class }, new DebugListenerHandler()); } public static class DebugListenerHandler implements InvocationHandler { @Override public Object invoke(Object proxy, Method method, Object[] args) throws Throwable { // simply print out the method call. System.out.print(method.getName()); if (args != null && args.length > 0) { System.out.print(": "); for (int i = 0, count = args.length; i < count; ++i) { Object arg = args[i]; if (arg == null) { System.out.printf("<(null)> "); } else { System.out.printf("<%s> ", arg.toString()); } } } System.out.println(); return null; } }实现开始，以最小的数量转储每个调用麻烦：

enterRule: <DebugMe.g> <compilationUnit> 
commence
location: <4> <1> 
enterAlt: <1> 
location: <5> <7> 
enterRule: <DebugMe.g> <statements> 
location: <7> <1> 
enterAlt: <1> 
location: <8> <7> 
enterSubRule: <1> 
enterDecision: <1> <false> 
LT: <1> <[@0,0:2='var',<11>,1:0]> 
exitDecision: <1> 
enterAlt: <1> 
location: <8> <7> 
enterRule: <DebugMe.g> <statement> 
location: <10> <1> 
enterDecision: <2> <false> 
LT: <1> <[@0,0:2='var',<11>,1:0]> 
exitDecision: <2> 
enterAlt: <3> 
location: <13> <7> 
enterRule: <DebugMe.g> <decl> 
...

输出很广泛，但它可以很好地了解听众听到的内容。

//TestDebugMeGrammar.g
    //redefinition
    private static DebugEventListener newEventListener() {
        return new SimpleDebugEventListener();
    }

    private static class SimpleDebugEventListener extends
            BlankDebugEventListener {

        private Token lastToken;
        @Override
        public void LT(int i, Object t) {
            System.out.println("Read object \"" + t + "\"");
        }

        @Override
        public void LT(int i, Token t) {
            if (!t.equals(lastToken)){
                System.out.println("Read input \"" + t.getText() + "\"");
                lastToken = t;
            }
        }

        @Override
        //public void enterRule(String ruleName) { // <-- ANTLR 3.0.1
        public void enterRule(String grammarFileName, String ruleName) { //<-- ANTLR 3.4
            System.out.println("Entered rule " + ruleName);
        }

        @Override
        //public void exitRule(String ruleName) { // <-- ANTLR 3.0.1
        public void exitRule(String grammarFileName, String ruleName) { //<-- ANTLR 3.4
            System.out.println("Exited rule " + ruleName);
        }

        @Override
        public void consumeToken(Token token) {
            System.out.println("Consumed \"" + token.getText() + "\"");
        }
    }

根据我从上面收集的内容，这里有一个小而专注的听众。输出更接近你想要的，可以作为一个有用的起点。

Entered rule compilationUnit
Entered rule statements
Read input "var"
Entered rule statement
Entered rule decl
Consumed "var"
Read input "x"
Consumed "x"
Read input "="
Consumed "="
Entered rule expr
Entered rule add_expr
Entered rule primary_expr
Read input "3"
Consumed "3"
Exited rule primary_expr
Read input ";"
Exited rule add_expr
Exited rule expr
Consumed ";"
Exited rule decl
Exited rule statement
Read input "print"
Entered rule statement
Entered rule call
Consumed "print"
Read input "("
Consumed "("
Read input "x"
Entered rule arglist
Consumed "x"
Read input ")"
Exited rule arglist
Consumed ")"
Read input ";"
Consumed ";"
Exited rule call
Exited rule statement
Read input "<EOF>"
Exited rule statements
Consumed "<EOF>"
Exited rule compilationUnit

这是输出：

SimpleDebugEventListener

我最初使用ANTLR 3.4测试并运行了上述代码。我根据您的规范对ANTLR 3.0.1进行了重新测试，并且您需要进行此项工作的唯一更改是SimpleDebugEventListener类。我已更新代码以指示需要更改的位置以及更改的内容。

只是为了好玩，这里有一个修改后的private static class SimpleDebugEventListener extends BlankDebugEventListener { private LinkedList<String> activeRules = new LinkedList<String>(); @Override public void enterRule(String grammar, String ruleName) { //ANTLR 3.4 activeRules.add(ruleName); } @Override public void exitRule(String grammar, String ruleName) { //ANTLR 3.4 activeRules.removeLast(); } @Override public void consumeToken(Token token) { System.out.printf("%s consumed \"%s\"%n", formatRules(), token.getText()); } private String formatRules() { if (activeRules.size() == 1) { return activeRules.getLast(); } else { StringBuilder builder = new StringBuilder(); boolean first = true; for (String rule : activeRules){ if (!first){ builder.append(" -> "); } else { first = false; } builder.append(rule); } return builder.toString(); } } }打印输出，我认为这些输出与您的记录目标更相似。

compilationUnit -> statements -> statement -> decl consumed "var"
compilationUnit -> statements -> statement -> decl consumed "x"
compilationUnit -> statements -> statement -> decl consumed "="
compilationUnit -> statements -> statement -> decl -> expr -> add_expr -> primary_expr consumed "3"
compilationUnit -> statements -> statement -> decl consumed ";"
compilationUnit -> statements -> statement -> call consumed "print"
compilationUnit -> statements -> statement -> call consumed "("
compilationUnit -> statements -> statement -> call -> arglist consumed "x"
compilationUnit -> statements -> statement -> call consumed ")"
compilationUnit -> statements -> statement -> call consumed ";"
compilationUnit consumed "<EOF>"

输出：

{{1}}

记录antlr v3解析器活动

1 个答案:

DebugMe.g

TestDebugMeGrammar.java