使用Tree Walker和布尔检查+捕获整个表达式

时间：2013-03-27 06:01:36

标签： antlr antlr3

我实际上有两个问题，我希望可以回答这些问题，因为它们完全取决于我的工作。下面是语法+树语法+ Java测试文件。

我实际想要实现的目标如下：

问题1：

我的语法正确地解析了我的语言。我想对变量声明进行一些语义检查。所以我创造了一个树木行走者，到目前为止它已经半成品了。我的问题是它没有捕获整个表达式。例如，

float x = 10 + 10;

它只捕获第一部分，即10.我不确定我做错了什么。如果我一次性完成，它就有效。不知何故，如果我将工作分成语法和树语法，它就不会捕获整个字符串。

问题2：

我想检查规则，如果我的条件返回true，我想删除该子树。例如，

float x = 10;
float x; // <================ I would like this to be removed.

我尝试过使用重写规则，但我认为它比这更复杂。

Test.g： 

grammar Test;
options {
  language = Java;
  output = AST;
}

parse : varDeclare+
      ;

varDeclare : type id equalExp? ';'
           ;

equalExp : ('=' (expression | '...'))
         ;

expression : binaryExpression
           ;

binaryExpression : addingExpression (('=='|'!='|'<='|'>='|'>'|'<') addingExpression)*
                 ;

addingExpression : multiplyingExpression (('+'|'-') multiplyingExpression)*
                 ;

multiplyingExpression : unaryExpression 
                        (('*'|'/') unaryExpression)*
                      ;

unaryExpression: ('!'|'-')* primitiveElement;   

primitiveElement : literalExpression
                 | id
                 | '(' expression ')'
                 ;  

literalExpression : INT
                  ;              

id : IDENTIFIER
   ;

type : 'int'    
     | 'float'
     ; 

// L E X I C A L   R U L E S      

INT : DIGITS ;   

IDENTIFIER : LETTER (LETTER | DIGIT)*;

WS  :   ( ' '
        | '\t'
        | '\r'
        | '\n'
        ) {$channel=HIDDEN;}
    ;

fragment LETTER : ('a'..'z' | 'A'..'Z' | '_') ;
fragment DIGITS: DIGIT+;
fragment DIGIT : '0'..'9';

TestTree.g： 

tree grammar TestTree;

options {
  language = Java;
  tokenVocab = Test;
  ASTLabelType = CommonTree;
}

@members {
    SemanticCheck s;

    public TestTree(TreeNodeStream input, SemanticCheck s) {
        this(input);
        this.s = s;
    }

}

parse[SemanticCheck s]
      : varDeclare+
      ;

varDeclare : type id equalExp? ';'
           {s.check($type.name, $id.text, $equalExp.expr);}
           ;

equalExp returns [String expr]
         : ('=' (expression {$expr = $expression.e;} | '...' {$expr = "...";}))
         ;

expression returns [String e]
@after {$e = $expression.text;}
           : binaryExpression
           ;

binaryExpression : addingExpression (('=='|'!='|'<='|'>='|'>'|'<') addingExpression)*
                 ;

addingExpression : multiplyingExpression (('+'|'-') multiplyingExpression)*
                 ;

multiplyingExpression : unaryExpression 
                        (('*'|'/') unaryExpression)*
                      ;

unaryExpression: ('!'|'-')* primitiveElement;   

primitiveElement : literalExpression
                 | id
                 | '(' expression ')'
                 ;  

literalExpression : INT
                  ;              

id : IDENTIFIER
   ;

type returns [String name]
@after { $name = $type.text; }
     : 'int'    
     | 'float'
     ;

Java测试文件， Test.java： 

import java.util.ArrayList;
import java.util.List;

import org.antlr.runtime.ANTLRStringStream;
import org.antlr.runtime.CommonTokenStream;
import org.antlr.runtime.RuleReturnScope;
import org.antlr.runtime.tree.CommonTree;
import org.antlr.runtime.tree.CommonTreeNodeStream;

public class Test {
    public static void main(String[] args) throws Exception {
        SemanticCheck s = new SemanticCheck();

        String src = 
                "float x = 10+y; \n" + 
                "float x; \n";
        TestLexer lexer = new TestLexer(new ANTLRStringStream(src));
        //TestLexer lexer = new TestLexer(new ANTLRFileStream("input.txt"));
        CommonTokenStream tokenStream = new CommonTokenStream(lexer);
        TestParser parser = new TestParser(tokenStream);
        RuleReturnScope r = parser.parse();

        System.out.println("Parse Tree:\n" + tokenStream.toString());

        CommonTree t = (CommonTree)r.getTree();
        CommonTreeNodeStream nodes = new CommonTreeNodeStream(t);
        nodes.setTokenStream(tokenStream);

        TestTree walker = new TestTree(nodes, s);
        walker.parse(s);

    }
}

class SemanticCheck {
    List<String> names;

    public SemanticCheck() {
        this.names = new ArrayList<String>();

    }

    public boolean check(String type, String variableName, String exp) {
        System.out.println("Type: " + type + "  variableName: " + variableName + "  exp: " + exp);
        if(names.contains(variableName)) {
            System.out.println("Remove statement! Already defined!");
            return true;
        }

        names.add(variableName);
        return false;
    }
}

提前致谢！

1 个答案:

答案 0 :(得分：0)

我弄清楚了我的问题，事实证明我需要先做一个AST才能做任何事情。这将有助于理解什么是扁平树看起来像建立AST。

How to output the AST built using ANTLR?

感谢StackOverFlow中的Bart's无尽的例子，我能够做语义谓词来完成上面例子中我需要的。

以下是更新的代码：

<强> Test.g 

 grammar Test;
    options {
      language = Java;
      output = AST;
    }
    tokens {
    VARDECL;
    Assign = '=';
    EqT    = '==';  
    NEq    = '!=';
    LT     = '<';
    LTEq   = '<=';
    GT     = '>';
    GTEq   = '>=';
    NOT    = '!';
    PLUS   = '+';
    MINUS  = '-';
    MULT   = '*';
    DIV    = '/';
    }

    parse : varDeclare+
          ;

    varDeclare : type id equalExp ';' -> ^(VARDECL type id equalExp)
               ;

    equalExp : (Assign^ (expression | '...' )) 
             ;

    expression : binaryExpression
               ;

    binaryExpression : addingExpression ((EqT|NEq|LTEq|GTEq|LT|GT)^ addingExpression)*
                     ;

    addingExpression : multiplyingExpression ((PLUS|MINUS)^ multiplyingExpression)*
                     ;

    multiplyingExpression : unaryExpression 
                            ((MULT|DIV)^ unaryExpression)*
                          ;

    unaryExpression: ((NOT|MINUS))^ primitiveElement
                   | primitiveElement
                   ;   

    primitiveElement : literalExpression
                     | id
                     | '(' expression ')' -> expression
                     ;  

    literalExpression : INT
                      ;              

    id : IDENTIFIER
       ;

    type : 'int'    
         | 'float'
         ; 

    // L E X I C A L   R U L E S      

    INT : DIGITS ;   

    IDENTIFIER : LETTER (LETTER | DIGIT)*;

    WS  :   ( ' '
            | '\t'
            | '\r'
            | '\n'
            ) {$channel=HIDDEN;}
        ;

    fragment LETTER : ('a'..'z' | 'A'..'Z' | '_') ;
    fragment DIGITS: DIGIT+;
    fragment DIGIT : '0'..'9';

每当你有varDeclare时，这应该会自动构建一个AST。现在进入树语法/步行者。

<强> TestTree.g 

tree grammar TestTree;

options {
  language = Java;
  tokenVocab = Test;
  ASTLabelType = CommonTree;
  output = AST;
}

tokens {
    REMOVED;
}

@members {
    SemanticCheck s;

    public TestTree(TreeNodeStream input, SemanticCheck s) {
        this(input);
        this.s = s;
    }

}

start[SemanticCheck s] : varDeclare+
      ;

varDeclare : ^(VARDECL type id equalExp)
            -> {s.check($type.text, $id.text, $equalExp.text)}? REMOVED
            -> ^(VARDECL type id equalExp)  
           ;

equalExp : ^(Assign expression)
         | ^(Assign '...')
         ;

expression : ^(('!') expression)
        | ^(('+'|'-'|'*'|'/') expression expression*)
        | ^(('=='|'<='|'<'|'>='|'>'|'!=') expression expression*)
        | literalExpression
       ;                                  

literalExpression : INT
                  | id
                  ;  

id : IDENTIFIER
   ;

type : 'int'    
     | 'float'
     ;

现在开始测试它：

<强> Test.java 

import java.util.ArrayList;
import java.util.List;

import org.antlr.runtime.ANTLRStringStream;
import org.antlr.runtime.CommonTokenStream;
import org.antlr.runtime.tree.*;

public class Test {
    public static void main(String[] args) throws Exception {
        SemanticCheck s = new SemanticCheck();

        String src = 
                "float x = 10; \n" + 
                "int x = 1; \n";
        TestLexer lexer = new TestLexer(new ANTLRStringStream(src));
        CommonTokenStream tokenStream = new CommonTokenStream(lexer);
        TestParser parser = new TestParser(tokenStream);
        TestParser.parse_return r = parser.parse();
        System.out.println("Tree:" + ((Tree)r.tree).toStringTree() + "\n");

        CommonTreeNodeStream nodes = new CommonTreeNodeStream((Tree)r.tree);
        nodes.setTokenStream(tokenStream);
        TestTree walker = new TestTree(nodes, s);
        TestTree.start_return r2 = walker.start(s);
        System.out.println("\nTree Walker: "+((Tree)r2.tree).toStringTree());

    }
}

class SemanticCheck {
    List<String> names;

    public SemanticCheck() {
        this.names = new ArrayList<String>();

    }

    public boolean check(String type, String variableName, String exp) {
        System.out.println("Type: " + type + "  variableName: " + variableName + "  exp: " + exp);
        if(names.contains(variableName)) {
            return true;
        }

        names.add(variableName);
        return false;
    }
}

<强>输出： 

Tree:(VARDECL float x (= 10)) (VARDECL int x (= 1))

Type: float  variableName: x  exp: = 10
Type: int  variableName: x  exp: = 1

Tree Walker: (VARDECL float x (= 10)) REMOVED

希望这有帮助！如果我做错了，请随意指出任何错误。

相关问题
最新问题