如何操纵树语法

时间:2012-10-30 13:49:23

标签: java dsl antlr3 parser-generator antlrworks

PARSER GRAMMAR
protocol.g

grammar protocol; 

options {
  language = Java;
  output = AST;
  ASTLabelType=CommonTree;
}

tokens{ 
TRANSITIONS;
PAIR;
}

@header {
package com.javadude.antlr3.x.tutorial;
}

@lexer::header {
  package com.javadude.antlr3.x.tutorial;
}

parse
 : transitions EOF!
   {
     CommonTree root = $transitions.tree;

     int count = root.getChildCount();

     Tree child1 = root.getChild(0);
     Tree child2 = root.getChild(1);
     Tree child3 = root.getChild(2);
     Tree child4 = root.getChild(3);

     System.out.println("root=" + root.getToken().getText() + " has " + count + " child nodes:");
     System.out.println(" - child1=" + child1.toStringTree());
     System.out.println(" - child2=" + child2.toStringTree());
     System.out.println(" - child3=" + child3.toStringTree());
     System.out.println(" - child4=" + child4.toStringTree());
   }
 ;
transitions
 : 'transitions' '=' INT pair+ ';' -> ^(TRANSITIONS INT pair+)
 ;
pair
 : '(' INT ',' INT ')' -> ^(PAIR INT INT)
 ;

INT 
    : ('0'..'9')+;
WHITESPACE
    : ('\t' | ' ' | '\r' | '\n' | '\u000C')+ {$channel = HIDDEN;};

TREE GRAMMAR
protocolWalker.g

tree grammar protocolWalker;

options {
  language = Java;
  tokenVocab = protocol;   
  ASTLabelType = CommonTree;
}


@header {
package com.javadude.antlr3.x.tutorial;
}

transitions
 : ^(TRANSITIONS INT pair+) 
 {
 System.out.println("transitions=" + $INT.text);
 }
 ;

pair
 : ^(PAIR a=INT b=INT) 
 {
 System.out.println("pair=" + $a.text + ", " + $b.text);

 }
 ;

JAVA TEST RIG
Protocoltest.java

package com.javadude.antlr3.x.tutorial;
import org.antlr.runtime.*;
import org.antlr.runtime.tree.CommonTree;
import org.antlr.runtime.tree.CommonTreeNodeStream;
public class Protocoltest {

    /**
     * @param args
     */
    public static void main(String[] args) throws Exception {
        //create input stream from standard input
        ANTLRInputStream input = new ANTLRInputStream(System.in);
        //create a lexer attached to that input stream
        protocolLexer lexer = new protocolLexer(input);
        //create a stream of tokens pulled from the lexer
        CommonTokenStream tokens = new CommonTokenStream(lexer);

        //create a parser attached to teh token stream
        protocolParser parser = new protocolParser(tokens);
        //invoke the program rule in get return value
        protocolParser.parse_return r =parser.parse();

        CommonTree t = (CommonTree)r.getTree();
        //output the extracted tree to the console
        System.out.println("\nAST is: " + t.toStringTree());

        //walk resulting tree; create treenode stream first
        CommonTreeNodeStream nodes = new CommonTreeNodeStream(t);
        //AST nodes have payloads that point into token stream
        nodes.setTokenStream(tokens);

        //create a tree walker attached to the nodes stream
        protocolWalker walker = new protocolWalker(nodes);

        //invoke the start symbol, rule parse
        walker.transitions();
        }
}

INPUT

transitions = 3(5,0) (5,1) (5,2);

输出

root=TRANSITIONS has 4 child nodes:
 - child1=3
 - child2=(PAIR 5 0)
 - child3=(PAIR 5 1)
 - child4=(PAIR 5 2)

AST is: (TRANSITIONS 3 (PAIR 5 0) (PAIR 5 1) (PAIR 5 2))
pair=5, 0
pair=5, 1
pair=5, 2
transitions=3  

问题
你可以在上面看到,在解析器语法(protocol.g)中我可以将转换根的所有子节点存储为child1,child2,child3和child4。另外,我打印过这些。   在树语法中,我如何存储这些并可以对它们执行操作? 谢谢

1 个答案:

答案 0 :(得分:1)

  

我将实例化java类(将创建java对象),例如,树中的第一个数字将决定将创建多少个对象,然后,PAIR 5 0将创建一个具有2个参数的对象(5,0) ,PAIR 5 1将创建具有2个参数(5,1)的第2个对象,PAIR 5 2将创建具有2个参数(5,2)的第3个对象。

这是一种创建过渡并为其添加对的简单方法,它只需要对protocolWalker.g进行少量更改。不过,首先,我将使用虚拟TransitionsPair类:

Transitions.java

import java.util.ArrayList;


public class Transitions {
    private ArrayList<Pair> pairs = new ArrayList<Pair>();

    public void addPair(Pair pair){
        System.out.println(String.format("Added pair %s to transitions", pair));
        pairs.add(pair);
    }

    @Override
    public String toString() {
        return "Pairs: " + pairs;
    }
}

Pair.java

public class Pair {
    private int a;
    private int b;

    public Pair(int a, int b){
        this.a = a;
        this.b = b;
    }

    @Override
    public String toString() {
        return String.format("(%d, %d)", a, b);
    }
}

以下是修改后的protocolWalker.g

protocolWalker.g(已修改)

tree grammar protocolWalker;

options {
  language = Java;
  tokenVocab = protocol;   
  ASTLabelType = CommonTree;
}    


@header {
    package com.javadude.antlr3.x.tutorial;
    import java.util.List;
    import java.util.ArrayList;
}

@members { 
  //stores all the transitions objects as they get processed
  private ArrayList<Transitions> allTransitions = new ArrayList<Transitions>();

  //returns all the transitions
  public List<Transitions> getAllTransitions() { 
    return allTransitions;
  }
}


transitions
@init { 
        //create a Transitions object when the rule is hit
        Transitions transitions = new Transitions();

        //store it to be accessed later.
        allTransitions.add(transitions);
      } 
 : ^(TRANSITIONS INT transitions_pair[transitions]+) //pass the object to transitions_pair for each PAIR encountered
 {
     System.out.println("transitions=" + $INT.text);
 }
 ;

transitions_pair[Transitions transitions]
 : ^(PAIR a=INT b=INT) 
 {
     System.out.println("pair=" + $a.text + ", " + $b.text);
     //make a call to the Transitions object that was passed to this rule.
     transitions.addPair(new Pair($a.int, $b.int));
 }
 ;

(我将pair重命名为transitions_pair,因为该规则现在与转换构建相关联。)规则transitions调用transitions_pair,传递新的Transitions }对象同时。 transitions_pair会向收到的Pair对象添加新的Transitions对象。

可以使用[ArgType argname,...]方式编写树解析器和令牌解析器中的规则以接受对象。在这种情况下,它可以更轻松地访问子PAIR树。

我在Protocoltest.java添加了一个小的更改,以打印出存储的转换:

        ...
        //invoke the start symbol, rule parse
        walker.transitions();

        //get the stored transitions and print them out.            
        List<Transitions> transitions = walker.getAllTransitions();
        System.out.println(transitions);
        ...

以下是助行器的新输出:

pair=5, 0
Added pair (5, 0) to transitions
pair=5, 1
Added pair (5, 1) to transitions
pair=5, 2
Added pair (5, 2) to transitions
transitions=3
[Pairs: [(5, 0), (5, 1), (5, 2)]]

以下是我所做的重大改变的回顾:

  • 添加了一种存储和返回walker转换的方法。
  • 添加了在规则Transitions中创建transitions对象的代码。
  • 添加了将对象传递给transitions_pair的代码。
  • 在测试仪中添加了代码,以便从助行器中检索过渡并将其打印出来。

我认为,一旦您实施了自己的Transitions课程,您就会全部投入使用。