ANTLR词法分析器不匹配令牌

时间:2010-04-09 06:18:24

标签: compiler-construction parsing antlr tokenize lexer

我有一个简单的ANTLR语法,我已经删除了它的基本要点来证明我遇到的这个问题。我使用的是ANTLRworks 1.3.1。

grammar sample;

assignment  :  IDENT ':=' NUM ';' ;

IDENT       :  ('a'..'z')+ ;

NUM         :  ('0'..'9')+ ;

WS          :  (' '|'\n'|'\t'|'\r')+ {$channel=HIDDEN;} ;

显然,语法接受了这个陈述:

x := 99;

但这个也是:

x := @!$()()%99***;

ANTLRworks Interpreter的输出:

ANTLR Interpreter diagram http://cs.sierracollege.edu/~barry/antlr-lexer.png

我做错了什么?即使是ANTLR附带的其他示例语法(例如CMinus语法)也会出现这种行为。

1 个答案:

答案 0 :(得分:2)

如果您查看ANTLRWorks IDE的控制台,您会看到很多词法错误。

在命令行上尝试:

grammar Sample;

@members {
  public static void main(String[] args) throws Exception {
    ANTLRStringStream in = new ANTLRStringStream("x := @!$()()\%99***;");
    SampleLexer lexer = new SampleLexer(in);
    CommonTokenStream tokens = new CommonTokenStream(lexer);
    SampleParser parser = new SampleParser(tokens);
    parser.assignment();
  }
}

assignment  :  IDENT ':=' NUM ';' ;

IDENT       :  ('a'..'z')+ ;

NUM         :  ('0'..'9')+ ;

WS          :  (' '|'\n'|'\t'|'\r')+ {$channel=HIDDEN;} ;

然后:

// generate parser/lexer
java -cp antlr-3.2.jar org.antlr.Tool Sample.g

// compile
javac -cp antlr-3.2.jar *.java

// run Windows
java -cp .;antlr-3.2.jar SampleParser
// or run *nix/MacOS
java -cp .:antlr-3.2.jar SampleParser

将产生:

line 1:5 no viable alternative at character '@'
line 1:6 no viable alternative at character '!'
line 1:7 no viable alternative at character '$'
line 1:8 no viable alternative at character '('
line 1:9 no viable alternative at character ')'
line 1:10 no viable alternative at character '('
line 1:11 no viable alternative at character ')'
line 1:12 no viable alternative at character '%'
line 1:15 no viable alternative at character '*'
line 1:16 no viable alternative at character '*'
line 1:17 no viable alternative at character '*'