我正在使用野牛为玩具编程语言编写一个解析器,但我已经碰壁了:
我的grammar.y
文件如下:
%{
#include <stdio.h>
#include "util.h"
#include "errormsg.h"
#define YYDEBUG 1
int yylex(void); /* function prototype */
void yyerror(char *s)
{
EM_error(EM_tokPos, "%s", s);
}
%}
%union {
int pos;
int ival;
string sval;
}
%token <sval> TK_ID TK_STRING
%token <ival> TK_INT
%token <pos>
TK_COMMA TK_COLON TK_SEMICOLON TK_LPAREN TK_RPAREN TK_LBRACK TK_RBRACK
TK_LBRACE TK_RBRACE TK_DOT TK_ASSIGN
TK_ARRAY TK_IF TK_THEN TK_ELSE TK_WHILE TK_FOR TK_TO TK_DO TK_LET TK_IN
TK_END TK_OF TK_BREAK TK_NIL
TK_FUNCTION TK_VAR TK_TYPE
/* Precedence in Bison is weird: lower is higher. Take a look at the spec too. */
%left <pos> TK_OR
%left <pos> TK_AND
%nonassoc <pos> TK_EQ TK_NEQ TK_LT TK_LE TK_GT TK_GE
%left <pos> TK_PLUS TK_MINUS
%left <pos> TK_TIMES TK_DIVIDE
%left <pos> TK_UMINUS
%error-verbose
%start program
%%
/* According to the spec, Tiger programs are just an expression exp. */
program: exp
/* An expression can be many things; consult the spec for more info: Expressions. */
/* For the %prec rule, take a look at 5.4 Context-Dependent Precedence on bison manual */
exp:
lvalue
| TK_NIL
| exp exp_seq_aug
| TK_LPAREN TK_RPAREN
| TK_LET TK_IN TK_END
| TK_INT
| TK_STRING
| TK_MINUS exp %prec TK_UMINUS
| TK_ID TK_LPAREN TK_RPAREN
| TK_ID TK_LPAREN exp params TK_RPAREN
| exp TK_PLUS exp
| exp TK_MINUS exp
| exp TK_TIMES exp
| exp TK_DIVIDE exp
| exp TK_EQ exp
| exp TK_NEQ exp
| exp TK_GT exp
| exp TK_LT exp
| exp TK_GE exp
| exp TK_LE exp
| exp TK_AND exp
| exp TK_OR exp
| TK_ID TK_LBRACE TK_RBRACE
| TK_ID TK_LBRACE TK_ID TK_EQ exp record_exp TK_RBRACE
| TK_ID TK_LBRACK exp TK_RBRACK TK_OF exp
| lvalue TK_ASSIGN exp
| TK_IF exp TK_THEN exp TK_ELSE exp
| TK_IF exp TK_THEN exp
| TK_WHILE exp TK_DO exp
| TK_FOR TK_ID TK_ASSIGN exp TK_TO exp TK_DO exp
| TK_BREAK
| TK_LET decl_seq TK_IN exp_seq_aug TK_END
;
decl_seq:
/* empty */
| decl_seq decl
;
decl:
type_decl
| var_decl
| func_decl
;
var_decl:
TK_VAR TK_ID TK_ASSIGN exp
| TK_VAR TK_ID TK_COLON TK_ID TK_ASSIGN exp
;
func_decl:
TK_FUNCTION TK_ID TK_LPAREN type_fields TK_RPAREN TK_EQ exp
| TK_FUNCTION TK_ID TK_LPAREN type_fields TK_COLON TK_ID TK_EQ exp
;
type_decl:
TK_TYPE TK_ID TK_EQ type
;
type:
TK_TYPE
| TK_LBRACE type_fields TK_RBRACE
| TK_ARRAY TK_OF TK_ID
;
type_fields:
/* empty */
| TK_ID TK_COLON TK_ID type_fields
| TK_COMMA TK_ID TK_COLON TK_ID type_fields
;
lvalue:
TK_ID
| lvalue TK_DOT TK_ID
| lvalue TK_LBRACK exp TK_RBRACK
;
exp_seq:
/* epsilon */
| TK_SEMICOLON exp
| exp_seq TK_SEMICOLON exp
;
exp_seq_aug:
TK_LPAREN exp_seq TK_RPAREN
;
params:
/* epsilon */
| params TK_COMMA exp
;
record_exp:
/* epsilon */
| record_exp TK_COMMA TK_ID TK_EQ exp
;
这没什么太花哨的,并且很少(96)转移/减少冲突(我猜这很可能是由于if语句和函数调用语句)。我知道它应该没有必须清楚,但同一练习的其他替代实现干净地解析更多的移位/减少冲突,所以考虑到我也得到的错误消息,它应该没有太大关系。
令牌文件由bison从%token
指令(y.tab.h和y.tab.c)生成,我得到的具体错误消息是:
nlightnfotis@frodo ~/Software/tigerc $ ./a.out tests/test4.tig
tests/test4.tig:2.1: syntax error, unexpected TK_GE
Parsing failed
这非常令人沮丧,因为当测试文件没有时,解析器会发现它找到了一个更大或相等的标记:
/* define a recursive function */
let
/* calculate n! */
function nfactor(n: int): int =
if n = 0
then 1
else n * nfactor(n-1)
in
nfactor(10)
end
我怎么可能调试这个?
[编辑] :根据要求,这是我的flex
词法分析器的源代码:
%{
#include <string.h>
#include "util.h"
#include "tokens.h"
#include "errormsg.h"
int charPos = 1;
int
yywrap (void)
{
charPos = 1;
return 1;
}
// Adjust the token position in the string
// Mainly used for error checking
void
adjust (void)
{
EM_tokPos = charPos;
charPos += yyleng;
}
%}
/* Will be used for conditional activation of the comment rule. */
%x C_COMMENT
digits [0-9]+
letters [_a-zA-Z]+
%%
" " {adjust(); continue;}
\n {adjust(); EM_newline(); continue;}
\t {adjust(); continue;}
"/*" {adjust(); BEGIN(C_COMMENT);}
<C_COMMENT>[^*\n] {adjust();}
<C_COMMENT>"*/" {adjust(); BEGIN(INITIAL);}
\"(\\.|[^"])*\" {adjust(); yylval.sval = String(yytext); return STRING;}
"," {adjust(); return COMMA;}
";" {adjust(); return SEMICOLON;}
":" {adjust(); return COLON;}
"." {adjust(); return DOT;}
"+" {adjust(); return PLUS;}
"-" {adjust(); return MINUS;}
"*" {adjust(); return TIMES;}
"/" {adjust(); return DIVIDE;}
"=" {adjust(); return EQ;}
"<>" {adjust(); return NEQ;}
"<" {adjust(); return LT;}
"<=" {adjust(); return LE;}
">" {adjust(); return GT;}
">=" {adjust(); return GE;}
"&" {adjust(); return AND;}
"|" {adjust(); return OR;}
":=" {adjust(); return ASSIGN;}
"(" {adjust(); return LPAREN;}
")" {adjust(); return RPAREN;}
"{" {adjust(); return LBRACE;}
"}" {adjust(); return RBRACE;}
"[" {adjust(); return LBRACK;}
"]" {adjust(); return RBRACK;}
for {adjust(); return FOR;}
if {adjust(); return IF;}
then {adjust(); return THEN;}
else {adjust(); return ELSE;}
while {adjust(); return WHILE;}
to {adjust(); return TO;}
do {adjust(); return DO;}
let {adjust(); return LET;}
in {adjust(); return IN;}
end {adjust(); return END;}
of {adjust(); return OF;}
break {adjust(); return BREAK;}
nil {adjust(); return NIL;}
function {adjust(); return FUNCTION;}
var {adjust(); return VAR;}
type {adjust(); return TYPE;}
array {adjust(); return ARRAY;}
{digits} {adjust(); yylval.ival = atoi (yytext); return INT;}
{letters}[a-zA-Z0-9_]* {adjust(); yylval.sval = String (yytext); return ID;}
. {adjust(); EM_error (EM_tokPos,"illegal token");}
答案 0 :(得分:2)
How can I possibly debug this?
对于初学者,您需要学习使用Bison调试选项。这将输出所有状态的转储,并且被授予,需要花费大量的耐心和时间来调试它们,首先看你通常至少可以缩小导致问题的规则。
就你的问题而言,你的词法分子没有返回野牛定义的标记。
在Bison中,您有%token TK_GE
,但您的词法分析器会返回GE
。 Bison语法只知道TK_GE,这就是它所期望的。如果我记得的话,它会将标记定义为ASCII序列上方的数字递增序列,并且你必须在词法分析器中使用这些值。
除非你正在做一些我在tokens.h中看不到的重新定义,否则你需要重写lexer来做:
">=" {adjust(); return TK_GE;}
可能你在某个地方有#define GE 42
,但野牛正在生成一个#define TK_GE 21
的令牌文件(示例值)。