javacc bug ??它给了我一个我从未在输入中写过的标记

时间:2015-08-12 18:53:48

标签: compiler-construction token javacc

我将此输入字符串提供给基于javacc的解析器:

"  SELECT  a\n.run_id,\nx.*"

但它给了我一个例外:

net.sf.jsqlparser.parser.ParseException: Encountered " ";" "; "" at line 3, column 1.
Was expecting one of:
    "*" ...
    "*" ...

    at net.sf.jsqlparser.parser.CCJSqlParser.generateParseException(CCJSqlParser.java:14079)
    at net.sf.jsqlparser.parser.CCJSqlParser.jj_consume_token(CCJSqlParser.java:13929)
    at net.sf.jsqlparser.parser.CCJSqlParser.SelectItem(CCJSqlParser.java:2092)
    at net.sf.jsqlparser.parser.CCJSqlParser.SelectItemsList(CCJSqlParser.java:2017)
    at net.sf.jsqlparser.parser.CCJSqlParser.PlainSelect(CCJSqlParser.java:1676)
    at net.sf.jsqlparser.parser.CCJSqlParser.SelectBody(CCJSqlParser.java:1574)
    at net.sf.jsqlparser.parser.CCJSqlParser.Select(CCJSqlParser.java:1550)
    at net.sf.jsqlparser.parser.CCJSqlParser.SingleStatement(CCJSqlParser.java:98)
    at net.sf.jsqlparser.parser.CCJSqlParser.Statements(CCJSqlParser.java:186)
    at com.paypal.dt.roe.parser.SqlConvertor.convert(SqlConvertor.java:286)
    at com.paypal.dt.roe.parser.SqlConvertorTest.test1(SqlConvertorTest.java:247)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
    at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
    at org.springframework.test.context.junit4.statements.RunBeforeTestMethodCallbacks.evaluate(RunBeforeTestMethodCallbacks.java:74)
    at org.springframework.test.context.junit4.statements.RunAfterTestMethodCallbacks.evaluate(RunAfterTestMethodCallbacks.java:83)
    at org.springframework.test.context.junit4.statements.SpringRepeat.evaluate(SpringRepeat.java:72)
    at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:231)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
    at org.springframework.test.context.junit4.statements.RunBeforeTestClassCallbacks.evaluate(RunBeforeTestClassCallbacks.java:61)
    at org.springframework.test.context.junit4.statements.RunAfterTestClassCallbacks.evaluate(RunAfterTestClassCallbacks.java:71)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
    at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.run(SpringJUnit4ClassRunner.java:174)
    at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
    at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:459)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:675)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:382)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:192)

这真是令人惊讶,因为我没有写“;”真的很棒!

实际上,如果我删除所有“\ n”或进行其他格式调整,则错误消失了。

我在javacc中启用了tokenmanager登录,这里是日志记录中的相关行:

$ grep -B 4 FOUND /tmp/a
   Starting NFA to match one of : { <S_IDENTIFIER> }
Current character : 32 (32) at line 1 column 9
   Currently matched the first 6 characters as a "SELECT" token.
   Putting back 1 characters into the input stream.
****** FOUND A "SELECT" MATCH (SELECT) ******
--
   Starting NFA to match one of : { <S_IDENTIFIER> }
Current character : 10 (10) at line 1 column 12
   Currently matched the first 1 characters as a <S_IDENTIFIER> token.
   Putting back 1 characters into the input stream.
****** FOUND A <S_IDENTIFIER> MATCH (a) ******
--
   Starting NFA to match one of : { <S_DOUBLE> }
Current character : 114 (114) at line 2 column 2
   Currently matched the first 1 characters as a "." token.
   Putting back 1 characters into the input stream.
****** FOUND A "." MATCH (.) ******
--
   Possible kinds of longer matches : { <K_TIMEUNIT>, <S_IDENTIFIER> }
Current character : 44 (44) at line 2 column 8
   Currently matched the first 6 characters as a <S_IDENTIFIER> token.
   Putting back 1 characters into the input stream.
****** FOUND A <S_IDENTIFIER> MATCH (run_id) ******
--
                    Return: RelObjectName(LOOKAHEAD SUCCEEDED)
Current character : 44 (44) at line 2 column 8
   No more string literal token matches are possible.
   Currently matched the first 1 characters as a "," token.
****** FOUND A "," MATCH (,) ******
--
Skipping character : 10 (10)
Current character : 59 (59) at line 3 column 1
   No more string literal token matches are possible.
   Currently matched the first 1 characters as a ";" token.
****** FOUND A ";" MATCH (;) ******
--
   Starting NFA to match one of : { <S_IDENTIFIER> }
Current character : 46 (46) at line 3 column 3
   Currently matched the first 1 characters as a <S_IDENTIFIER> token.
   Putting back 1 characters into the input stream.
****** FOUND A <S_IDENTIFIER> MATCH (x) ******

0 个答案:

没有答案