Question

我正在尝试使用java为Delphi创建一个词法分析器。这是示例代码：

String[] keywords={"array","as","asm","begin","case","class","const","constructor","destructor","dispinterface","div","do","downto","else","end","except","exports","file","finalization","finally","for","function","goto","if","implementation","inherited","initialization","inline","interface","is","label","library","mod","nil","object","of","out","packed","procedure","program","property","raise","record","repeat","resourcestring","set","shl","shr","string","then","threadvar","to","try","type","unit","until","uses","var","while","with"};
String[] relation={"=","<>","<",">","<=",">="};
String[] logical={"and","not","or","xor"};
Matcher matcher = null;
for(int i=0;i<keywords.length;i++){
   matcher=Pattern.compile(keywords[i]).matcher(line); 
   if(matcher.find()){
    System.out.println("Keyword"+"\t\t"+matcher.group());
   }
}
for(int i1=0;i1<logical.length;i1++){
   matcher=Pattern.compile(logical[i1]).matcher(line); 
   if(matcher.find()){
     System.out.println("logic_op"+"\t\t"+matcher.group());
   }
}    
for(int i2=0;i2<relation.length;i2++){
   matcher=Pattern.compile(relation[i2]).matcher(line); 
   if(matcher.find()){
     System.out.println("relational_op"+"\t\t"+matcher.group());
   }
}

所以，当我运行该程序时，它可以工作，但它会重新读取程序认为是2个标记的某些单词，例如：记录是一个关键字，但重新读取它以找到字符或表示来自 rec“或”d 的令牌逻辑运算符。如何取消重读单词？谢谢！

Answer 1

将\b添加到正则表达式中，以便在单词之间中断。所以：

Pattern.compile("\\b" + keywords[i] + "\\b")

将确保您单词两边的字符不是字母。

这种方式＆＃34;记录＆＃34;只会与＆＃34;记录相匹配，＆＃34;不是＆＃34;或。＆＃34;

Answer 2

如answer by EvanM中所述，您需要在关键字之前和之后添加\b字边界匹配器，以防止字词内的子字符串匹配。

为了获得更好的性能，您还应该使用|逻辑正则表达式运算符来匹配多个值中的一个，而不是创建多个匹配器，因此您只需扫描line一次，并且只有编译一个正则表达式。

您甚至可以在单个正则表达式中组合您要查找的3种不同类型的令牌，并使用捕获组来区分它们，因此您只需要扫描line一次。

像这样：

String regex = "\\b(array|as|asm|begin|case|class|const|constructor|destructor|dispinterface|div|do|downto|else|end|except|exports|file|finalization|finally|for|function|goto|if|implementation|inherited|initialization|inline|interface|is|label|library|mod|nil|object|of|out|packed|procedure|program|property|raise|record|repeat|resourcestring|set|shl|shr|string|then|threadvar|to|try|type|unit|until|uses|var|while|with)\\b" +
              "|(=|<[>=]?|>=?)" +
              "|\\b(and|not|or|xor)\\b";
for (Matcher m = Pattern.compile(regex).matcher(line); m.find(); ) {
    if (m.start(1) != -1) {
        System.out.println("Keyword\t\t" + m.group(1));
    } else if (m.start(2) != -1) {
        System.out.println("logic_op\t\t" + m.group(2));
    } else {
        System.out.println("relational_op\t\t" + m.group(3));
    }
}

您甚至可以通过将关键字与公共前缀相结合来进一步优化它，例如： as|asm可能会成为asm?，即as可选地后跟m。会使关键字列表的可读性降低，但效果会更好。

在上面的代码中，我为逻辑操作执行了此操作，以显示如何以及修复原始代码中的匹配错误，其中>=中的line将显示3次按此顺序列为=，>，>=，问题类似于问题中要求的子关键字问题。

Java - matcher重读单词

2 个答案: