Question

我正在尝试创建一个读取表达式的tokenizer。当它获得令牌时，我打印出类型和lexeme，它是一个操作符号。它目前单独打印每个字符，但我需要它来打印两位数和十进制数。这是我的代码。

    public class Tokenizer {
String type;
String lexeme;
/**
 * @param args the command line arguments
 */
void getToken(String expression, int i) {

        if (expression.charAt(i) == '+')
        {
            type = "operator";
            lexeme = "+";       
        }
        else if (expression.charAt(i) == '-')
        {
            type = "operator";
            lexeme = "-";
        }
        else if (expression.charAt(i) == '*')
        {
            type = "operator";
            lexeme = "*";
        }
        else if (expression.charAt(i) == '/')
        {
            type = "operator";
            lexeme = "/";
        }
        else if (expression.charAt(i) == ' ')
        {
            ; //empty statement. Does nothing if the character is white space
        }
        else 
        {
            type = "number";
            lexeme = Character.toString(expression.charAt(i));
        }
}
public static void main(String[] args) {
    // TODO code application logic here
    String expression = "3+66*2.5";
    Tokenizer tokenizer = new Tokenizer();
    for (int i = 0; i < expression.length(); i++)
    {
        tokenizer.getToken(expression, i);
        System.out.print("Type: " + tokenizer.type + "\t" + "Lexeme: " + tokenizer.lexeme + "\n");
    }
  }
}

示例输出 - 实际

Type: number    Lexeme: 3
Type: operator  Lexeme: +
Type: number    Lexeme: 6
Type: number    Lexeme: 6
Type: operator  Lexeme: *
Type: number    Lexeme: 2
Type: number    Lexeme: .
Type: number    Lexeme: 5

样本输出 - 预期

Type: number    Lexeme: 3
Type: operator  Lexeme: +
Type: number    Lexeme: 66
Type: operator  Lexeme: *
Type: number    Lexeme: 2.5

Answer 1

此语句将覆盖每次执行的词汇

 lexeme = Character.toString(expression.charAt(i));

应该是

lexeme += Character.toString(expression.charAt(i));

即。识别单个数字后，将其添加到词汇中。

除此之外，你的getToken方法不正确然后将i传递给它你假设一个词汇只有一个字符长，实际上它可以是1或更多，例如{ {1}}或66。所以你需要重新设计你的2.5课程。您应该使用表达式初始化Tokenizer对象，然后让它具有无参数方法Tokenizer，该方法根据需要进行尽可能多的迭代以识别令牌然后返回它。它还应该有一种方法来检查是否还有任何输入，例如getNextToken()。

这是你可以建立的东西

hasMoreTokens()

<强>输出

public class Tokenizer {
    int pos;
    char[] expression;

    Tokenizer(String expression) {
        this.expression = expression.toCharArray();
        this.pos = 0;
    }

    enum Type { OPERATOR, NUMBER, UNKNOWN }

    class Lexeme {
        String type, token;
        Lexeme(String type, String token) {
            this.type = type;
            this.token = token;
        }
    }

    Lexeme getNextToken() {
        StringBuilder token = new StringBuilder();
        boolean endOfToken = false;
        Type type = Type.UNKNOWN;
        while (!endOfToken && hasMoreTokens()) {
            while(expression[pos] == ' ' && hasMoreTokens())
                pos++;
            switch (expression[pos]) {
                case '+':
                case '-':
                case '*':
                case '/':
                    if(type != Type.NUMBER) {
                        type = Type.OPERATOR;
                        token.append(expression[pos]);
                        pos++;
                    }
                    endOfToken = true;
                    break;
                case ' ':
                    endOfToken = true;
                    pos++;
                    break;
                default:
                    if(Character.isDigit(expression[pos]) || expression[pos] == '.') {
                        token.append(expression[pos]);
                        type = Type.NUMBER;
                    } else {
                        System.out.println("Systax error at position: " + pos);
                    }
                    pos++;
                    break;
            }
        }
        return new Lexeme(type.name().toLowerCase(), token.toString());
    }

    boolean hasMoreTokens() {
        return pos < expression.length;
    }

    public static void main(String[] args) {
        String expression = "3+66*2.5";
        Tokenizer tokenizer = new Tokenizer(expression);
        while (tokenizer.hasMoreTokens()) {
            Lexeme nextToken = tokenizer.getNextToken();
            System.out.print("Type: " + nextToken.type + "\tLexeme: " + nextToken.token + "\n");
        }
    }
}

表达式标记符

示例输出 - 实际

样本输出 - 预期

1 个答案: