如何丢弃不需要的令牌

时间:2015-09-28 19:40:16

标签: java loops iterator token

我创建了这个令牌迭代器类,它按每个字符扫描输入行并创建String。我能够让类读取所有标记并将它们分开,但是我无法删除输入中的所有无效/不需要的标记。

我尝试用character.toString(counter)做一些事情,所以这些字符被编成一个字符串,然后编写if语句,当令牌不是“not”,“true”,“false”或“and”然后丢弃它并继续下一个标记。

import java.util.Iterator;

public class Tokiter implements Iterator<String>{
    private char counter = 0;
    //input line to be tokenized
    private String line;

    // the next Token, null if no next Token
    private String nextToken;

        // implement
    public TokIter(String line){
        this.line = line;
    }


    @Override
    // implement
    public boolean hasNext() {

        if (counter >= line.length())
            return false;
        else if (line.charAt(counter) == ' ')
        {
            counter++;
            return hasNext();
        }

        else
            return true;
    }


    @Override
    //implement
    public String next() {
        String s = "";
        if (!hasNext())
        {
    //          System.out.println("Null");
            return null;
        }
        else if( line.charAt(counter) == ('('))
        {
    //          System.out.println("Token");
            s += line.charAt(counter);
            counter++;
            return s;
        }
        else if( line.charAt(counter) == (')'))
        {   
    //          System.out.println("Token");
            s += line.charAt(counter);
            counter++;
            return s;
        }
        else
            s += line.charAt(counter);
        counter++;
        if (counter >= line.length()){
            return s;
        }
        while (Character.isLetter(line.charAt(counter)))
        {
            s += line.charAt(counter);
            counter++;
            if (counter >= line.length()){
                return s;
            }


        }
        return s;
    }   


    @Override
    public void remove() {
        // TODO Auto-generated method stub
        throw new UnsupportedOperationException();
    }

        // provided
    public static void main(String[] args){
        String line;
        // you can play with other inputs on the command line
        if(args.length>0)
            line = args[0];
        // or do the standard test
        else
            line = " not random (true or false) ** whatever ";
        System.out.println("line: [" + line + "]");
        Tokiter tokIt = new Tokiter(line);
        while(tokIt.hasNext()){
            System.out.println("next token: [" + tokIt.next() + "]");
        }
    }
}

因此,例如当程序运行输入行not random (true or false) ** whatever时,输出将为:

line: [  not random (true or false) ** whatever  ]    
next token: [not]    
next token: [(]    
next token: [true]    
next token: [or]    
next token: [false]    
next token: [)]

0 个答案:

没有答案