如何将文本转换为MOA实例?

时间:2019-07-11 03:17:06

标签: text weka text-classification moa

我正在MOA中为文本应用程序进行增量学习。这要求创建一个Instance对象,以数字形式表示文本,例如词汇表中每个词干的TF-IDF分数。我的MOA版本是2019.05.0。

我在MOA中寻找了文本处理工具,但找不到它们。

我看到Weka有一个StringToWordVector类,所以我决定尝试一下。 Weka的类与MOA的类不同,但是有一个名为WekaToSamoaInstanceConverter的类,我认为我可以创建一个Weka实例,通过StringToWordVector运行它,并将其转换为MOA实例。也许这是错误的曲目,或者这是正确的曲目,但我的语法中缺少某些内容。

public static Instances convertDirectoryToInstances(String directory) throws Exception {
    //Create an object that reads training or test files from a directory.
    //In the future, I'll want to add one file at a time. That's not the part I'm worried about at the moment.
    TextDirectoryLoader loader = new TextDirectoryLoader();
    String[] options = new String[] {"-dir", directory, "-charset", "UTF-8"};
    loader.setOptions(options);
    loader.getStructure();

    //Create Weka Instances that represent unprocessed text.
    weka.core.Instances plainTextInstances = loader.getDataSet();

    //A StringToWordVector is a Filter that converts text to text vectors.
    //I'm not using any bells and whistles for this example, so I expect each Instance to be a set of terms in the document.
    StringToWordVector stringToWordVector = new StringToWordVector();
    stringToWordVector.setInputFormat(plainTextInstances);
    weka.core.Instances wekaWordVectors = Filter.useFilter(plainTextInstances, stringToWordVector);

    //A MOA Instance is different from a Weka Instance, so we need to convert them.
    WekaToSamoaInstanceConverter converter = new WekaToSamoaInstanceConverter();

    //This is what fails.
    Instances moaWordVectors = converter.samoaInstances(wekaWordVectors);
    return moaWordVectors;
}

wekaWordVectors.size()是子目录中的文件数,所以这就是我所期望的。

对samoaInstances()的调用失败。第220行尝试调用locateIndex(0)。 0处没有类,因此返回-1。此-1用作数组索引,因此我得到了ArrayIndexOutOfBoundsException。我不知道类0的含义,但是我知道ArrayIndexOutOfBoundsException意味着我做错了。

0 个答案:

没有答案