我正在MOA中为文本应用程序进行增量学习。这要求创建一个Instance对象,以数字形式表示文本,例如词汇表中每个词干的TF-IDF分数。我的MOA版本是2019.05.0。
我在MOA中寻找了文本处理工具,但找不到它们。
我看到Weka有一个StringToWordVector类,所以我决定尝试一下。 Weka的类与MOA的类不同,但是有一个名为WekaToSamoaInstanceConverter的类,我认为我可以创建一个Weka实例,通过StringToWordVector运行它,并将其转换为MOA实例。也许这是错误的曲目,或者这是正确的曲目,但我的语法中缺少某些内容。
public static Instances convertDirectoryToInstances(String directory) throws Exception {
//Create an object that reads training or test files from a directory.
//In the future, I'll want to add one file at a time. That's not the part I'm worried about at the moment.
TextDirectoryLoader loader = new TextDirectoryLoader();
String[] options = new String[] {"-dir", directory, "-charset", "UTF-8"};
loader.setOptions(options);
loader.getStructure();
//Create Weka Instances that represent unprocessed text.
weka.core.Instances plainTextInstances = loader.getDataSet();
//A StringToWordVector is a Filter that converts text to text vectors.
//I'm not using any bells and whistles for this example, so I expect each Instance to be a set of terms in the document.
StringToWordVector stringToWordVector = new StringToWordVector();
stringToWordVector.setInputFormat(plainTextInstances);
weka.core.Instances wekaWordVectors = Filter.useFilter(plainTextInstances, stringToWordVector);
//A MOA Instance is different from a Weka Instance, so we need to convert them.
WekaToSamoaInstanceConverter converter = new WekaToSamoaInstanceConverter();
//This is what fails.
Instances moaWordVectors = converter.samoaInstances(wekaWordVectors);
return moaWordVectors;
}
wekaWordVectors.size()是子目录中的文件数,所以这就是我所期望的。
对samoaInstances()的调用失败。第220行尝试调用locateIndex(0)。 0处没有类,因此返回-1。此-1用作数组索引,因此我得到了ArrayIndexOutOfBoundsException。我不知道类0的含义,但是我知道ArrayIndexOutOfBoundsException意味着我做错了。