Question

我想传递一个float数组作为mapper中的键，

Such as [0.6, 0, 10]

我使用StringUtils.join将其转换为字符串，并写入

context.write(new Text(str), value);

但在减速机课上，我发现我得到的是：

"0 0 1"

如何解决？我正在使用hadoop 1.0.4

Answer 1

您可以使用ArrayWritable发出浮点数组。我不确定将数组作为键发射，下面是您发出与值相同的方式。

ArrayWritable does not implement WritableComparable, so it can't currently
be used as a mapper output key - those keys have to be sorted during the
shuffle, and thus the type must be WritableComparable.

通常，对于发出密钥，我们使用自定义可写比较类。

为此，您应该在驱动程序类中定义相同的内容公共类司机{

public static class DoubleArrayWritable extends ArrayWritable {
    public DoubleArrayWritable() {
        super(DoubleWritable.class);
    }
}

public static void main(String[] args) {

在Mapper中

DoubleArrayWritable pointArray = new DoubleArrayWritable();
DoubleWritable[] data = new DoubleWritable[numAttributes];
for (int k = 0; k < numAttributes; k++) {
    data[k] = new DoubleWritable(point[k]);
}
pointArray.set(data);
context.write(new Text("Array"), pointArray);

希望这在某些方面有所帮助。

Hadoop MapReduce输入键错误

1 个答案: