我观察到意外的行为,其中看到由化简器创建的自定义writable
在同一化简器中被重用。
例如
public class myWritable implements Writable {
private Map<Long, MyObject> myObjectMap = new HashMap<>();
private myInt;
@Override
public void write(DataOutput out) throws IOException {
out.writeInt(myInt);
out.writeInt(myObjectMap.size());
for (MyObject myObjectInstance: myObjectMap.values()) {
out.writeLong(myObjectInstance.getId());
}
}
@Override
public void readFields(DataInput in) throws IOException {
myInt = in.readInt();
int myObjectMapSize = in.readInt();
int myObjectId;
for (int i = 0; i < myObjectMapSize; i++) {
myObjectId = in.readLong();
myObjectMap.put(myObjectId, new MyObject(myObjectId));
}
}
}
我发现myObjectMap
只会增加我处理的后续密钥的大小。这意味着每个reducer都在重用已创建的可写实例。
这对我来说不太有意义,因为我没有看到doc
中记录的这种行为我做错什么了吗?