如何单元测试Hadoop Writable

时间:2012-11-08 11:30:36

标签: unit-testing hadoop

我已经实现了WritableComparable我自己的write,但我找不到合理测试readFields和{{1}}方法的好方法。

有什么想法吗?

1 个答案:

答案 0 :(得分:5)

可能您可以找到更简单的方法来测试您的可写,但手动执行序列化/反序列化也可以。 E.g:

MyUtils.java:

...
import org.apache.commons.io.IOUtils;
...
public static byte[] serialize(Writable writable) throws IOException {
        ByteArrayOutputStream out = new ByteArrayOutputStream();
        DataOutputStream dataOut = null;
        try {
            dataOut = new DataOutputStream(out);
            writable.write(dataOut);
            return out.toByteArray();
        } 
        finally {
            IOUtils.closeQuietly(dataOut);
        }
    }

public static <T extends Writable> T asWritable(byte[] bytes, Class<T> clazz)
            throws IOException {
        T result = null;
        DataInputStream dataIn = null;
        try {
            result = clazz.newInstance();
            ByteArrayInputStream in = new ByteArrayInputStream(bytes);
            dataIn = new DataInputStream(in);
            result.readFields(dataIn);
        } catch (InstantiationException e) {
            // should not happen
            assert false;
        } catch (IllegalAccessException e) {
            // should not happen
            assert false;
        } finally {
            IOUtils.closeQuietly(dataIn);
        }
        return result;
    }

然后在你的测试课程中:

CustomWritable record = ... ; //your initialized Writable
byte[] serializedBytes = MyUtils.serialize(record);

CustomWritable deserialized = 
  MyUtils.asWritable(serializedBytes, CustomWritable.class);

assertEquals("Value mismatch!", record.getFieldA(), deserialized.getFieldA());
...