Java 8 - 将List <byte []>合并到byte []的最有效方法

时间:2016-10-06 08:19:44

标签: java performance stream inputstream

我有一个库,它返回一些二进制数据作为二进制数组列表。那些byte []需要合并到一个InputStream中。

这是我目前的实施:

public static InputStream foo(List<byte[]> binary) {
    byte[] streamArray = null;
    binary.forEach(bin -> {
        org.apache.commons.lang.ArrayUtils.addAll(streamArray, bin);
    });
    return new ByteArrayInputStream(streamArray);
}

但这非常激烈。还有更好的方法吗?

感谢所有答案。我做了一次性能测试。这些是我的结果:

  • 功能:'NicolasFilotto'=&gt;平均每次通话时间为68,04毫秒
  • 功能:'NicolasFilottoEstSize'=&gt;平均每次拨打65,24毫秒
  • 功能:'NicolasFilottoSequenceInputStream'=&gt;平均每次拨打电话63,09毫秒
  • 功能:'Saka1029_1'=&gt;平均每次拨打电话63,06毫秒
  • 功能:'Saka1029_2'=&gt; 100次通话平均0,79毫秒
  • 功能:'Coco'=&gt;平均10次通话541,60 ms

我不确定'Saka1029_2'是否正确测量...

这是执行功能:

private static double execute(Callable<InputStream> funct, int times) throws Exception {
    List<Long> executions = new ArrayList<>(times);

    for (int idx = 0; idx < times; idx++) {
        BufferedReader br = null;
        long startTime = System.currentTimeMillis();
        InputStream is = funct.call();
        br = new BufferedReader(new InputStreamReader(is));
        String line = null;
        while ((line = br.readLine()) != null) {}
        executions.add(System.currentTimeMillis() - startTime);
    }

    return calculateAverage(executions);
}

请注意我读取了每个输入流

这些是用过的实现:

public static class NicolasFilotto implements Callable<InputStream> {

    private final List<byte[]> binary;

    public NicolasFilotto(List<byte[]> binary) {
        this.binary = binary;
    }

    @Override
    public InputStream call() throws Exception {
        ByteArrayOutputStream baos = new ByteArrayOutputStream();
        for (byte[] bytes : binary) {
            baos.write(bytes, 0, bytes.length);
        }
        return new ByteArrayInputStream(baos.toByteArray());
    }

}

public static class NicolasFilottoSequenceInputStream implements Callable<InputStream> {

    private final List<byte[]> binary;

    public NicolasFilottoSequenceInputStream(List<byte[]> binary) {
        this.binary = binary;
    }

    @Override
    public InputStream call() throws Exception {
        return new SequenceInputStream(
                Collections.enumeration(
                        binary.stream().map(ByteArrayInputStream::new).collect(Collectors.toList())));
    }

}

public static class NicolasFilottoEstSize implements Callable<InputStream> {

    private final List<byte[]> binary;
    private final int lineSize;

    public NicolasFilottoEstSize(List<byte[]> binary, int lineSize) {
        this.binary = binary;
        this.lineSize = lineSize;
    }

    @Override
    public InputStream call() throws Exception {
        ByteArrayOutputStream baos = new ByteArrayOutputStream(binary.size() * lineSize);
        for (byte[] bytes : binary) {
            baos.write(bytes, 0, bytes.length);
        }
        return new ByteArrayInputStream(baos.toByteArray());
    }

}

public static class Saka1029_1 implements Callable<InputStream> {

    private final List<byte[]> binary;

    public Saka1029_1(List<byte[]> binary) {
        this.binary = binary;
    }

    @Override
    public InputStream call() throws Exception {
        byte[] all = new byte[binary.stream().mapToInt(a -> a.length).sum()];
        int pos = 0;
        for (byte[] bin : binary) {
            int length = bin.length;
            System.arraycopy(bin, 0, all, pos, length);
            pos += length;
        }
        return new ByteArrayInputStream(all);
    }

}

public static class Saka1029_2 implements Callable<InputStream> {

    private final List<byte[]> binary;

    public Saka1029_2(List<byte[]> binary) {
        this.binary = binary;
    }

    @Override
    public InputStream call() throws Exception {
        int size = binary.size();
        return new InputStream() {
            int i = 0, j = 0;

            @Override
            public int read() throws IOException {
                if (i >= size) return -1;
                if (j >= binary.get(i).length) {
                    ++i;
                    j = 0;
                }
                if (i >= size) return -1;
                return binary.get(i)[j++];
            }
        };
    }

}

public static class Coco implements Callable<InputStream> {

    private final List<byte[]> binary;

    public Coco(List<byte[]> binary) {
        this.binary = binary;
    }

    @Override
    public InputStream call() throws Exception {
        byte[] streamArray = new byte[0];
        for (byte[] bin : binary) {
            streamArray = org.apache.commons.lang.ArrayUtils.addAll(streamArray, bin);
        }
        return new ByteArrayInputStream(streamArray);
    }

}

2 个答案:

答案 0 :(得分:3)

您可以使用ByteArrayOutputStream来存储列表中每个字节数组的内容,但为了提高效率,我们需要创建ByteArrayOutputStream 的实例,其初始大小为< / strong>尽可能匹配目标大小,所以如果你知道字节数组的大小或至少是平均大小,你应该使用它,代码是:

public static InputStream foo(List<byte[]> binary) {
    ByteArrayOutputStream baos = new ByteArrayOutputStream(ARRAY_SIZE * binary.size());
    for (byte[] bytes : binary) {
        baos.write(bytes, 0, bytes.length);
    }
    return new ByteArrayInputStream(baos.toByteArray());
}

另一种方法是使用SequenceInputStream来逻辑连接表示列表中一个元素的所有ByteArrayInputStream个实例,如下所示:

public static InputStream foo(List<byte[]> binary) {
    return new SequenceInputStream(
        Collections.enumeration(
            binary.stream().map(ByteArrayInputStream::new).collect(Collectors.toList())
        )
    );
}

这种方法的有趣之处在于你不需要复制任何东西,你只创建ByteArrayInputStream的实例,它将按原样使用字节数组。

为了避免以List收取结果,特别是如果您的初始List很大,您可以按照@Holger的建议直接拨打iterator(),然后我们只需要将iterator转换为enumeration,这可以通过IteratorUtils.asEnumeration(iterator)中的Apache Commons Collection转换,最终的代码将是:

public static InputStream foo(List<byte[]> binary) {
    return new SequenceInputStream(
        IteratorUtils.asEnumeration(
            binary.stream().map(ByteArrayInputStream::new).iterator()
        )
    );
}

答案 1 :(得分:2)

试试这个。

public static InputStream foo(List<byte[]> binary) {
    byte[] all = new byte[binary.stream().mapToInt(a -> a.length).sum()];
    int pos = 0;
    for (byte[] bin : binary) {
        int length = bin.length;
        System.arraycopy(bin, 0, all, pos, length);
        pos += length;
    }
    return new ByteArrayInputStream(all);
}

或者

public static InputStream foo(List<byte[]> binary) {
    int size = binary.size();
    return new InputStream() {
        int i = 0, j = 0;
        @Override
        public int read() throws IOException {
            if (i >= size) return -1;
            if (j >= binary.get(i).length) {
                ++i;
                j = 0;
            }
            if (i >= size) return -1;
            return binary.get(i)[j++];
        }
    };
}