最快的字节数组串联方法

时间:2012-05-02 14:53:24

标签: java performance bytearray concatenation

我得到了一个包含消息的n部分作为字节数组的映射。在最后一篇文章进入地图后,必须连接消息。我找到了两个应该满足要求的解决方案。第一个是使用System.arraycopy:

public byte[] getMessageBytes() throws IOException {
    byte[] bytes = new byte[0];
    for (final Map.Entry<Short,byte[]> entry : myMap.entrySet()) {
        byte[] entryBytes = entry.getValue();
        byte[] temp = new byte[bytes.length + entryBytes.length];
        System.arraycopy(bytes, 0, temp, 0, bytes.length);
        System.arraycopy(entryBytes, 0, temp, bytes.length, entryBytes.length);
        bytes = temp;
    }
    return bytes;
}

第二个是使用ByteArrayOutputStream:

public byte[] getMessageBytes() throws IOException {
    final ByteArrayOutputStream baos = new ByteArrayOutputStream();
    for (final Map.Entry<Short,byte[]> entry : myMap.entrySet()) {
        baos.write(entry.getValue());
    }
    baos.flush();
    return baos.toByteArray();
}

从性能和内存使用角度看,哪种方法更好? 是否有另一种方法可以进行更好的连接?

6 个答案:

答案 0 :(得分:9)

由于您可以通过累加片段的长度来找出消息的大小,我会:

  1. 加上各个部分的长度,并分配输出数组;
  2. 使用循环将arraycopy()每个部分放入输出数组中的正确位置。
  3. 这可能是内存效率高且速度快的。但是,只有剖析才能说出完整的故事。

答案 1 :(得分:4)

这应该比你的第一个版本(未经测试)表现得更好

public byte[] getMessageBytes() throws IOException {
    long amount = 0L;
    long offset = 0L;
    // no reason to use entrySet() if you just use the values
    for (byte[] arr : myMap.values()) {
        amount += arr.length;
    }
    byte[] dest = new byte[amount];
    for (byte[] arr : myMap.values()) {
        System.arraycopy(arr, 0, dest, offset, arr.length);
        offset += arr.length;
    }
    return dest;
}

(这个答案大致相当于aix的)

答案 2 :(得分:0)

正确的答案是测试和比较您的具体情况。

这是一个SortedMap,就像TreeMap一样,或者你实际上是在随机合并这些字节吗?

答案 3 :(得分:0)

为每次迭代创建新数组的第一个解决方案是O(n ^ 2),如果您有许多条目,这是一个问题。而且它相当复杂。

使用ByteArrayOutputStream更好有两个原因:它在O(n)中运行,而且非常简单。

最有可能的是:如果您首先计算总大小,然后使用System.arraycopy。但是,如果ByteArrayOutputStream 真的太慢,我只会这样做。

答案 4 :(得分:0)

首先计算大小并分配结果一次应该是返回字节数组的最快解决方案。

如果稍后使用生成的字节数组作为InputStream,并且根据数组的组合大小,最快的方法可能是根本不连接。在这种情况下,您可以创建SequenceInputStream包裹多个ByteArrayInputStreams。未经测试的示例代码:

Collection<byte[]> values = map.values();
List<ByteArrayInputStream> streams = new ArrayList<ByteArrayInputStream>(values.size());
for (byte[] bytes : values) {
    streams.add(new ByteArrayInputStream(bytes));
}
return new SequenceInputStream(Collections.enumeration(streams));

答案 5 :(得分:0)

这是本地代码。

    /*
     * public static void arraycopy(Object src, int srcPos, Object dest,
     *      int destPos, int length)
     *
     * The description of this function is long, and describes a multitude
     * of checks and exceptions.
     */
    static void Dalvik_java_lang_System_arraycopy(const u4* args, JValue* pResult)
    {
        ArrayObject* srcArray = (ArrayObject*) args[0];
        int srcPos = args[1];
        ArrayObject* dstArray = (ArrayObject*) args[2];
        int dstPos = args[3];
        int length = args[4];
        /* Check for null pointers. */
        if (srcArray == NULL) {
            dvmThrowNullPointerException("src == null");
            RETURN_VOID();
        }
        if (dstArray == NULL) {
            dvmThrowNullPointerException("dst == null");
            RETURN_VOID();
        }
        /* Make sure source and destination are arrays. */
        if (!dvmIsArray(srcArray)) {
            dvmThrowArrayStoreExceptionNotArray(((Object*)srcArray)->clazz, "source");
            RETURN_VOID();
        }
        if (!dvmIsArray(dstArray)) {
            dvmThrowArrayStoreExceptionNotArray(((Object*)dstArray)->clazz, "destination");
            RETURN_VOID();
        }
        /* avoid int overflow */
        if (srcPos < 0 || dstPos < 0 || length < 0 ||
            srcPos > (int) srcArray->length - length ||
            dstPos > (int) dstArray->length - length)
        {
            dvmThrowExceptionFmt(gDvm.exArrayIndexOutOfBoundsException,
                "src.length=%d srcPos=%d dst.length=%d dstPos=%d length=%d",
                srcArray->length, srcPos, dstArray->length, dstPos, length);
            RETURN_VOID();
        }
        ClassObject* srcClass = srcArray->clazz;
        ClassObject* dstClass = dstArray->clazz;
        char srcType = srcClass->descriptor[1];
        char dstType = dstClass->descriptor[1];
        /*
         * If one of the arrays holds a primitive type, the other array must
         * hold the same type.
         */
        bool srcPrim = (srcType != '[' && srcType != 'L');
        bool dstPrim = (dstType != '[' && dstType != 'L');
        if (srcPrim || dstPrim) {
            if (srcPrim != dstPrim || srcType != dstType) {
                dvmThrowArrayStoreExceptionIncompatibleArrays(srcClass, dstClass);
                RETURN_VOID();
            }
            if (false) ALOGD("arraycopy prim[%c] dst=%p %d src=%p %d len=%d",
                srcType, dstArray->contents, dstPos,
                srcArray->contents, srcPos, length);
            switch (srcType) {
            case 'B':
            case 'Z':
                /* 1 byte per element */
                memmove((u1*) dstArray->contents + dstPos,
                    (const u1*) srcArray->contents + srcPos,
                    length);
                break;
            case 'C':
            case 'S':
                /* 2 bytes per element */
                move16((u1*) dstArray->contents + dstPos * 2,
                    (const u1*) srcArray->contents + srcPos * 2,
                    length * 2);
                break;
            case 'F':
            case 'I':
                /* 4 bytes per element */
                move32((u1*) dstArray->contents + dstPos * 4,
                    (const u1*) srcArray->contents + srcPos * 4,
                    length * 4);
                break;
            case 'D':
            case 'J':
                /*
                 * 8 bytes per element.  We don't need to guarantee atomicity
                 * of the entire 64-bit word, so we can use the 32-bit copier.
                 */
                move32((u1*) dstArray->contents + dstPos * 8,
                    (const u1*) srcArray->contents + srcPos * 8,
                    length * 8);
                break;
            default:        /* illegal array type */
                ALOGE("Weird array type '%s'", srcClass->descriptor);
                dvmAbort();
            }
        } else {
            /*
             * Neither class is primitive.  See if elements in "src" are instances
             * of elements in "dst" (e.g. copy String to String or String to
             * Object).
             */
            const int width = sizeof(Object*);
            if (srcClass->arrayDim == dstClass->arrayDim &&
                dvmInstanceof(srcClass, dstClass))
            {
                /*
                 * "dst" can hold "src"; copy the whole thing.
                 */
                if (false) ALOGD("arraycopy ref dst=%p %d src=%p %d len=%d",
                    dstArray->contents, dstPos * width,
                    srcArray->contents, srcPos * width,
                    length * width);
                move32((u1*)dstArray->contents + dstPos * width,
                    (const u1*)srcArray->contents + srcPos * width,
                    length * width);
                dvmWriteBarrierArray(dstArray, dstPos, dstPos+length);
            } else {
                /*
                 * The arrays are not fundamentally compatible.  However, we
                 * may still be able to do this if the destination object is
                 * compatible (e.g. copy Object[] to String[], but the Object
                 * being copied is actually a String).  We need to copy elements
                 * one by one until something goes wrong.
                 *
                 * Because of overlapping moves, what we really want to do
                 * is compare the types and count up how many we can move,
                 * then call move32() to shift the actual data.  If we just
                 * start from the front we could do a smear rather than a move.
                 */
                Object** srcObj;
                int copyCount;
                ClassObject*   clazz = NULL;
                srcObj = ((Object**)(void*)srcArray->contents) + srcPos;
                if (length > 0 && srcObj[0] != NULL)
                {
                    clazz = srcObj[0]->clazz;
                    if (!dvmCanPutArrayElement(clazz, dstClass))
                        clazz = NULL;
                }
                for (copyCount = 0; copyCount < length; copyCount++)
                {
                    if (srcObj[copyCount] != NULL &&
                        srcObj[copyCount]->clazz != clazz &&
                        !dvmCanPutArrayElement(srcObj[copyCount]->clazz, dstClass))
                    {
                        /* can't put this element into the array */
                        break;
                    }
                }
                if (false) ALOGD("arraycopy iref dst=%p %d src=%p %d count=%d of %d",
                    dstArray->contents, dstPos * width,
                    srcArray->contents, srcPos * width,
                    copyCount, length);
                move32((u1*)dstArray->contents + dstPos * width,
                    (const u1*)srcArray->contents + srcPos * width,
                    copyCount * width);
                dvmWriteBarrierArray(dstArray, 0, copyCount);
                if (copyCount != length) {
                    dvmThrowArrayStoreExceptionIncompatibleArrayElement(srcPos + copyCount,
                            srcObj[copyCount]->clazz, dstClass);
                    RETURN_VOID();
                }
            }
        }
        RETURN_VOID();
    }