将字节数组从utf-16转换为utf-8

时间:2015-05-14 18:00:53

标签: c++ utf-8 utf-16

我有一个字节数组

uint8_t array[] = {0x00, 0x72, 0x00, 0x6f,  0x00, 0x6f, 0x00, 0x74};

我知道,在文本中这是“根”; 我有一个应该将utf-16转换为utf-8的函数。这是代码:

inline bool convertUcs2ToUtf8(const std::vector<char> &from, std::string* const to) {
    return ucnvConvert("UTF-16", "UTF-8", from, to);
}

static inline bool ucnvConvert(const char *enc_from,
                               const char *enc_to,
                               const std::vector<char> &from,
                               std::string* const to)
{
    if (from.empty()) {
        to->clear();
        return true;
    }

    unsigned int maxOutSize = from.size() * 3 + 1;
    std::vector<char> outBuf(maxOutSize);

    iconv_t c = iconv_open(enc_to, enc_from);
    ASSERT_MSG(c != NULL, "convert: illegal encodings");
    char *from_ptr = const_cast<char*>(from.data());
    char *to_ptr = &outBuf[0];

    size_t inleft = from.size(), outleft = maxOutSize;
    size_t n = iconv(c, &from_ptr, &inleft, &to_ptr, &outleft);
    bool success = true;
    if (n == (size_t)-1) {
        success = false;
        if (errno == E2BIG) {
            ELOG("convert: insufficient space from");
        } else if (errno == EILSEQ) {
            ELOG("convert: invalid input sequence");
        } else if (errno == EINVAL) {
            ELOG("convert: incomplete input sequence");
        }
    }
    if (success) {
        to->assign(&outBuf[0], maxOutSize - outleft);
    }
    iconv_close(c);
    return success;
}

它适用于西里尔语(它以0x04开头),但是当我尝试将我的数组放入其中时,我会得到类似的结果:

爀漀漀琀开㌀㜀

依旧...... 这有什么不对?

1 个答案:

答案 0 :(得分:3)

必须为UTF-16输入指定字节顺序。由于您传递的是utf16-be(big-endian)编码缓冲区,因此应在其前面加上相应的字节顺序标记:

uint8_t array[] = { 0xfe, 0xff, 0x00, 0x72, 0x00, 0x6f, 0x00, 0x6f, 0x00, 0x74 };

但这会产生UTF-8输出,带有您可能不想要的字节顺序标记。最有效的方法是以这种方式指定字节顺序:

ucnvConvert("UTF-16BE", "UTF-8", from, to);