Question

我有一个外观和行为如下的字符串（提供Python代码）。 WTF？它的编码是什么？

s = u'\x00Q\x00u\x00i\x00c\x00k'
>>> print s
Quick
>>>
>>> s == 'Quick'
False
>>>
>>> import re
>>> re.search('Quick', s)
>>>
>>> import chardet
>>> chardet.detect(s)
/usr/lib/pymodules/python2.6/chardet/universaldetector.py:69: UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal
  if aBuf[:3] == '\xEF\xBB\xBF':
/usr/lib/pymodules/python2.6/chardet/universaldetector.py:72: UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal
  elif aBuf[:4] == '\xFF\xFE\x00\x00':
/usr/lib/pymodules/python2.6/chardet/universaldetector.py:75: UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal
  elif aBuf[:4] == '\x00\x00\xFE\xFF':
/usr/lib/pymodules/python2.6/chardet/universaldetector.py:78: UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal
  elif aBuf[:4] == '\xFE\xFF\x00\x00':
/usr/lib/pymodules/python2.6/chardet/universaldetector.py:81: UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal
  elif aBuf[:4] == '\x00\x00\xFF\xFE':
/usr/lib/pymodules/python2.6/chardet/universaldetector.py:84: UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal
  elif aBuf[:2] == '\xFF\xFE':
/usr/lib/pymodules/python2.6/chardet/universaldetector.py:87: UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal
  elif aBuf[:2] == '\xFE\xFF':
{'confidence': 1.0, 'encoding': 'ascii'}
>>>
>>> chardet.detect(s)
{'confidence': 1.0, 'encoding': 'ascii'}
>>>

Answer 1

UTF-16大端

Answer 2

你有没有BOM的UTF-16BE。如上所述，chardet没有没有BOM就没有UTF-nnxE。

>>> s = '\x00Q\x00u\x00i\x00c\x00k' #### Note: dropping the spurious `u` prefix
>>> s.decode('utf_16be')
u'Quick'
>>>

如果你提供unicode，chardet也不够聪明，不能提出DontBeSilly异常： - ）

什么编码看起来完全像ASCII，但每个字节前有NULL字节？

2 个答案: