我试图将字典的具体内容保存到文件中但是当我尝试编写它时，我收到以下错误：
Traceback (most recent call last):
File "P4.py", line 83, in <module>
outfile.write(u"{}\t{}\n".format(keyword, str(tagSugerido)).encode("utf-8"))
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 40: ordinal not in range(128)
以下是代码：
from collections import Counter
with open("corpus.txt") as inf:
wordtagcount = Counter(line.decode("latin_1").rstrip() for line in inf)
with open("lexic.txt", "w") as outf:
outf.write('Palabra\tTag\tApariciones\n'.encode("utf-8"))
for word,count in wordtagcount.iteritems():
outf.write(u"{}\t{}\n".format(word, count).encode("utf-8"))
"""
2) TAGGING USING THE MODEL
Dados los ficheros de test, para cada palabra, asignarle el tag mas
probable segun el modelo. Guardar el resultado en ficheros que tengan
este formato para cada linea: Palabra Prediccion
"""
file=open("lexic.txt", "r") # abrimos el fichero lexic (nuestro modelo) (probar con este)
data=file.readlines()
file.close()
diccionario = {}
"""
In this portion of code we iterate the lines of the .txt document and we create a dictionary with a word as a key and a List as a value
Key: word
Value: List ([tag, #ocurrencesWithTheTag])
"""
for linea in data:
aux = linea.decode('latin_1').encode('utf-8')
sintagma = aux.split('\t') # Here we separate the String in a list: [word, tag, ocurrences], word=sintagma[0], tag=sintagma[1], ocurrences=sintagma[2]
if (sintagma[0] != "Palabra" and sintagma[1] != "Tag"): #We are not interested in the first line of the file, this is the filter
if (diccionario.has_key(sintagma[0])): #Here we check if the word was included before in the dictionary
aux_list = diccionario.get(sintagma[0]) #We know the name already exists in the dic, so we create a List for every value
aux_list.append([sintagma[1], sintagma[2]]) #We add to the list the tag and th ocurrences for this concrete word
diccionario.update({sintagma[0]:aux_list}) #Update the value with the new list (new list = previous list + new appended element to the list)
else: #If in the dic do not exist the key, que add the values to the empty list (no need to append)
aux_list_else = ([sintagma[1],sintagma[2]])
diccionario.update({sintagma[0]:aux_list_else})
"""
Here we create a new dictionary based on the dictionary created before, in this new dictionary (diccionario2) we want to keep the next
information:
Key: word
Value: List ([suggestedTag, #ocurrencesOfTheWordInTheDocument, probability])
For retrieve the information from diccionario, we have to keep in mind:
In case we have more than 1 Tag associated to a word (keyword ), we access to the first tag with keyword[0], and for ocurrencesWithTheTag with keyword[1],
from the second case and forward, we access to the information by this way:
diccionario.get(keyword)[2][0] -> with this we access to the second tag
diccionario.get(keyword)[2][1] -> with this we access to the second ocurrencesWithTheTag
diccionario.get(keyword)[3][0] -> with this we access to the third tag
...
..
.
etc.
"""
diccionario2 = dict.fromkeys(diccionario.keys())#We create a dictionary with the keys from diccionario and we set all the values to None
with open("estimation.txt", "w") as outfile:
for keyword in diccionario:
tagSugerido = unicode(diccionario.get(keyword[0]).decode('utf-8')) #tagSugerido is the tag with more ocurrences for a concrete keyword
maximo = float(diccionario.get(keyword)[1]) #maximo is a variable for the maximum number of ocurrences in a keyword
if ((len(diccionario.get(keyword))) > 2): #in case we have > 2 tags for a concrete word
suma = float(diccionario.get(keyword)[1])
for i in range (2, len(diccionario.get(keyword))):
suma += float(diccionario.get(keyword)[i][1])
if (diccionario.get(keyword)[i][1] > maximo):
tagSugerido = unicode(diccionario.get(keyword)[i][0]).decode('utf-8'))
maximo = float(diccionario.get(keyword)[i][1])
probabilidad = float(maximo/suma);
diccionario2.update({keyword:([tagSugerido, suma, probabilidad])})
else:
diccionario2.update({keyword:([diccionario.get(keyword)[0],diccionario.get(keyword)[1], 1])})
outfile.write(u"{}\t{}\n".format(keyword, tagSugerido).encode("utf-8"))
所需的输出如下所示：
keyword(String) tagSugerido(String):
Hello NC
Friend N
Run V
...etc
冲突线是：
outfile.write(u"{}\t{}\n".format(keyword, str(tagSugerido)).encode("utf-8"))
谢谢。
答案 0 :(得分：2)
由于您没有提供简单明了的简洁代码来说明您的问题，我们只是就错误应该给您一般性建议：
如果您收到解码错误，那么tagSugerido将被读取为ASCII而不是Unicode。要解决这个问题，你应该这样做：
tagSugerido = unicode(diccionario.get(keyword[0]).decode('utf-8'))
将其存储为unicode。
然后你可能会在write()阶段遇到编码错误，你应该按照以下方式修改你的写作：
outfile.write(u"{}\t{}\n".format(keyword, str(tagSugerido)).encode("utf-8"))
应该是：
outfile.write(u"{}\t{}\n".format(keyword, tagSugerido.encode("utf-8")))
我经常回答一个非常相似的问题moments ago。使用unicode字符串时， 切换到python3 ，它会让您的生活更轻松！
如果你还不能切换到python3，你可以使用python-future import语句使你的python2表现得几乎像python3：
from __future__ import absolute_import, division, print_function, unicode_literals
N.B。：而不是：
file=open("lexic.txt", "r") # abrimos el fichero lexic (nuestro modelo) (probar con este)
data=file.readlines()
file.close()
在读取线失败时无法正确关闭文件描述符，你最好这样做：
with open("lexic.txt", "r") as f:
data=f.readlines()
即使失败也会始终关闭文件。
N.B.2：避免使用 file ，因为这是一个你正在遮蔽的python类型，但使用f或lexic_file ...
答案 1 :(得分：2)
像zmo建议：
outfile.write(u"{}\t{}\n".format(keyword, str(tagSugerido)).encode("utf-8"))
应该是：
outfile.write(u"{}\t{}\n".format(keyword, tagSugerido.encode("utf-8")))
关于Python 2中unicode的说明
您的软件应该只在内部使用unicode字符串，在输出时转换为特定的编码。
要防止一遍又一遍地犯同样的错误，你应该确保你理解 ascii 和 utf-8 编码之间以及 str之间的区别和Python中的 unicode 对象。
ASCII和UTF-8编码之间的区别：
Ascii只需要一个字节来表示ascii字符集/编码中的所有可能字符。 UTF-8最多需要四个字节来表示完整的字符集。
ascii (default)
1 If the code point is < 128, each byte is the same as the value of the code point.
2 If the code point is 128 or greater, the Unicode string can’t be represented in this encoding. (Python raises a UnicodeEncodeError exception in this case.)
utf-8 (unicode transformation format)
1 If the code point is <128, it’s represented by the corresponding byte value.
2 If the code point is between 128 and 0x7ff, it’s turned into two byte values between 128 and 255.
3 Code points >0x7ff are turned into three- or four-byte sequences, where each byte of the sequence is between 128 and 255.
str和unicode对象之间的区别：
你可以说str是baiscally字节字符串而unicode是unicode字符串。两者都可以使用不同的编码，如ascii或utf-8。
str vs. unicode
1 str = byte string (8-bit) - uses \x and two digits
2 unicode = unicode string - uses \u and four digits
3 basestring
/\
/ \
str unicode
如果您遵循一些简单的规则，您应该可以处理不同编码的str / unicode对象，如ascii或utf-8或您必须使用的任何编码：
Rules
1 encode(): Gets you from Unicode -> bytes
encode([encoding], [errors='strict']), returns an 8-bit string version of the Unicode string,
2 decode(): Gets you from bytes -> Unicode
decode([encoding], [errors]) method that interprets the 8-bit string using the given encoding
3 codecs.open(encoding=”utf-8″): Read and write files directly to/from Unicode (you can use any encoding, not just utf-8, but utf-8 is most common).
4 u”: Makes your string literals into Unicode objects rather than byte sequences.
5 unicode(string[, encoding, errors])
警告：不要在字节上使用encode（）或在Unicode对象上使用decode（）
再次：软件应该只在内部使用Unicode字符串，在输出时转换为特定的编码。