pyrouge的UnicodeDecodeError

时间:2018-08-10 16:09:05

标签: python-3.x summarization precision-recall rouge

我想用pyrouge评估我的摘要。 pyrouge是ROUGE摘要评估包的python包装器。

我的问题是当我想使用pyrouge评估我的摘要时。我写了这些命令:

from pyrouge import Rouge155

r = Rouge155()
r.system_dir = '/Users/raghad/Desktop/Articles/system_summaries/'
r.model_dir = '/Users/raghad/Desktop/Articles/model_summaries/'
r.system_filename_pattern = 'summary.(\d+).txt'
r.model_filename_pattern = 'summary.[A-Z].#ID#.txt'

output = r.convert_and_evaluate()
print(output)
output_dict = r.output_to_dict(output)

但我收到此错误:

2018-08-10 19:05:34,194 [MainThread  ] [INFO ]  Writing summaries.
2018-08-10 19:05:34,197 [MainThread  ] [INFO ]  Processing summaries. Saving system files to /var/folders/yq/dcjh2lr14676s_8kl6s6vjgc0000gn/T/tmptyzxws4q/system and model files to /var/folders/yq/dcjh2lr14676s_8kl6s6vjgc0000gn/T/tmptyzxws4q/model.
2018-08-10 19:05:34,197 [MainThread  ] [INFO ]  Processing files in /Users/raghad/Desktop/Articles/system_summaries/.
2018-08-10 19:05:34,197 [MainThread  ] [INFO ]  Processing .DS_Store.
Traceback (most recent call last):
  File "rouge3.py", line 12, in <module>
    output = r.convert_and_evaluate()
  File "/usr/local/lib/python3.6/site-packages/pyrouge/Rouge155.py", line 360, in convert_and_evaluate
    self.__write_summaries()
  File "/usr/local/lib/python3.6/site-packages/pyrouge/Rouge155.py", line 489, in __write_summaries
    self.__process_summaries(self.convert_summaries_to_rouge_format)
  File "/usr/local/lib/python3.6/site-packages/pyrouge/Rouge155.py", line 482, in __process_summaries
    process_func(self._system_dir, new_system_dir)
  File "/usr/local/lib/python3.6/site-packages/pyrouge/Rouge155.py", line 202, in convert_summaries_to_rouge_format
    input_dir, output_dir, Rouge155.convert_text_to_rouge_format)
  File "/usr/local/lib/python3.6/site-packages/pyrouge/utils/file_utils.py", line 29, in process
    input_string = f.read()
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/codecs.py", line 700, in read
    return self.reader.read(size)
  File "/usr/local/Cellar/python/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/python3.6/codecs.py", line 503, in read
    newchars, decodedbytes = self.decode(data, self.errors)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x86 in position 23: invalid start byte

我该怎么办? 我的默认编码是UTF-8

1 个答案:

答案 0 :(得分:1)

如果在MacO上运行此文件,则同一文件夹中将存在一个名为“ .DS_Store”的文件(隐藏文件)。删除它。

请参阅:https://github.com/bheinzerling/pyrouge/issues/22

如果您使用的是其他系统,请在同一目录中查找隐藏文件。