Question

我有一个非常大的字典，我想存储为json，并在需要时放入内存。我可以将字典转换为json并存储，但是当我再次读取它时，代码崩溃时出现额外数据错误。这是代码和错误：

In [43]: with io.open('big_dict_utf.json' ,"w" , encoding="utf8") as json_file:
             json_file.write(unicode(json.dumps(final_dict, json_file, ensure_ascii=False)))


In [9]: with io.open('big_dict_utf.json' ,"r" , encoding="utf8") as json_file:
    d = json.load(json_file)
     ...:     
   ---------------------------------------------------------------------------
    ValueError                                Traceback (most recent call last)
    <ipython-input-9-78876f8fb69d> in <module>()
       1 with io.open('big_dict_utf.json' ,"r" , encoding="utf8") as json_file:
  ----> 2     d = json.load(json_file)
     3 

   /usr/lib/python2.7/json/__init__.pyc in load(fp, encoding, cls, object_hook, parse_float, parse_int, parse_constant, object_pairs_hook, **kw)
276         parse_float=parse_float, parse_int=parse_int,
277         parse_constant=parse_constant, object_pairs_hook=object_pairs_hook,  
 --> 278         **kw)
279 
280 

  /usr/lib/python2.7/json/__init__.pyc in loads(s, encoding, cls, object_hook, parse_float, parse_int, parse_constant, object_pairs_hook, **kw)
324             parse_int is None and parse_float is None and
325             parse_constant is None and object_pairs_hook is None and not kw):
--> 326         return _default_decoder.decode(s)
327     if cls is None:
328         cls = JSONDecoder

  /usr/lib/python2.7/json/decoder.pyc in decode(self, s, _w)
366         end = _w(s, end).end()
367         if end != len(s):
--> 368             raise ValueError(errmsg("Extra data", s, end, len(s)))
369         return obj
370 

 ValueError: Extra data: line 1 column -1686421006 - line 1 column 2608546290 (char -1686421006 - 2608546290)

该文件可能包含unicode字符。有关如何修复或调试它的任何建议？另外，我的字典是defaultdict（dict）。

-Amit

在python中加载一个非常大的json文件时的ValueError

0 个答案: