Question

我正在尝试切片json文件，文件看起来像这样：

{"price": 17.95, "categories": [["Musical Instruments", "Instrument Accessories", "General Accessories", "Sheet Music Folders"]], "imUrl": "http://ecx.images-amazon.com/images/I/41EpRmh8MEL._SY300_.jpg", "title": "Six Sonatas For Two Flutes Or Violins, Volume 2 (#4-6)", "salesRank": {"Musical Instruments": 207315}, "asin": "0006428320"}
{"description": "Composer: J.S. Bach.Peters Edition.For two violins and pianos.", "related": {"also_viewed": ["B0058DK7RA"], "buy_after_viewing": ["B0058DK7RA"]}, "categories": [["Musical Instruments"]], "brand": "", "imUrl": "http://ecx.images-amazon.com/images/I/41m6ygCqc8L._SY300_.jpg", "title": "Double Concerto in D Minor By Johann Sebastian Bach. Edited By David Oistrach. For Violin I, Violin Ii and Piano Accompaniment. Urtext. Baroque. Medium. Set of Performance Parts. Solo Parts, Piano Reduction and Introductory Text. BWV 1043.", "salesRank": {"Musical Instruments": 94593}, "asin": "0014072149", "price": 18.77}
{"asin": "0041291905", "categories": [["Musical Instruments", "Instrument Accessories", "General Accessories", "Sheet Music Folders"]], "imUrl": "http://ecx.images-amazon.com/images/I/41maAqSO9hL._SY300_.jpg", "title": "Hal Leonard Vivaldi Four Seasons for Piano (Original Italian Text)", "salesRank": {"Musical Instruments": 222972}, "description": "Vivaldi's famous set of four violin concertos certainly ranks among the all-time top ten classical favorites. Features include an introduction about the history of The Four Seasons and Vivaldi's original vivid Italian score markings. A must for classical purists."}

你可以看到字段没有严格排列在所有行中，我只需要部分字段。所以我写了这段代码：

import json, csv

infile = open("sample_output.strict", "r")
outfile = open("output.csv", "w")
writer = csv.writer(outfile)

fileds = ["asin","price"]    
for product in json.loads(infile.read()):
    line = []
    for f in fields:
        if product.has_key(f):
            line.append(product[f])
        else:
            line.append("")
        writer.write(line)

我收到以下错误信息：

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-14-3e335b184eea> in <module>()
      6 
      7 fileds = ["asin","price"]
----> 8 for product in json.loads(infile.read()):
      9     line = []
     10     for f in fields:

C:\Anaconda3\lib\json\__init__.py in loads(s, encoding, cls, object_hook, parse_float, parse_int, parse_constant, object_pairs_hook, **kw)
    316             parse_int is None and parse_float is None and
    317             parse_constant is None and object_pairs_hook is None and not kw):
--> 318         return _default_decoder.decode(s)
    319     if cls is None:
    320         cls = JSONDecoder

C:\Anaconda3\lib\json\decoder.py in decode(self, s, _w)
    344         end = _w(s, end).end()
    345         if end != len(s):
--> 346             raise ValueError(errmsg("Extra data", s, end, len(s)))
    347         return obj
    348 

ValueError: Extra data: line 2 column 1 - line 3 column 617 (char 339 - 1581)

Answer 1

你所拥有的是json的行，而不是单个json文档。更改程序以读取每一行并将其转换为json，然后以这种方式查看每个文档。这实际上很常见，我收到的数据都是以这种格式加载的。

如果您最终处理大文件，那么逐行执行将为您节省大量内存。

import json, csv

with open("sample_output.strict", "r") as infile:
    with open("output.csv", "w") as outfile:
        writer = csv.writer(outfile)

        fields = ["asin","price"]  
        for json_line in infile:  
            product = json.loads(json_line)
            line = []
            for f in fields:
                if product.has_key(f):
                    line.append(product[f])
                else:
                    line.append("")
            writer.writerow(line)

Answer 2

您的输入json文件格式错误。这就是你看到这个错误的原因。简而言之，你不能拥有多个JSON＆＃34;对象＆＃34;在一个文件中。但是，在您的情况下，可以看到3个哈希值。对此的一个解决方案是使用这样的顶级列表来包含它们：

[
    {"price": 17.95, "categories": [["Musical Instruments", "Instrument Accessories", "General Accessories", "Sheet Music Folders"]], "imUrl": "http://ecx.images-amazon.com/images/I/41EpRmh8MEL._SY300_.jpg", "title": "Six Sonatas For Two Flutes Or Violins, Volume 2 (#4-6)", "salesRank": {"Musical Instruments": 207315}, "asin": "0006428320"},
    {"description": "Composer: J.S. Bach.Peters Edition.For two violins and pianos.", "related": {"also_viewed": ["B0058DK7RA"], "buy_after_viewing": ["B0058DK7RA"]}, "categories": [["Musical Instruments"]], "brand": "", "imUrl": "http://ecx.images-amazon.com/images/I/41m6ygCqc8L._SY300_.jpg", "title": "Double Concerto in D Minor By Johann Sebastian Bach. Edited By David Oistrach. For Violin I, Violin Ii and Piano Accompaniment. Urtext. Baroque. Medium. Set of Performance Parts. Solo Parts, Piano Reduction and Introductory Text. BWV 1043.", "salesRank": {"Musical Instruments": 94593}, "asin": "0014072149", "price": 18.77},
    {"asin": "0041291905", "categories": [["Musical Instruments", "Instrument Accessories", "General Accessories", "Sheet Music Folders"]], "imUrl": "http://ecx.images-amazon.com/images/I/41maAqSO9hL._SY300_.jpg", "title": "Hal Leonard Vivaldi Four Seasons for Piano (Original Italian Text)", "salesRank": {"Musical Instruments": 222972}, "description": "Vivaldi's famous set of four violin concertos certainly ranks among the all-time top ten classical favorites. Features include an introduction about the history of The Four Seasons and Vivaldi's original vivid Italian score markings. A must for classical purists."}
]

然后您可以使用以下代码片段进行切片：

import json, csv
infile = open("sample_output.strict", "r")
jsondata = json.loads(infile.read())
outfile = open("output.csv", "w")
writer = csv.writer(outfile)
fields = ["asin","price"]
for product in jsondata:
    line = []
    for f in fields:
        if f in product:
            line.append(product)
            break   # I assume you need to print only once per match!?
        else:
            line.append("")
    writer.write(line)

我不明白你尝试用csv输出做什么，所以我只是按原样复制它，以演示修复。

如何切片json文件，只提取部分字段

2 个答案: