如何将JSON完全展平以提取到CSV

时间:2018-12-01 03:01:03

标签: python json csv export-to-csv

我有一个json文件,已使用通过stackoverflow找到的示例进行了部分扁平化。但是,我似乎仍然没有完全扁平化的JSON,因此无法提取到CSV中。我在这里粘贴我的完整代码以及错误。任何帮助将不胜感激!

postcodes = ['P5N2J1', 'P5N2J2','P5N2J3']
response = []

for postcode in postcodes:
    rr = requests.get('https://represent.opennorth.ca/postcodes/{}'.format(postcode))
    data = json.loads(rr.text)
    response.append(data)

#print(response)

x = response 

def flatten(items):
    non_list_items = []

    for item in items:
        if isinstance(item, list):
            for inner_item in flatten(item):
                yield inner_item
        else:
            non_list_items.append(item)

    yield non_list_items

flatten(x)

x

输出:

{'representatives_centroid': [{'url': 'https://www.ola.org/en/members/all/guy-bourgouin',
   'email': 'GBourgouin-QP@ndp.on.ca',
   'personal_url': '',
   'related': {'boundary_url': '/boundaries/ontario-electoral-districts-representation-act-2015/mushkegowuk-james-bay/',
    'representative_set_url': '/representative-sets/ontario-legislature/'},
   'offices': [{'tel': '1 416 326-7351', 'type': 'legislature'},
    {'tel': '1 705 335-6400', 'type': 'constituency'}],
   'photo_url': 'https://www.ola.org/sites/default/files/member/profile-photo/guy_bourgouin.jpg',
   'representative_set_name': 'Legislative Assembly of Ontario',
   'party_name': 'New Democratic Party of Ontario',
   'gender': '',
   'district_name': 'Mushkegowuk—James Bay',
   'source_url': 'https://www.ola.org/en/members/current/contact-information',
   'last_name': 'Bourgouin',
   'name': 'Guy Bourgouin',
   'first_name': 'Guy',
   'extra': {},
   'elected_office': 'MPP'},
  {'url': 'http://www.parl.gc.ca/Parliamentarians/en/members/Carol-Hughes(31289)',
   'email': 'carol.hughes@parl.gc.ca',
   'personal_url': 'http://carolhughes.ndp.ca',
   'related': {'boundary_url': '/boundaries/federal-electoral-districts/35002/',
    'representative_set_url': '/representative-sets/house-of-commons/'},
   'offices': [{'postal': 'House of Commons\nOttawa ON  K1A 0A6',
     'tel': '1 613 996-5376',
     'fax': '1 613 995-6661',
     'type': 'legislature'},
    {'postal': '289-A Hillside Drive South (Main Office)\n\nElliot Lake ON  P5A 1N7',
     'tel': '1 705 848-8080',
     'fax': '1 705 848-1818',
     'type': 'constituency'},
    {'postal': '12-B Byng Road\n\nKapuskasing ON  P5N 1W3',
     'tel': '1 705 335-5533',
     'fax': '1 705 337-6869',
     'type': 'constituency'}],
   'photo_url': 'http://www.ourcommons.ca/Parliamentarians/Images/OfficialMPPhotos/42/HughesCarol_NDP.jpg',
   'representative_set_name': 'House of Commons',
   'party_name': 'NDP',
   'gender': '',
   'district_name': 'Algoma—Manitoulin—Kapuskasing',
   'source_url': 'http://www.parl.gc.ca/Parliamentarians/en/members?view=ListAll',
   'last_name': 'Hughes',
   'name': 'Carol Hughes',
   'first_name': 'Carol',
   'extra': {},
   'elected_office': 'MP'},
  {'url': '',
   'email': 'general@kapuskasing.ca',
   'personal_url': '',
   'related': {'boundary_url': '/boundaries/census-subdivisions/3556066/',
    'representative_set_url': '/representative-sets/campaign-set-4/'},
   'offices': [{'tel': '1 705 335-2341'}],
   'photo_url': '',
   'representative_set_name': 'Municipal councils without subdivisions in Canada',
   'party_name': '',
   'gender': 'M',
   'district_name': 'Kapuskasing',
   'source_url': 'http://www.kapuskasing.ca/en/governing/Mayor-and-Council.aspx',
   'last_name': 'Spacek',
   'name': 'Alan Spacek',
   'first_name': 'Alan',
   'extra': {'address': '88 Riverside Drive, Kapuskasing,\nON, P5N 1B3',
    'organization': 'Kapuskasing, Town of',
    'validation': 'TRUE',
    'office_type': 'legislature'},
   'elected_office': 'Mayor'}],
 'boundaries_centroid': [{'url': '/boundaries/federal-electoral-districts/35002/',
   'name': 'Algoma—Manitoulin—Kapuskasing',
   'external_id': '35002',
   'boundary_set_name': 'Federal electoral district',
   'related': {'boundary_set_url': '/boundary-sets/federal-electoral-districts/'}},
  {'url': '/boundaries/ontario-electoral-districts-representation-act-2015/mushkegowuk-james-bay/',
   'name': 'Mushkegowuk—James Bay',
   'external_id': '124',
   'boundary_set_name': 'Ontario electoral district',
   'related': {'boundary_set_url': '/boundary-sets/ontario-electoral-districts-representation-act-2015/'}},
  {'url': '/boundaries/ontario-electoral-districts-representation-act-2005/timmins-james-bay/',
   'name': 'Timmins—James Bay',
   'external_id': '93',
   'boundary_set_name': 'Ontario electoral district',
   'related': {'boundary_set_url': '/boundary-sets/ontario-electoral-districts-representation-act-2005/'}},
  {'url': '/boundaries/federal-electoral-districts-2003-representation-order/35002/',
   'name': 'Algoma—Manitoulin—Kapuskasing',
   'external_id': '35002',
   'boundary_set_name': 'Federal electoral district',
   'related': {'boundary_set_url': '/boundary-sets/federal-electoral-districts-2003-representation-order/'}},
  {'url': '/boundaries/census-divisions/3556/',
   'name': 'Cochrane',
   'external_id': '3556',
   'boundary_set_name': 'Census division',
   'related': {'boundary_set_url': '/boundary-sets/census-divisions/'}},
  {'url': '/boundaries/census-subdivisions/3556066/',
   'name': 'Kapuskasing',
   'external_id': '3556066',
   'boundary_set_name': 'Census subdivision',
   'related': {'boundary_set_url': '/boundary-sets/census-subdivisions/'}}],
 'boundaries_concordance': [],
 'province': 'ON',
 'code': 'P5N2J1',
 'centroid': {'coordinates': [-82.40755, 49.42085], 'type': 'Point'},
 'city': 'Kapuskasing'}

提取为CSV:

f = csv.writer(open("sample.csv","w+"))

f.writerow(["name", "elected_office", "district_name", "code"])
for x in x:
    f.writerow([x["representatives_centroid"]["name"],
                x["representatives_centroid"]["elected_office"],
                x["representatives_centroid"]["district_name"],
                x["code"]
               ])

错误消息:


TypeError                                 Traceback (most recent call last)
<ipython-input-55-3da67b2acc45> in <module>()
      3 f.writerow(["name", "elected_office", "district_name", "code"])
      4 for x in x:
----> 5     f.writerow([x["representatives_centroid"]["name"],
      6                 x["representatives_centroid"]["elected_office"],
      7                 x["representatives_centroid"]["district_name"],

TypeError: string indices must be integers

1 个答案:

答案 0 :(得分:0)

由于x["representatives_centriod"]返回list而不是dict,因此可以通过添加嵌套循环来存储数据。此外,"code"包含在x中,而不包含在x["representatives_centriod"]中。因此,您应该在嵌套循环之前存储"code"数据。

例如,

for i in x:
    code = i['code']
    for j in i['representatives_centroid']:
         f.writerow([j['name'], j['elected_office'], j['district_name'], code])