Question

我整天都在为这个而苦苦挣扎，我想转向一个csv。

它代表在UK Company House API中编号为“ OC418979”的公司的相关人员。

我已经将json截断为在“ items”中仅包含2个对象。

我想得到的是这样的csv

OC418979, country_of_residence, officer_role, appointed_on
OC418979, country_of_residence, officer_role, appointed_on
OC418979, country_of_residence, officer_role, appointed_on
OC418979, country_of_residence, officer_role, appointed_on
...

有2种额外的复杂性：“官员”有2种类型，有些是人，有些是公司，因此并非所有关键人物都存在于对方中，反之亦然。我希望这些条目为“ null”。第二个复杂度是那些嵌套的对象，例如“名称”，其中包含逗号！或地址，其中包含几个子对象（我想我可以用熊猫将其展平）。

{
  "total_results": 13,
  "resigned_count": 9,
  "links": {
    "self": "/company/OC418979/officers"
  },
  "items_per_page": 35,
  "etag": "bc7955679916b089445c9dfb4bc597aa0daaf17d",
  "kind": "officer-list",
  "active_count": 4,
  "inactive_count": 0,
  "start_index": 0,
  "items": [
    {
      "officer_role": "llp-designated-member",
      "name": "BARRICK, David James",
      "date_of_birth": {
        "year": 1984,
        "month": 1
      },
      "appointed_on": "2017-09-15",
      "country_of_residence": "England",
      "address": {
        "country": "United Kingdom",
        "address_line_1": "Old Gloucester Street",
        "locality": "London",
        "premises": "27",
        "postal_code": "WC1N 3AX"
      },
      "links": {
        "officer": {
          "appointments": "/officers/d_PT9xVxze6rpzYwkN_6b7og9-k/appointments"
        }
      }
    },
    {
      "links": {
        "officer": {
          "appointments": "/officers/M2Ndc7ZjpyrjzCXdFZyFsykJn-U/appointments"
        }
      },
      "address": {
        "locality": "Tadcaster",
        "country": "United Kingdom",
        "address_line_1": "Westgate",
        "postal_code": "LS24 9AB",
        "premises": "5a"
      },
      "identification": {
        "legal_authority": "UK",
        "identification_type": "non-eea",
        "legal_form": "UK"
      },
      "name": "PREMIER DRIVER LIMITED",
      "officer_role": "corporate-llp-designated-member",
      "appointed_on": "2017-09-15"
    }
  ]
}

我一直在做的是创建新的json对象，这样提取我需要的字段：

{officer_address:.items[]?.address, appointed_on:.items[]?.appointed_on, country_of_residence:.items[]?.country_of_residence, officer_role:.items[]?.officer_role, officer_dob:items.date_of_birth, officer_nationality:.items[]?.nationality, officer_occupation:.items[]?.occupation}

但是查询运行了几个小时-我敢肯定有一种更快的方法。

现在，我正在尝试这种新方法-创建一个json，其根是公司编号，并作为参数列出其人员名单。

{(.links.self | split("/")[2]): .items[]}

Answer 1

使用jq，可以更轻松地从将要共享的顶级对象中提取值并生成所需的行。您需要将浏览项目的时间限制为最多一次。

$ jq -r '(.links.self | split("/")[2]) as $companyCode 
   | .items[]
   | [ $companyCode, .country_of_residence, .officer_role, .appointed_on ]
   | @csv
' input.json

Answer 2

好吧，您想扫描人员列表，从那里提取一些字段（如果存在）并以csv格式编写。

第一部分是从json中提取数据。假设您加载的是一个data Python对象，您将：

print(data['items'][0]['officer_role'], data['items'][0]['appointed_on'],
      data['items'][0]['country_of_residence'])

给予：

llp-designated-member 2017-09-15 England

该将所有内容与csv模块放在一起了：

import csv
...
with open('output.csv', 'w', newline='') as fd:
    wr = csv.writer(fd)
    for officer in data['items']:
        _ = wr.writerow(('OC418979',
                 officer.get('country_of_residence',''),
                 officer.get('officer_role', ''),
                 officer.get('appointed_on', '')
                 ))

如果不存在键，则字典中的get方法允许使用默认值（此处为空字符串），并且csv模块确保如果字段包含逗号，则使用将用引号引起来。

使用您的示例输入，它会给出：

OC418979,England,llp-designated-member,2017-09-15
OC418979,,corporate-llp-designated-member,2017-09-15

重塑jq嵌套文件并制作csv

2 个答案: