Question

我的IBM cloudant已连接到python。我需要从cloudant到python环境获得嵌套的JSON数据。我的目标是获取必要的数据并创建一个新的数据帧。

这里是嵌套的JSON

   {
  "_id": "1",
  "topic": "",
  "payload": {
    "devices": [
      {
        "_id": "70",
        "last_date": 1499503900,
        "modules": [
          {
            "_id": "0",
            "last_seen": 1497151699,
            "dashboard_data": {
              "time_utc": 1497150981,
              "Temperature": 41.8,
              "Humidity": 24,
              "date_max_temp": 1497150519,
              "date_min_temp": 1497146892,
              "min_temp": 13.9,
              "max_temp": 45.6
            },
            "data_type": [
              "Temperature",
              "Humidity"
            ],
            "module_name": "X",
            "last_setup": 1487170865,
            "battery_vp": 3474,
            "battery_percent": 0,
            "rf_status": 79,
            "firmware": 44
          },
          {
            "_id": "03",
            "last_message": 1499503897,
            "last_seen": 1499503897,
            "dashboard_data": {
              "time_utc": 1499503795,
              "Temperature": 19.9,
              "temp_trend": "stable",
              "Humidity": 49,
              "CO2": 390,
              "date_max_temp": 1499468625,
              "date_min_temp": 1499490158,
              "min_temp": 19.4,
              "max_temp": 21.2
            },
            "data_type": [
              "Temperature",
              "CO2",
              "Humidity"
            ],
            "module_name": "Y",
            "last_setup": 1487173334,
            "battery_vp": 5484,
            "battery_percent": 71,
            "rf_status": 70,
            "firmware": 44
          },
          {
            "_id": "03",
            "last_message": 1499503897,
            "last_seen": 1499503852,
            "dashboard_data": {
              "time_utc": 1499503801,
              "Temperature": 19.7,
              "temp_trend": "stable",
              "Humidity": 51,
              "CO2": 414,
              "date_max_temp": 1499468631,
              "date_min_temp": 1499489242,
              "min_temp": 18.5,
              "max_temp": 20.7
            },
            "data_type": [
              "Temperature",
              "CO2",
              "Humidity"
            ],
            "module_name": "M",
            "last_setup": 1487173386,
            "battery_vp": 5518,
            "battery_percent": 73,
            "rf_status": 83,
            "firmware": 44
          },
          {
            "_id": "03",
            "last_message": 1499503897,
            "last_seen": 1499503872,
            "dashboard_data": {
              "time_utc": 1499503820,
              "Temperature": 22,
              "temp_trend": "stable",
              "Humidity": 49,
              "CO2": 419,
              "date_max_temp": 1499468599,
              "date_min_temp": 1499495002,
              "min_temp": 22,
              "max_temp": 22.9
            },
            "data_type": [
              "Temperature",
              "CO2",
              "Humidity"
            ],
            "module_name": "N",
            "last_setup": 1487173462,
            "battery_vp": 5494,
            "battery_percent": 72,
            "rf_status": 77,
            "firmware": 44
          }
        ],
        "place": {
          "country": "GB",
        },
        "type": "NAMain",
        "read_only": "true",
        "dashboard_data": {
          "AbsolutePressure": 1010.5,
          "Noise": 45,
          "Temperature": 20.5,
          "temp_trend": "stable",
          "Humidity": 52,
          "Pressure": 1010.5,
          "pressure_trend": "up",
          "CO2": 406,
          "date_max_temp": 1499468648,
          "date_min_temp": 1499500180,
          "min_temp": 20.4,
          "max_temp": 22.1
        },
        "data_type": [
          "Temperature",
          "CO2",
          "Humidity",
          "Noise",
          "Pressure"
        ],
        "co2_calibrating": "false",
        "date_setup": 1487170943,
        "last_setup": 1487170943,
        "module_name": "K",
        "firmware": 124,
        "last_upgrade": 1487170889,
        "wifi_status": 56
      }
    ]
  }
}

我有近50000个像这样的json数据。

我的目标是创建一个新的pandas数据框，其中包括last_date开头和Temperature每个_id。

我试过了：

import cloudant
import pandas as pd
import json

database2 = client['net']

for item in database2:
    print(item['payload']['devices'][0]['last_date'])

但它给了我

KeyError: 'payload'

其次是：

wanted = ['las_date', 'module_name']
for item in database2:
    if not 'las_date' in item:
        continue
    data = item['las_date']
    for key in wanted:
        if key in details:
            print(key, ':', json.dumps(details[key], indent=4))

但我得到了这个：

HTTPError: 429 Client Error: Too Many Requests

Answer 1

在更正了json数据后，我尝试了

item['payload']['devices'][0]['last_date']

我输出为：

1499503900

Answer 2

您看到的错误：

HTTPError: 429 Client Error: Too Many Requests

是Cloudant在IBM Cloud上受到速率限制的结果。 Cloudant（“Lite”）的免费版本允许每秒100次查找，10次写入和5次查询，之后您将收到上述错误。

您可以从Cloudant信息中心增加限制（需要付费）。您使用的Cloudant Python库能够重试 - 退回429错误，这可能会缓解您的问题。见

http://python-cloudant.readthedocs.io/en/latest/getting_started.html?highlight=429

解析嵌套的json并在cloudant

2 个答案: