基本上,我有从某个Weather API中提取的原始数据。通过SQL查询,数据被格式化为带有以下几列的数据框:纬度(lats),经度(lngs),日期和“ blob”。
blob是嵌套在2层中的JSON数据。您将在下面看到的数据首先从第一层的“每日”摘要开始,到第二层的“每小时”摘要24小时摘要。
lats lngs date blob
-46 168 2015-01-31 {"daily": {"apparentTemperatureMaxTime": 1422680400, "temperatureMax": 21.33, "temperatureMinTime": 1422615600, "temperatureMin": 16.06, "icon": "clear-day", "apparentTemperatureMax": 21.33, "summary": "Clear throughout the day.", "pressure": 1010.91, "temperatureMaxTime": 1422680400, "humidity": 0.81, "dewPoint": 15.14, "sunsetTime": 1422692673, "precipType": "rain", "windSpeed": 3.01, "apparentTemperatureMin": 16.06, "sunriseTime": 1422639631, "apparentTemperatureMinTime": 1422615600, "time": 1422615600, "visibility": 16.09, "windBearing": 75, "moonPhase": 0.38}, "hourly": [{"apparentTemperature": 16.06, "windSpeed": 4.29, "icon": "clear-night", "temperature": 16.06, "summary": "Clear", "pressure": 1015.05, "humidity": 0.89, "dewPoint": 14.23, "precipType": "rain", "time": 1422615600, "visibility": 16.09, "windBearing": 97}, {"apparentTemperature": 16.17, "windSpeed": 4.22, "icon": "clear-night", "temperature": 16.17, "summary": "Clear", "pressure": 1014.91, "humidity": 0.88, "dewPoint": 14.19, "precipType": "rain", "time": 1422619200, "visibility": 16.09, "windBearing": 94}, {"apparentTemperature": 16.27, "windSpeed": 4.09, "icon": "clear-night", "temperature": 16.27, "summary": "Clear", "pressure": 1014.51, "humidity": 0.87, "dewPoint": 14.14, "precipType": "rain", "time": 1422622800, "visibility": 16.09, "windBearing": 87}, {"apparentTemperature": 16.36, "windSpeed": 4, "icon": "clear-night", "temperature": 16.36, "summary": "Clear", "pressure": 1013.94, "humidity": 0.86, "dewPoint": 14.09, "precipType": "rain", "time": 1422626400, "visibility": 16.09, "windBearing": 80}, {"apparentTemperature": 16.4, "windSpeed": 3.9, "icon": "clear-night", "temperature": 16.4, "summary": "Clear", "pressure": 1013.43, "humidity": 0.86, "dewPoint": 14.07, "precipType": "rain", "time": 1422630000, "visibility": 16.09, "windBearing": 75},
下面是1个日期的2组数据的字典。
{'lat': {0: -45, 1: -45}, 'lng': {0: 169, 1: 170}, 'date': {0: datetime.date(2015, 1, 1), 1: datetime.date(2015, 1, 1)}, 'blob': {0: {'daily': {'apparentTemperatureMaxTime': 1420088400, 'temperatureMax': 19.06, 'temperatureMinTime': 1420045200, 'temperatureMin': 7.86, 'icon': 'clear-day', 'apparentTemperatureMax': 19.06, 'summary': 'Clear throughout the day.', 'pressure': 1013.08, 'temperatureMaxTime': 1420088400, 'humidity': 0.61, 'dewPoint': 5.49, 'sunsetTime': 1420101288, 'precipType': 'rain', 'windSpeed': 3.18, 'apparentTemperatureMin': 6.76, 'sunriseTime': 1420045310, 'apparentTemperatureMinTime': 1420041600, 'time': 1420023600, 'visibility': 16, 'windBearing': 241, 'moonPhase': 0.36}, 'hourly': [{'apparentTemperature': 6.78, 'windSpeed': 6.98, 'icon': 'clear-night', 'temperature': 9.88, 'summary': 'Clear', 'pressure': 1005.77, 'humidity': 0.81, 'dewPoint': 6.74, 'precipType': 'rain', 'time': 1420023600, 'visibility': 12.59, 'windBearing': 208}, {'apparentTemperature': 7.23, 'windSpeed': 4.95, 'icon': 'clear-night', 'temperature': 9.7, 'summary': 'Clear', 'pressure': 1007.34, 'humidity': 0.81, 'dewPoint': 6.51, 'precipType': 'rain', 'time': 1420027200, 'visibility': 16.09, 'windBearing': 217}, {'apparentTemperature': 7.19, 'windSpeed': 4.13, 'icon': 'clear-night', 'temperature': 9.39, 'summary': 'Clear', 'pressure': 1008.35, 'humidity': 0.81, 'dewPoint': 6.29, 'precipType': 'rain', 'time': 1420030800, 'visibility': 16.09, 'windBearing': 226}, {'apparentTemperature': 6.96, 'windSpeed': 3.77, 'icon': 'clear-night', 'temperature': 9.06, 'summary': 'Clear', 'pressure': 1009.02, 'humidity': 0.82, 'dewPoint': 6.09, 'precipType': 'rain', 'time': 1420034400, 'visibility': 16.09, 'windBearing': 235}, {'apparentTemperature': 6.79, 'windSpeed': 3.38, 'icon': 'clear-night', 'temperature': 8.76, 'summary': 'Clear', 'pressure': 1009.81, 'humidity': 0.82, 'dewPoint': 5.83, 'precipType': 'rain', 'time': 1420038000, 'visibility': 16.09, 'windBearing': 243}, {'apparentTemperature': 6.76, 'windSpeed': 2.56, 'icon': 'clear-night', 'temperature': 8.29, 'summary': 'Clear', 'pressure': 1010.94, 'humidity': 0.82, 'dewPoint': 5.33, 'precipType': 'rain', 'time': 1420041600, 'visibility': 16.09, 'windBearing': 249}, {'apparentTemperature': 7.09, 'windSpeed': 1.6, 'icon': 'clear-night', 'temperature': 7.86, 'summary': 'Clear', 'pressure': 1012.19, 'humidity': 0.81, 'dewPoint': 4.76, 'precipType': 'rain', 'time': 1420045200, 'visibility': 16.09, 'windBearing': 255}, {'apparentTemperature': 8.08, 'windSpeed': 1.15, 'icon': 'clear-day', 'temperature': 8.08, 'summary': 'Clear', 'pressure': 1013.28, 'humidity': 0.78, 'dewPoint': 4.47, 'precipType': 'rain', 'time': 1420048800, 'visibility': 16.09, 'windBearing': 265}, {'apparentTemperature': 8.95, 'windSpeed': 1.62, 'icon': 'clear-day', 'temperature': 9.49, 'summary': 'Clear', 'pressure': 1014.17, 'humidity': 0.71, 'dewPoint': 4.59, 'precipType': 'rain', 'time': 1420052400, 'visibility': 16.09, 'windBearing': 265}, {'apparentTemperature': 11.57, 'windSpeed': 2.55, 'icon': 'clear-day', 'temperature': 11.57, 'summary': 'Clear', 'pressure': 1014.91, 'humidity': 0.63, 'dewPoint': 4.76, 'precipType': 'rain', 'time': 1420056000, 'visibility': 16.09, 'windBearing': 261}, {'apparentTemperature': 13.41, 'windSpeed': 3.3, 'icon': 'clear-day', 'temperature': 13.41, 'summary': 'Clear', 'pressure': 1015.39, 'humidity': 0.56, 'dewPoint': 4.79, 'precipType': 'rain', 'time': 1420059600, 'visibility': 16.09, 'windBearing': 260}, {'apparentTemperature': 14.68, 'windSpeed': 3.65, 'icon': 'clear-day', 'temperature': 14.68, 'summary': 'Clear', 'pressure': 1015.52, 'humidity': 0.52, 'dewPoint': 4.83, 'precipType': 'rain', 'time': 1420063200, 'visibility': 16.09, 'windBearing': 260}, {'apparentTemperature': 15.71, 'windSpeed': 3.81, 'icon': 'clear-day', 'temperature': 15.71, 'summary': 'Clear', 'pressure': 1015.39, 'humidity': 0.49, 'dewPoint': 4.96, 'precipType': 'rain', 'time': 1420066800, 'visibility': 16.09, 'windBearing': 261}, {'apparentTemperature': 16.59, 'windSpeed': 3.77, 'icon': 'clear-day', 'temperature': 16.59, 'summary': 'Clear', 'pressure': 1015.18, 'humidity': 0.47, 'dewPoint': 5.17, 'precipType': 'rain', 'time': 1420070400, 'visibility': 16.09, 'windBearing': 260}, {'apparentTemperature': 17.33, 'windSpeed': 3.4, 'icon': 'clear-day', 'temperature': 17.33, 'summary': 'Clear', 'pressure': 1014.95, 'humidity': 0.46, 'dewPoint': 5.55, 'precipType': 'rain', 'time': 1420074000, 'visibility': 16.09, 'windBearing': 253}, {'apparentTemperature': 17.92, 'windSpeed': 2.94, 'icon': 'clear-day', 'temperature': 17.92, 'summary': 'Clear', 'pressure': 1014.64, 'humidity': 0.46, 'dewPoint': 6.06, 'precipType': 'rain', 'time': 1420077600, 'visibility': 16.09, 'windBearing': 240}, {'apparentTemperature': 18.36, 'windSpeed': 2.81, 'icon': 'clear-day', 'temperature': 18.36, 'summary': 'Clear', 'pressure': 1014.31, 'humidity': 0.46, 'dewPoint': 6.39, 'precipType': 'rain', 'time': 1420081200, 'visibility': 16.09, 'windBearing': 229}, {'apparentTemperature': 18.78, 'windSpeed': 3.08, 'icon': 'clear-day', 'temperature': 18.78, 'summary': 'Clear', 'pressure': 1013.85, 'humidity': 0.44, 'dewPoint': 6.43, 'precipType': 'rain', 'time': 1420084800, 'visibility': 16.09, 'windBearing': 227}, {'apparentTemperature': 19.06, 'windSpeed': 3.5, 'icon': 'clear-day', 'temperature': 19.06, 'summary': 'Clear', 'pressure': 1013.37, 'humidity': 0.43, 'dewPoint': 6.29, 'precipType': 'rain', 'time': 1420088400, 'visibility': 16.09, 'windBearing': 230}, {'apparentTemperature': 18.78, 'windSpeed': 3.76, 'icon': 'clear-day', 'temperature': 18.78, 'summary': 'Clear', 'pressure': 1013.31, 'humidity': 0.43, 'dewPoint': 6.06, 'precipType': 'rain', 'time': 1420092000, 'visibility': 16.09, 'windBearing': 233}, {'apparentTemperature': 17.53, 'windSpeed': 3.78, 'icon': 'clear-day', 'temperature': 17.53, 'summary': 'Clear', 'pressure': 1014.01, 'humidity': 0.45, 'dewPoint': 5.54, 'precipType': 'rain', 'time': 1420095600, 'visibility': 16.09, 'windBearing': 238}, {'apparentTemperature': 15.72, 'windSpeed': 3.68, 'icon': 'clear-day', 'temperature': 15.72, 'summary': 'Clear', 'pressure': 1015.13, 'humidity': 0.48, 'dewPoint': 4.85, 'precipType': 'rain', 'time': 1420099200, 'visibility': 16.09, 'windBearing': 244}, {'apparentTemperature': 14.18, 'windSpeed': 3.29, 'icon': 'clear-night', 'temperature': 14.18, 'summary': 'Clear', 'pressure': 1016.13, 'humidity': 0.52, 'dewPoint': 4.51, 'precipType': 'rain', 'time': 1420102800, 'visibility': 16.09, 'windBearing': 250}, {'apparentTemperature': 13.23, 'windSpeed': 2.39, 'icon': 'clear-night', 'temperature': 13.23, 'summary': 'Clear', 'pressure': 1016.88, 'humidity': 0.57, 'dewPoint': 4.88, 'precipType': 'rain', 'time': 1420106400, 'visibility': 16.09, 'windBearing': 255}]}, 1: {'daily': {'apparentTemperatureMaxTime': 1420081200, 'temperatureMax': 18.18, 'temperatureMinTime': 1420045200, 'temperatureMin': 8.68, 'icon': 'clear-day', 'apparentTemperatureMax': 18.18, 'summary': 'Clear throughout the day.', 'pressure': 1013.16, 'temperatureMaxTime': 1420081200, 'humidity': 0.63, 'dewPoint': 6.58, 'sunsetTime': 1420101048, 'precipType': 'rain', 'windSpeed': 1.6, 'apparentTemperatureMin': 7.85, 'sunriseTime': 1420045069, 'apparentTemperatureMinTime': 1420041600, 'time': 1420023600, 'visibility': 16.06, 'windBearing': 232, 'moonPhase': 0.36}, 'hourly': [{'apparentTemperature': 11.77, 'windSpeed': 6.44, 'icon': 'clear-night', 'temperature': 11.77, 'summary': 'Clear', 'pressure': 1004.34, 'humidity': 0.77, 'dewPoint': 7.78, 'precipType': 'rain', 'time': 1420023600, 'visibility': 14.73, 'windBearing': 222}, {'apparentTemperature': 11.13, 'windSpeed': 5.33, 'icon': 'clear-night', 'temperature': 11.13, 'summary': 'Clear', 'pressure': 1006.11, 'humidity': 0.78, 'dewPoint': 7.52, 'precipType': 'rain', 'time': 1420027200, 'visibility': 16.09, 'windBearing': 218}, {'apparentTemperature': 10.48, 'windSpeed': 4.48, 'icon': 'clear-night', 'temperature': 10.48, 'summary': 'Clear', 'pressure': 1007.3, 'humidity': 0.79, 'dewPoint': 6.95, 'precipType': 'rain', 'time': 1420030800, 'visibility': 16.09, 'windBearing': 218}, {'apparentTemperature': 8.01, 'windSpeed': 3.65, 'icon': 'clear-night', 'temperature': 9.87, 'summary': 'Clear', 'pressure': 1008.18, 'humidity': 0.78, 'dewPoint': 6.27, 'precipType': 'rain', 'time': 1420034400, 'visibility': 16.09, 'windBearing': 222}, {'apparentTemperature': 7.87, 'windSpeed': 2.9, 'icon': 'clear-night', 'temperature': 9.42, 'summary': 'Clear', 'pressure': 1009.12, 'humidity': 0.78, 'dewPoint': 5.77, 'precipType': 'rain', 'time': 1420038000, 'visibility': 16.09, 'windBearing': 228}, {'apparentTemperature': 7.85, 'windSpeed': 2.18, 'icon': 'clear-night', 'temperature': 8.98, 'summary': 'Clear', 'pressure': 1010.33, 'humidity': 0.79, 'dewPoint': 5.55, 'precipType': 'rain', 'time': 1420041600, 'visibility': 16.09, 'windBearing': 235}, {'apparentTemperature': 8.07, 'windSpeed': 1.56, 'icon': 'clear-night', 'temperature': 8.68, 'summary': 'Clear', 'pressure': 1011.62, 'humidity': 0.8, 'dewPoint': 5.5, 'precipType': 'rain', 'time': 1420045200, 'visibility': 16.09, 'windBearing': 247}, {'apparentTemperature': 9.08, 'windSpeed': 1.31, 'icon': 'clear-day', 'temperature': 9.08, 'summary': 'Clear', 'pressure': 1012.76, 'humidity': 0.79, 'dewPoint': 5.6, 'precipType': 'rain', 'time': 1420048800, 'visibility': 16.09, 'windBearing': 265}, {'apparentTemperature': 10.71, 'windSpeed': 1.57, 'icon': 'clear-day', 'temperature': 10.71, 'summary': 'Clear', 'pressure': 1013.74, 'humidity': 0.72, 'dewPoint': 5.8, 'precipType': 'rain', 'time': 1420052400, 'visibility': 16.09, 'windBearing': 277}, {'apparentTemperature': 13.04, 'windSpeed': 2.04, 'icon': 'clear-day', 'temperature': 13.04, 'summary': 'Clear', 'pressure': 1014.59, 'humidity': 0.62, 'dewPoint': 5.82, 'precipType': 'rain', 'time': 1420056000, 'visibility': 16.09, 'windBearing': 280}, {'apparentTemperature': 15, 'windSpeed': 2.33, 'icon': 'clear-day', 'temperature': 15, 'summary': 'Clear', 'pressure': 1015.2, 'humidity': 0.53, 'dewPoint': 5.62, 'precipType': 'rain', 'time': 1420059600, 'visibility': 16.09, 'windBearing': 280}, {'apparentTemperature': 16.19, 'windSpeed': 2.35, 'icon': 'clear-day', 'temperature': 16.19, 'summary': 'Clear', 'pressure': 1015.48, 'humidity': 0.49, 'dewPoint': 5.54, 'precipType': 'rain', 'time': 1420063200, 'visibility': 16.09, 'windBearing': 277}, {'apparentTemperature': 17.03, 'windSpeed': 2.2, 'icon': 'clear-day', 'temperature': 17.03, 'summary': 'Clear', 'pressure': 1015.53, 'humidity': 0.47, 'dewPoint': 5.64, 'precipType': 'rain', 'time': 1420066800, 'visibility': 16.09, 'windBearing': 271}, {'apparentTemperature': 17.63, 'windSpeed': 1.76, 'icon': 'clear-day', 'temperature': 17.63, 'summary': 'Clear', 'pressure': 1015.51, 'humidity': 0.46, 'dewPoint': 5.91, 'precipType': 'rain', 'time': 1420070400, 'visibility': 16.09, 'windBearing': 263}, {'apparentTemperature': 18.01, 'windSpeed': 0.8, 'icon': 'clear-day', 'temperature': 18.01, 'summary': 'Clear', 'pressure': 1015.5, 'humidity': 0.47, 'dewPoint': 6.51, 'precipType': 'rain', 'time': 1420074000, 'visibility': 16.09, 'windBearing': 236}, {'apparentTemperature': 18.17, 'windSpeed': 1.01, 'icon': 'clear-day', 'temperature': 18.17, 'summary': 'Clear', 'pressure': 1015.42, 'humidity': 0.49, 'dewPoint': 7.28, 'precipType': 'rain', 'time': 1420077600, 'visibility': 16.09, 'windBearing': 134}, {'apparentTemperature': 18.18, 'windSpeed': 1.86, 'icon': 'clear-day', 'temperature': 18.18, 'summary': 'Clear', 'pressure': 1015.32, 'humidity': 0.51, 'dewPoint': 7.79, 'precipType': 'rain', 'time': 1420081200, 'visibility': 16.09, 'windBearing': 118}, {'apparentTemperature': 18.16, 'windSpeed': 1.94, 'icon': 'clear-day', 'temperature': 18.16, 'summary': 'Clear', 'pressure': 1015.08, 'humidity': 0.51, 'dewPoint': 7.83, 'precipType': 'rain', 'time': 1420084800, 'visibility': 16.09, 'windBearing': 118}, {'apparentTemperature': 17.98, 'windSpeed': 1.56, 'icon': 'clear-day', 'temperature': 17.98, 'summary': 'Clear', 'pressure': 1014.82, 'humidity': 0.51, 'dewPoint': 7.63, 'precipType': 'rain', 'time': 1420088400, 'visibility': 16.09, 'windBearing': 123}, {'apparentTemperature': 17.43, 'windSpeed': 1.06, 'icon': 'clear-day', 'temperature': 17.43, 'summary': 'Clear', 'pressure': 1014.82, 'humidity': 0.52, 'dewPoint': 7.46, 'precipType': 'rain', 'time': 1420092000, 'visibility': 16.09, 'windBearing': 136}, {'apparentTemperature': 16.22, 'windSpeed': 0.63, 'icon': 'clear-day', 'temperature': 16.22, 'summary': 'Clear', 'pressure': 1015.3, 'humidity': 0.56, 'dewPoint': 7.32, 'precipType': 'rain', 'time': 1420095600, 'visibility': 16.09, 'windBearing': 191}, {'apparentTemperature': 14.62, 'windSpeed': 1.22, 'icon': 'clear-day', 'temperature': 14.62, 'summary': 'Clear', 'pressure': 1016.05, 'humidity': 0.61, 'dewPoint': 7.07, 'precipType': 'rain', 'time': 1420099200, 'visibility': 16.09, 'windBearing': 249}, {'apparentTemperature': 13.17, 'windSpeed': 1.68, 'icon': 'clear-night', 'temperature': 13.17, 'summary': 'Clear', 'pressure': 1016.68, 'humidity': 0.66, 'dewPoint': 6.86, 'precipType': 'rain', 'time': 1420102800, 'visibility': 16.09, 'windBearing': 262}, {'apparentTemperature': 11.98, 'windSpeed': 1.21, 'icon': 'clear-night', 'temperature': 11.98, 'summary': 'Clear', 'pressure': 1017.11, 'humidity': 0.71, 'dewPoint': 6.86, 'precipType': 'rain', 'time': 1420106400, 'visibility': 16.09, 'windBearing': 271}]}}}
现在,为了最终将其转换为函数,我一直在逐步进行以期获得所有步骤,但是我一直陷于嵌套的JSON / DICT转换中数据。
目标是分解斑点,以便拉出并分开24小时,同时使它们与原始的经度,纬度和日期配对。
从上面提到的SQL查询中,我已经在上面显示的4列数据框中获取了数据。我可以使用以下方法隔离“斑点”:
test_df = temps_df.iloc[:,3]
并获得以下输出:
id blob
0 {'daily': {'apparentTemperatureMaxTime': 14215...
然后我尝试使用以下方法将其标准化:
test_df = pd.DataFrame.from_dict(json_normalize(test_df))
然后将其分为一层,其中包含所有日常情况(垃圾),然后将所有24小时情况放入另一个鲍勃中(没有巧妙的方法将此22列表放在此处。
尝试进一步深入,我尝试:
hourly_df = json_normalize(data=test_df, record_path = 'hourly')
但这给了我
TypeError: string indices must be integers
temps_df = db.get_historical_weather(lats, lngs, start_date, end_date)
temps_df.head()
lat lng date blob
0 -45 170 2015-01-18 {'daily': {'apparentTemperatureMaxTime': 14215...
1 -45 170 2015-01-19 {'daily': {'apparentTemperatureMaxTime': 14216...
2 -45 170 2015-01-20 {'daily': {'apparentTemperatureMaxTime': 14217...
3 -45 170 2015-01-21 {'daily': {'apparentTemperatureMaxTime': 14218...
4 -45 170 2015-01-22 {'daily': {'apparentTemperatureMaxTime': 14219...
test_df = temps_df.iloc[:,3]
test_df.head()
0 {'daily': {'apparentTemperatureMaxTime': 14215...
1 {'daily': {'apparentTemperatureMaxTime': 14216...
2 {'daily': {'apparentTemperatureMaxTime': 14217...
3 {'daily': {'apparentTemperatureMaxTime': 14218...
4 {'daily': {'apparentTemperatureMaxTime': 14219...
Name: blob, dtype: object
test_df = pd.DataFrame.from_dict(json_normalize(test_df))
test_df.head()
daily.apparentTemperatureMax daily.apparentTemperatureMaxTime daily.apparentTemperatureMin daily.apparentTemperatureMinTime daily.dewPoint daily.humidity daily.icon daily.moonPhase daily.precipType daily.pressure ... daily.sunsetTime daily.temperatureMax daily.temperatureMaxTime daily.temperatureMin daily.temperatureMinTime daily.time daily.visibility daily.windBearing daily.windSpeed hourly
0 21.17 1421542800 12.39 1421514000 10.78 0.74 clear-day 0.90 rain 995.62 ... 1421569528 21.17 1421542800 12.39 1421514000 1421492400 14.27 232 1.13 [{'apparentTemperature': 14.21, 'windSpeed': 0...
1 15.69 1421632800 9.66 1421600400 9.34 0.79 clear-day 0.94 rain 1000.24 ... 1421655887 15.69 1421632800 9.66 1421600400 1421578800 13.74 223 0.53 [{'apparentTemperature': 11.41, 'windSpeed': 1...
2 16.73 1421719200 8.53 1421686800 7.86 0.74 clear-day 0.97 rain 1014.10 ... 1421742244 16.73 1421719200 8.53 1421686800 1421665200 15.85 208 1.94 [{'apparentTemperature': 10.08, 'windSpeed': 1...
hourly_df = json_normalize(data=test_df, record_path = 'hourly')
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-15-dd6793be4e4c> in <module>()
----> 1 hourly_df = json_normalize(data=test_df, record_path = 'hourly')
/opt/conda/lib/python3.6/site-packages/pandas/io/json/normalize.py in json_normalize(data, record_path, meta, meta_prefix, record_prefix, errors, sep)
260 records.extend(recs)
261
--> 262 _recursive_extract(data, record_path, {}, level=0)
263
264 result = DataFrame(records)
/opt/conda/lib/python3.6/site-packages/pandas/io/json/normalize.py in _recursive_extract(data, path, seen_meta, level)
236 else:
237 for obj in data:
--> 238 recs = _pull_field(obj, path[0])
239
240 # For repeating the metadata later
/opt/conda/lib/python3.6/site-packages/pandas/io/json/normalize.py in _pull_field(js, spec)
183 result = result[field]
184 else:
--> 185 result = result[spec]
186
187 return result
TypeError: string indices must be integers
再次,我正在逐行尝试此操作,希望得到一个产生我想要的结果的宏函数。最后,我将寻找一个仅具有...的数据框。
lat lng date hour temp
0 -45 170 2015-01-28 0 10
1 -45 170 2015-01-28 1 10
2 -45 170 2015-01-28 2 10
3 -45 170 2015-01-28 3 10
4 -45 170 2015-01-28 4 10
因此,它将显示一个经纬度和经度的一个日期的所有24小时数据,然后移至该经纬度和经度的下一个日期,直到经过框架中的所有日期,然后将递增到一对拉特。
答案 0 :(得分:0)
import json
from pandas.io.json import json_normalize
#from your data
df = pd.DataFrame(data)
#make the entire df a json file
df_json = df.to_json(orient = 'records', date_format='iso')
#use json_normalize to read in your json file, look at the hourly dict, and attach lat, lng and date.
df2 = json_normalize(json.loads(df_json), record_path = ['blob' , 'hourly'] , meta = ['lat', 'lng', 'date'])
#look at only the columns you want
df3 = df2.reindex(['lat', 'lng', 'date','temperature'], axis = 1)
#repeat 0-23 for the lenght of the df (since the time column in hourly isnt quite right, look at your dict that you posted)
df3['hour'] = (np.arange(0, 24).tolist())*(int(len(df3)/24))
df3.head()
lat lng date temperature hour
0 -44 169 2015-09-28T00:00:00.000Z 8.62 0
1 -44 169 2015-09-28T00:00:00.000Z 8.34 1
2 -44 169 2015-09-28T00:00:00.000Z 7.30 2
3 -44 169 2015-09-28T00:00:00.000Z 5.94 3
4 -44 169 2015-09-28T00:00:00.000Z 4.88 4
最简单的方法是将数据导出到json,然后使用json_normalize读取。 dict中的time列不太正确,因此我创建了自己的hour列(但是如果它们都是全天,那么就可以正常工作)。
#Output with two rows:
df3.iloc[np.r_[0:5, -5:0]]
lat lng date temperature hour
0 -45 169 2015-01-01T00:00:00.000Z 9.88 0
1 -45 169 2015-01-01T00:00:00.000Z 9.70 1
2 -45 169 2015-01-01T00:00:00.000Z 9.39 2
3 -45 169 2015-01-01T00:00:00.000Z 9.06 3
4 -45 169 2015-01-01T00:00:00.000Z 8.76 4
43 -45 170 2015-01-01T00:00:00.000Z 17.43 19
44 -45 170 2015-01-01T00:00:00.000Z 16.22 20
45 -45 170 2015-01-01T00:00:00.000Z 14.62 21
46 -45 170 2015-01-01T00:00:00.000Z 13.17 22
47 -45 170 2015-01-01T00:00:00.000Z 11.98 23