我在ipython工作;我有一个Yaml文件和一个与我的Yaml文件对应的[thomas] id列表（托马斯：文件中的第三行）。下面只是该文件的一小部分。完整的文件可以在这里找到（https://github.com/108michael/congress-legislators/blob/master/legislators-historical.yaml）
- id:
bioguide: C000858
thomas: '00246'
lis: S215
govtrack: 300029
opensecrets: N00002091
votesmart: 53288
icpsr: 14809
fec:
- S0ID00057
wikipedia: Larry Craig
house_history: 11530
name:
first: Larry
middle: E.
last: Craig
bio:
birthday: '1945-07-20'
gender: M
religion: Methodist
terms:
- type: rep
start: '1981-01-05'
end: '1983-01-03'
state: ID
district: 1
party: Republican
- type: rep
start: '1983-01-03'
end: '1985-01-03'
state: ID
district: 1
party: Republican
我想解析文件以及我的列表中与[thomas：]中的Id对应的每个id我要检索以下内容：[fec] :(可能有多个这些，我需要全部其中）[名称：] [第一名：] [中：] [最后：]; [生物：] [生日：]; [条款：]（可能有不止一个术语，我需要所有术语）[type：] [start：] [state：] [party：]。最后，可能还存在fec数据不可用的情况。
1）我应该如何存储数据？我仍然相对较新的Python（我的第一个编程语言），我不知道如何存储数据。直觉上，我会说字典;然而，最重要的是易于访问和数据检索。以前，我已经存储了与csv类似的嵌套数据。这种方法看起来有点笨重。如果我可以只列出一个列表（来自我所拥有的thomas id）（我正在检索的数据），这似乎是理想的。
2）我不确定如何设置for / while语句，以便我只检索与我的thomas id列表相对应的数据。
我开始编写我希望将信息写入CSV的代码：
import pandas as pd
import yaml
import glob
import CSV
df = pd.concat((pd.read_csv(f, names=['date','bill_id','sponsor_id']) for f in glob.glob('/home/jayaramdas/anaconda3/df/s11?_s_b')))
outputfile = open('sponsor_details', 'W', newline='')
outputwriter = csv.writer(outputfile)
df = df.drop_duplicates('sponsor_id')
sponsor_list = df['sponsor_id'].tolist()
with open('legislators-historical.yaml', 'r') as f:
data = yaml.load(f)
for sponsor in sponsor_list:
where sponsor == data[0]['thomas']:
x = data[0]['thomas']
a = data[0]['name']['first']
b = data[0]['name']['middle']
c = data[0]['name']['last']
d = data[0]['bio']['gender']
e = data[0]['bio']['religion']
for fec in data[0]['id']:
c = fec.get('fec')
for terms in data[0]['id']:
t = terms.get('type')
s = terms.get('start')
state = terms.get('state')
p = terms.get('party')
outputwriter.writerow([x, a, b, c, d, e, c, t, s, state, p])
outputfile.flush()
我收到以下错误：
---------------------------------------------------------------------------
KeyError Traceback (most recent call last)
<ipython-input-48-057d25de7e11> in <module>()
15
16 for sponsor in sponsor_list:
---> 17 if sponsor == data[0]['thomas']:
18 x = data[0]['thomas']
19 a = data[0]['name']['first']
KeyError: 'thomas'
答案 0 :(得分：8)
我认为您可能会尝试解析YAML并将其加载到数据框normalizing：
import pandas as pd
import yaml
with open('legislators-historical.yaml', 'r') as f:
df = pd.io.json.json_normalize(yaml.load(f))
print(df.head())
输出：
bio.birthday bio.gender bio.religion id.bioguide id.fec id.govtrack \
0 1943-12-02 M Protestant A000109 [S6CO00168] 300003
1 1745-04-02 M NaN B000226 NaN 401222
2 1742-03-21 M NaN B000546 NaN 401521
3 1743-06-16 M NaN B001086 NaN 402032
4 1730-07-22 M NaN C000187 NaN 402334
id.house_history id.icpsr id.lis id.opensecrets id.thomas id.votesmart \
0 8410 29108 S250 N00009082 00011 26783
1 NaN 507 NaN NaN NaN NaN
2 9479 786 NaN NaN NaN NaN
3 10177 1260 NaN NaN NaN NaN
4 10687 1538 NaN NaN NaN NaN
id.wikipedia name.first name.last name.middle \
0 Wayne Allard Wayne Allard A.
1 NaN Richard Bassett NaN
2 NaN Theodorick Bland NaN
3 Aedanus Burke Aedanus Burke NaN
4 Daniel Carroll Daniel Carroll NaN
terms
0 [{'party': 'Republican', 'type': 'rep', 'state...
1 [{'party': 'Anti-Administration', 'type': 'sen...
2 [{'end': '1791-03-03', 'district': 9, 'type': ...
3 [{'end': '1791-03-03', 'district': 2, 'type': ...
4 [{'end': '1791-03-03', 'district': 6, 'type': ...
<强>更新：
以下版本将过滤您的输入数据，因此只有包含＆＃34; thomas＆＃34;和＆＃34; fec＆＃34;将被处理：
#import ujson
#import pprint as pp
import yaml
import pandas as pd
from pandas.io.json import json_normalize
def read_yaml(fn):
with open(fn, 'r') as fi:
return yaml.load(fi)
def filter_data(data):
result_data = []
for x in data:
if 'id' not in x: continue
if 'fec' not in x['id']: continue
if 'thomas' not in x['id']: continue
result_data.append(x)
return result_data
fn = 'aaa.yaml'
df = json_normalize(filter_data(read_yaml(fn)), 'terms', [['id', 'fec'], ['id', 'thomas']])
print(df.head())
df.to_csv('out.csv')
输出：
class district end party start state type \
0 NaN 4 1993-01-03 Republican 1991-01-03 CO rep
1 NaN 4 1995-01-03 Republican 1993-01-05 CO rep
2 NaN 4 1997-01-03 Republican 1995-01-04 CO rep
3 2 NaN 2003-01-03 Republican 1997-01-07 CO sen
4 2 NaN 2009-01-03 Republican 2003-01-07 CO sen
url id.thomas id.fec
0 NaN 00011 S6CO00168
1 NaN 00011 S6CO00168
2 NaN 00011 S6CO00168
3 NaN 00011 S6CO00168
4 http://allard.senate.gov 00011 S6CO00168
PS，因为您看到这将复制您的行（请参阅：id.thomas和id.fec），以便它可以显示为数据框
<强> UPDATE2
您可能还想在＆＃39; id.fec＆＃39;中转换列表。到列，但我会在额外的数据框中执行：
df_fec = df['id.fec'].apply(pd.Series)
print(df_fec.head())
输出：
0 1
0 S8AR00112 H2AR01022
1 S8AR00112 H2AR01022
2 S8AR00112 H2AR01022
3 S8AR00112 H2AR01022
4 S6CO00168 NaN