Question

我正在尝试从下面提到的字典中创建以下数据框。有没有有效的解决方案？

data_dict = {
    'Total_Amount' : '150.00',
    'LinkAPI' : [{"ConfidenceScore":4},{"ConfidenceScore":9}],
    'RecordID' : 5687,
    'ClientId' : 45,
    'Customer_Number' : ["HDMO70232"],
    'RowNumber' : 0,
    'Invoice_Number' : '',
    'Customer_Name' : 'HD MOTORCYCLES SIS/SVC'
}

数据框中的行数应该等于＆＃39; LinkAPI＆＃39;列表中的项目数。上述数据的数据框应如下所示。

ClientId    Customer_Name   Customer_Number Invoice_Number  LinkAPI RecordID    RowNumber   Total_Amount
0   45  HD MOTORCYCLES SIS/SVC  [HDMO70232]     {'ConfidenceScore': 4}  5687    0   150.00
1   45  HD MOTORCYCLES SIS/SVC  [HDMO70232]     {'ConfidenceScore': 9}  5687    0   150.00

我尝试了两种解决方案来实现这一点。我希望有一种更好的方法来创建数据帧。溶液1：

items_number = len(data_dict['LinkAPI'])
df_dict = {k : [data_dict[k] for _ in range(items_number)] if k != 'LinkAPI' else data_dict[k]
           for k in data_dict.keys()}
df = pd.DataFrame(df_dict)

溶液-2：

LinkAPI = data_dict["LinkAPI"]

df_new = pd.DataFrame(columns=list(df))  # list(df) is ['ClientId','Customer_Name', 'Customer_Number', 
                                            # 'Invoice_Number', 'LinkAPI','RecordID', 'RowNumber', 'Total_Amount']
i=0
for conf in LinkAPI:
    df_new.loc[i] = [data_dict["Total_Amount"], conf, data_dict["RecordID"], data_dict["ClientId"], data_dict["Customer_Number"],
                    data_dict["RowNumber"], data_dict["Invoice_Number"], data_dict["Customer_Name"]]
    i+=1

Answer 1

使用json_normalize：

from pandas.io.json import json_normalize

cols = ['Total_Amount','RecordID','ClientId','Customer_Number',
        'RowNumber','Invoice_Number','Customer_Name']
df = json_normalize(data, 'LinkAPI', cols)
#data borrowed from HYRY
print (df)
   ConfidenceScore  test Total_Amount Invoice_Number  RowNumber  \
0              4.0   NaN       150.00                         0   
1              9.0   NaN       150.00                         0   
2              8.0   NaN      1500.00                         1   
3             10.0   NaN      1500.00                         1   
4             20.0   NaN      1500.00                         1   
5              NaN   2.0      1500.00                         1   

  Customer_Number  ClientId           Customer_Name  RecordID  
0       HDMO70232        45  HD MOTORCYCLES SIS/SVC      5687  
1       HDMO70232        45  HD MOTORCYCLES SIS/SVC      5687  
2       HDMO70232       415  HD MOTORCYCLES SIS/SVC     56287  
3       HDMO70232       415  HD MOTORCYCLES SIS/SVC     56287  
4       HDMO70232       415  HD MOTORCYCLES SIS/SVC     56287  
5       HDMO70232       415  HD MOTORCYCLES SIS/SVC     56287

Answer 2

我将您的数据更改为dict列表：

data = [
{
    'Total_Amount' : '150.00',
    'LinkAPI' : [{"ConfidenceScore":4},{"ConfidenceScore":9}],
    'RecordID' : 5687,
    'ClientId' : 45,
    'Customer_Number' : ["HDMO70232"],
    'RowNumber' : 0,
    'Invoice_Number' : '',
    'Customer_Name' : 'HD MOTORCYCLES SIS/SVC'
},
{
    'Total_Amount' : '1500.00',
    'LinkAPI' : [{"ConfidenceScore":8},{"ConfidenceScore":10}, {"ConfidenceScore":20}, {"test":2}],
    'RecordID' : 56287,
    'ClientId' : 415,
    'Customer_Number' : ["HDMO70232"],
    'RowNumber' : 1,
    'Invoice_Number' : '',
    'Customer_Name' : 'HD MOTORCYCLES SIS/SVC'
},
]

df = pd.DataFrame(data)

df2 = pd.DataFrame(np.concatenate(df.LinkAPI).tolist(), 
                   index=np.repeat(df.index, df.LinkAPI.str.len().astype(int)))

df.drop("LinkAPI", axis=1).join(df2)

输出：

   ClientId           Customer_Name Customer_Number Invoice_Number  RecordID  RowNumber Total_Amount  ConfidenceScore  test
0        45  HD MOTORCYCLES SIS/SVC     [HDMO70232]                     5687          0       150.00              4.0   NaN
0        45  HD MOTORCYCLES SIS/SVC     [HDMO70232]                     5687          0       150.00              9.0   NaN
1       415  HD MOTORCYCLES SIS/SVC     [HDMO70232]                    56287          1      1500.00              8.0   NaN
1       415  HD MOTORCYCLES SIS/SVC     [HDMO70232]                    56287          1      1500.00             10.0   NaN
1       415  HD MOTORCYCLES SIS/SVC     [HDMO70232]                    56287          1      1500.00             20.0   NaN
1       415  HD MOTORCYCLES SIS/SVC     [HDMO70232]                    56287          1      1500.00              NaN   2.0

Answer 3

我不知道它是否是一个选项，但是如果你可以改变你的词典以获得所有条目的等长列表（例如只重复data_dict中当前的值，你可以使用{ {1}}。在您的情况下，字典的每个条目的长度必须等于2，因为这是字典中最长的条目（pd.DataFrame(data_dict)：

LinkAPI)

它为您提供以下数据框：

import pandas as pd
pd.set_option("display.width", 300)  # You can ignore this

data_dict = {
    'Total_Amount' : '150.00',
    'LinkAPI' : [{"ConfidenceScore":4},{"ConfidenceScore":9}],
    'RecordID' : [5687] * 2,
    'ClientId' : [45] * 2,
    'Customer_Number' : ["HDMO70232"] * 2,
    'RowNumber' : [0] * 2,
    'Invoice_Number' : [''] * 2,
    'Customer_Name' : ['HD MOTORCYCLES SIS/SVC'] * 2
}

df = pd.DataFrame(data_dict)

print df

修改

为了澄清，要将字典读取到数据帧，pandas要求每个条目（字典中的键将是数据帧中的列）具有相同的长度。否则，它将抛出ClientId Customer_Name Customer_Number Invoice_Number LinkAPI RecordID RowNumber Total_Amount 0 45 HD MOTORCYCLES SIS/SVC HDMO70232 {u'ConfidenceScore': 4} 5687 0 150.00 1 45 HD MOTORCYCLES SIS/SVC HDMO70232 {u'ConfidenceScore': 9} 5687 0 150.00：

ValueError

使用pandas创建数据框架的有效方法，而不使用for循环

3 个答案: