如何在pandas / python

时间:2019-09-13 12:17:56

标签: python json python-3.x pandas dataframe

我要发送到python后端服务的POST请求如下,

{
    "updated_by": "969823826",
    "relation_on": "ID",
    "join_type": "inner",
    "sources": [
    {
        "json_obj": "path/demo8.json",
        "columns": [
            "ID",
            "FIRST_NAME",
            "LAST_NAME"
        ]
    },
    {
        "json_obj": "path/demo1.json",
        "columns": [
            "ID",
            "CITY",
            "SSN"
        ]
    }
  ]
}

因此,我正在尝试基于ID列将两个源对象合并为INNER JOIN。

我正在合并 FILE1 中的 ID,FIRST_NAME,LAST_NAME FILE2 中的 ID,CITY,SSN

通过使用静态方法,我可以做到这一点。

这是我的静态方法代码示例,

import json
import pandas as pd

file1 = "path\\demo1.json"
file2 = "path\\demo3.json"

df1 = pd.read_json(file1)
df2 = pd.read_json(file2)

#merge with specific columns and conditions
new_df = pd.merge(df1[['ID', 'FIRST_NAME', 'LAST_NAME']], df2[['ID', 'CITY', 'SSN']], on='ID', how="inner")   

#merging without any common column
df1['tmp'] = 1
df2['tmp'] = 1     

new_df = pd.merge(df1, df2, on=['tmp'])
new_df = new_df.drop('tmp', axis=1)

new_df.to_json("path\\merge-json.json", orient='records')

现在,如果我想使用for循环以动态方式合并数据帧,则会遇到麻烦。

尝试了几种选择,但是,我认为我的方向不对。

这是动态方法的代码

updated_by = request.get_json()['updated_by']
relation_on = request.get_json()['relation_on']
join_type = request.get_json()['join_type']

sources = request.get_json()['sources']
sources = str(sources).replace("'", '"')
sources = json.loads(sources)

for sources_key, sources_value in enumerate(sources):
    print(sources_key, sources_value)

到此为止,上面的代码已经执行了,我可以如下查看对象,

0 {'ctl_key': '969823826demo8txt', 'json_obj': 'path/demo8.json', 'columns': ['ID', 'FIRST_NAME', 'LAST_NAME']}
1 {'ctl_key': '969823826demo1csv', 'json_obj': 'path/demo1.json', 'columns': ['ID', 'CITY', 'SSN']}

现在,我最初的方法是根据文件输入创建新的数据框,然后合并这两个数据框并创建最后一个。

需要一个JSON obj作为下面的输出,

[
  {
    "ID": 1,
    "FIRST_NAME": "Albertine",
    "LAST_NAME": "Jan",
    "CITY": "Waymill",
    "SSN": "515-72-7353"
  },
  {
    "ID": 2,
    "FIRST_NAME": "Maryetta",
    "LAST_NAME": "Hoyt",
    "CITY": "Spellbridge",
    "SSN": "515-72-7354"
  },
  {
    "ID": 3,
    "FIRST_NAME": "Dustin",
    "LAST_NAME": "Divina",
    "CITY": "Stoneland",
    "SSN": "515-72-7355"
  },
  {
    "ID": 4,
    "FIRST_NAME": "Jenna",
    "LAST_NAME": "Sofia",
    "CITY": "Fayview",
    "SSN": "515-72-7356"
  }
]

任何准则,请...

1 个答案:

答案 0 :(得分:0)

当我外部联接数据框时,我想使用pd.set_index到要联接的列,然后使用pd.concat([df1, df2], axis=1)。 我认为这应该适用于这种情况。