Question

如何在Redshift中将JSON文件转换为表结构？我尝试了以下python代码。

import boto3
import json
import os
import sys
import psycopg2
import csv
from collections import defaultdict

def jsonfile(path):
    session = boto3.Session(
        aws_access_key_id='dfjfkgj',
        aws_secret_access_key='sdfg',
        region_name='us-west-2')
    s3 = session.resource('s3')
    bucket= s3.Bucket('ag-redshift-poc')
    with open(path, 'rb') as data:
        res=json.load(data)
        f = open('data.csv','wb')
        output = csv.writer(f) 
        output.writerow(res[0].keys())
        for row in res:
           output.writerow(row.values()) 


    bucket.put_object(Key=('C:\Python27\data.csv'),Body=res)
    print 'success'

def redshift():
    co=psycopg2.connect(dbname= 'redshiftpoc', host='shdjf', 
    port= '5439', user= 'admin', password= 'snd')
    curr = co.cursor()
    curr.execute("""copy sample from 's3://ag-redshift-poc/testfile/json.txt'
                CREDENTIALS 'aws_access_key_id=fdfd;aws_secret_access_key=sxhd'
                """)
    co.commit()
    print 'success'
    curr.close()
    co.close()

jsonfile('C:\Python27\json.txt')
redshift()

Answer 1

Redshift可以直接将JSON吸收到表中。（尽管不是很有效）。

根据您的情况，将COPY查询修改为

COPY sample FROM 's3://<bucket_name>/<path_to_json>'
CREDENTIALS 'aws_access_key_id=xxxx;aws_secret_access_key=xxxx'
JSON 'auto' ACCEPTINVCHARS;

请在查询中注意JSON 'auto'。这样会用JSON文件中的键映射表中的每一列。

更多详细信息，请参见COPY examples

如何使用Python在Redshift中将json文件转换为表结构

1 个答案: