Question

我需要使用python将三个表从magento加载到我的数据仓库中，但是在ETL操作仅需要将选定的列移至DWH之后，不需要加载整个表结构。

下面是需要在DWH中创建的带有列的表。

dl_magento.sales_flat_quote -
             customer_id,
             entity_id,
             store_id,
             created_at,
             updated_at,
             items_count,
             base_row_total,
             row_total
             base_discount_amount,
             base_subtotal_with_discount,
             base_to_global_rate
             is_active

 dl_magento.sales_flat_quote_item  
            product_id,
            name,
            item_id,
            quote_id

您能建议如何做到吗？下面是我现在正在使用的代码，它不是通用的代码，您能帮助我改善这一点吗？

import psycopg2

Conn_DWH = psycopg2.connect("host=postgres dbname=Postgres user=postgres password=*** ")
Conn_Magento = psycopg2.connect("host=Magento dbname=Magento user=Magento password=*** ")
conndwh = Conn_DWH.cursor()
curmag= Conn_Magento.cursor()

cur.execute("CREATE TABLE sales_flat_quote (customer_id,entity_id,store_id,created_at,updated_at,items_count,base_row_total,row_total,base_discount_amount,base_subtotal_with_discount,base_to_global_rate,is_active);")
sql = ('INSERT INTO "sales_flat_quote" ( customer_id,entity_id,store_id,created_at,updated_at,items_count,base_row_total,row_total,base_discount_amount,base_subtotal_with_discount,base_to_global_rate,is_active) values (%s, %s,%s, %s,%s,%s, %s,%s,%s, %s,%s,%s);')
conndwh.execute('select customer_id,entity_id,store_id,created_at,updated_at,items_count,base_row_total,row_total,base_discount_amount,base_subtotal_with_discount,base_to_global_rate,is_active from "sales_flat_quote" where is_active=1;')
tempG = conndwh.fetchall()
data = (tempG)

 if data:
    curmag.execute (sql, data)
    print('data loaded to warehouse db')
  else:
    print('data is empty')

#commit transactions
Conn_DWH.commit()
Conn_Magento.commit()

#close connections
conndwh.close()
curmag.close()

Conn_DWH.close()
Conn_Magento.close()

DWH服务器：PostgreSQL Python作为数据加载器

谢谢！

将数据从magento加载到数据仓库

0 个答案: