Question

我的代码适用于循环

一次处理后将生成100行秒之后，它将生成160行

它可以持续进行10000次并有近3亿行。

import pandas
from openpyxl import load_workbook

book = load_workbook('Masterfile.xlsx')
writer = pandas.ExcelWriter('Masterfile.xlsx', engine='openpyxl') 
writer.book = book
writer.sheets = dict((ws.title, ws) for ws in book.worksheets)

df.to_excel(writer, "Main", cols=['Diff1', 'Diff2'])

writer.save()

使用了此功能，但数据未保存，仍被覆盖

我的表只有3列，并且所有输出列都相同。

代码：

a = "SELECT id FROM USER WHERE time >'2018-03-01'"
dataa = pd.read_sql_query(a, con=engine)
print(dataa)

for userid in dataa:
   x=f"SELECT idbody FROM col1 WHERE user_id='{userid}'"
   data = pd.read_sql_query(x,con = engine)

在这里，我需要将这1000个userid行附加到单个excel文件中

Answer 1

您应该使用“追加”模式，该模式将在不覆盖的情况下追加数据。

import pandas
from openpyxl import load_workbook

book = load_workbook('Masterfile.xlsx')

with pandas.ExcelWriter('Masterfile.xlsx', engine='openpyxl', mode='a') as writer: 
    writer.book = book
    writer.sheets = dict((ws.title, ws) for ws in book.worksheets)

    df.to_excel(writer, "Main", cols=['Diff1', 'Diff2'])

    writer.save()

如何在不删除以前的数据的情况下写入现有的excel 10000次文件？

1 个答案: