Question

我目前有这个代码。它运作得很好。

它遍历文件夹中的excel文件，删除前两行，然后将它们保存为单独的excel文件，它还将文件保存在循环中作为附加文件。

目前，每次运行代码时，附加文件都会覆盖现有文件。

我需要将新数据附加到现有Excel工作表（＆＃39; master_data.xlsx）的底部

dfList = []
path = 'C:\\Test\\TestRawFile' 
newpath = 'C:\\Path\\To\\New\\Folder'

for fn in os.listdir(path): 
  # Absolute file path
  file = os.path.join(path, fn)
  if os.path.isfile(file): 
    # Import the excel file and call it xlsx_file 
    xlsx_file = pd.ExcelFile(file) 
    # View the excel files sheet names 
    xlsx_file.sheet_names 
    # Load the xlsx files Data sheet as a dataframe 
    df = xlsx_file.parse('Sheet1',header= None) 
    df_NoHeader = df[2:] 
    data = df_NoHeader 
    # Save individual dataframe
    data.to_excel(os.path.join(newpath, fn))

    dfList.append(data) 

appended_data = pd.concat(dfList)
appended_data.to_excel(os.path.join(newpath, 'master_data.xlsx'))

我认为这将是一项简单的任务，但我猜不是。我想我需要将master_data.xlsx文件作为数据帧引入，然后将索引与新的附加数据匹配，并将其保存回来。或者也许有一种更简单的方法。任何帮助表示赞赏。

Answer 1

用于将DataFrame附加到现有 Excel文件的辅助函数：

def append_df_to_excel(filename, df, sheet_name='Sheet1', startrow=None,
                       truncate_sheet=False, 
                       **to_excel_kwargs):
    """
    Append a DataFrame [df] to existing Excel file [filename]
    into [sheet_name] Sheet.
    If [filename] doesn't exist, then this function will create it.

    Parameters:
      filename : File path or existing ExcelWriter
                 (Example: '/path/to/file.xlsx')
      df : dataframe to save to workbook
      sheet_name : Name of sheet which will contain DataFrame.
                   (default: 'Sheet1')
      startrow : upper left cell row to dump data frame.
                 Per default (startrow=None) calculate the last row
                 in the existing DF and write to the next row...
      truncate_sheet : truncate (remove and recreate) [sheet_name]
                       before writing DataFrame to Excel file
      to_excel_kwargs : arguments which will be passed to `DataFrame.to_excel()`
                        [can be dictionary]

    Returns: None
    """
    from openpyxl import load_workbook

    import pandas as pd

    # ignore [engine] parameter if it was passed
    if 'engine' in to_excel_kwargs:
        to_excel_kwargs.pop('engine')

    writer = pd.ExcelWriter(filename, engine='openpyxl')

    # Python 2.x: define [FileNotFoundError] exception if it doesn't exist 
    try:
        FileNotFoundError
    except NameError:
        FileNotFoundError = IOError


    try:
        # try to open an existing workbook
        writer.book = load_workbook(filename)

        # get the last row in the existing Excel sheet
        # if it was not specified explicitly
        if startrow is None and sheet_name in writer.book.sheetnames:
            startrow = writer.book[sheet_name].max_row

        # truncate sheet
        if truncate_sheet and sheet_name in writer.book.sheetnames:
            # index of [sheet_name] sheet
            idx = writer.book.sheetnames.index(sheet_name)
            # remove [sheet_name]
            writer.book.remove(writer.book.worksheets[idx])
            # create an empty sheet [sheet_name] using old index
            writer.book.create_sheet(sheet_name, idx)

        # copy existing sheets
        writer.sheets = {ws.title:ws for ws in writer.book.worksheets}
    except FileNotFoundError:
        # file does not exist yet, we will create it
        pass

    if startrow is None:
        startrow = 0

    # write out the new sheet
    df.to_excel(writer, sheet_name, startrow=startrow, **to_excel_kwargs)

    # save the workbook
    writer.save()

Usage examples...

旧回答：它允许您将几个 DataFrames写入新的Excel文件。

您可以将openpyxl引擎与startrow参数结合使用：

In [48]: writer = pd.ExcelWriter('c:/temp/test.xlsx', engine='openpyxl')

In [49]: df.to_excel(writer, index=False)

In [50]: df.to_excel(writer, startrow=len(df)+2, index=False)

In [51]: writer.save()

C：/temp/test.xlsx：

PS如果您不想复制列名，您可能还想指定header=None ...

更新：您可能还想查看this solution

Answer 2

如果您每次都在工作表上使用 ExcelWriter，它将覆盖上一个工作表，并且所有可见的是您附加到工作簿的最后一个数据表。相反，您可以维护一个最初为 1 的计数器，您需要为其初始化 Excel 工作表并使用

的现有方法添加初始数据

writer = pd.ExcelWriter(output_file, engine='openpyxl')

df = pd.read_excel(output_file, sheet_name='TestSheet1')

或者您可以使用我使用的以下方法。下次要使用它时加载工作簿，否则如果您在第一种情况下尝试加载它，则文件未找到异常。

用法：

from bs4 import BeautifulSoup
import requests
import pandas as pd
from openpyxl import load_workbook

urls = ["http://millenniumcricketleague.com/Home/ShowTeam.aspx?tid=22",
        "http://millenniumcricketleague.com/Home/ShowTeam.aspx?tid=40"]
path = "F:\meta_1.xlsx"
writer = pd.ExcelWriter(path,engine='openpyxl')
counter = 1
for url in urls:
    table_data = []
    final = []
    html_content = requests.get(url).text
    soup = BeautifulSoup(html_content, "lxml")
    x = soup.find_all('table')
    for table in x[1:]:
        for tr in table.find_all("tr"):
            newrow = []
            for td in tr.find_all("td"):
                newrow.append(td.text.replace('\n', ' ').strip())
            table_data.append(newrow)
    df = pd.DataFrame(table_data)
    sheetname = 'Sheet%s' % counter
    if(counter!=1):
        writer.book = load_workbook(path)
    df.to_excel(writer, sheet_name=sheetname)
    counter = counter + 1
    writer.save()

无需关闭 excelwriter。它是一个自动功能。如果您明确定义，则会向您显示警告

Answer 3

如果您不是严格寻找excel文件，则将输出作为csv文件并将其复制到新的Excel文件中

df.to_csv('filepath', mode='a', index = False, header=None)

mode ='a'

a表示追加

这是一种环形交叉路口，但效果很好！

Answer 4

这个问题已经存在了一段时间。答案还可以，但是我相信这可以解决大多数人的问题。

只需使用glob访问特定目录中的文件，循环访问它们，创建每个文件的数据框，将其附加到最后一个文件，然后导出到文件夹。我还添加了注释掉的代码，以便通过csvs运行此代码。

import os
import pandas as pd
import glob

# put in path to folder with files you want to append
# *.xlsx or *.csv will get all files of that type
path = "C:/Users/Name/Folder/*.xlsx"
#path = "C:/Users/Name/Folder/*.csv"

# initialize a empty df
appended_data = pd.DataFrame()

#loop through each file in the path
for file in glob.glob(path):
    print(file)

    # create a df of that file path
    df = pd.read_excel(file, sheet_name = 0)
    #df = pd.read_csv(file, sep=',')

    # appened it
    appended_data = appended_data.append(df)

appended_data

# export the appeneded data to a folder of your choice
exportPath = 'C:/My/EXPORT/PATH/appended_dataExport.csv'
appended_data.to_csv(os.path.join(exportPath),index=False)

Answer 5

作为@david的补充，如果您不在乎索引，并且可以使用 .csv ，则此功能有助于将任何df附加到现有的csv

int generateHash(String s1, String s2) =>
    (<String>[s1, s2]..sort()).join().hashCode;

注意：

def append_df(self, path_file, df): with open(path_file, 'a+') as f: df.to_csv(f, header=f.tell() == 0, encoding='utf-8', index=False)创建文件（如果不存在）

a+如果第一行添加标题

Answer 6

import pandas as pd
import openpyxl

workbook = openpyxl.load_workbook("test.xlsx")
writer = pd.ExcelWriter('test.xlsx', engine='openpyxl')
writer.book = workbook
writer.sheets = dict((ws.title, ws) for ws in workbook.worksheets)
data_df.to_excel(writer, 'Existing_sheetname')
writer.save()
writer.close()

Answer 7

建立在 MaxU 和其他人的代码和注释的基础上，但简化为仅使用 pandas ExcelWriter 修复导致 to_excel 创建新工作表而不是在追加模式下追加到现有工作表的错误。

正如其他人所指出的，to_excel 使用 ExcelWriter.sheets 属性，而 ExcelWriter 不会填充该属性。

Fix 是单行代码，否则代码是 to_excel 中记录的标准 Pandas 方法。


    # xl_path is destination xlsx spreadsheet
    with pd.ExcelWriter(xl_path, 'openpyxl', mode='a') as writer:
        # fix line
        writer.sheets = dict((ws.title, ws) for ws in writer.book.worksheets)
        df.to_excel(writer, sheet_name)

使用python pandas使用新数据框附加现有excel表

7 个答案: