Question

我正在废弃有关硬币的信息。我需要将所有报废的数据附加到一个Dataframe。

我有以下代码：

import bs4 as bs
import urllib.request
import pandas as pd

def scraping_func(coin_id):
    """  
    This function recives "coins_id" from "coins_to_scrape list".
    It scrapes all data that I need and puts it to "df" Dataframe
    Then it saves this Dataframe to excel file (%coin_id%.xlsx)
    The last lines of the function are:  (I cut my code because it's big) 
    """   
    df = df[['Country', 'Value', 'Year', 'Metal', 'Marks', 'Mintage', 'Krause', 'Price', 'Quality', 'Details', 'Avers', 'Revers', 'Gcoins_link']]
    excel_name = '{}.xlsx'.format(coin_id)                               
    df.to_excel(excel_name)      
    for i in dfs:
        dfs = df.append(dfs, ignore_index=True)
        dfs.to_excel('adfasdf.xlsx')

coins_to_scrape = [514, 515, 179080, 45518, 521, 111429]        # The list of ID that I need to scrape
for i in coins_to_scrape:                                       # For each coin in the list
    scraping_func(i)                                            # call the "scraping_func" function

效果很好。但是这段代码为我传递给函数的每个硬币创建了单独的excel文档。但是有数千个硬币要刮，所以我想将每个Dataframes附加到一个，然后将其保存到excel文件中。

我试图在互联网上找到解决方案，但我不能。

Answer 1

您可以创建数据框列表，然后将它们连接到一个数据框中：

def scraping_func(coin_id):
    # do stuff to create dataframe
    return df[['Country', 'Value', ..., 'Revers', 'Gcoins_link']]

coins_to_scrape = [514, 515, 179080, 45518, 521, 111429]

# construct list of dataframes via list comprehension
df_list = [scraping_func(i) for i in coins_to_scrape]

# combine dataframes in list
df = pd.concat(df_list, ignore_index=True)

Python将几个数据框附加到一个内部函数中

1 个答案: