我想从 2 个链接中抓取两个表。我的代码是:
import pandas as pd
import xlwings as xw
from datetime import datetime
def last_row(symbol, name):
# Function that outputs if the last row of the df should be deleted or not,
# based on the 2 requirements below.
requirements = [symbol.lower()=="total", name.isdigit()]
return all(requirements)
# return True, if the last row should be deleted.
# The deletion will be performed in the next function.
def get_foreigncompanies_info():
df_list = []
links = ["https://stockmarketmba.com/nonuscompaniesonusexchanges.php",
"https://stockmarketmba.com/listofadrs.php"]
for i in links:
#Reads table with pandas read_html and only save the necessary columns.
df = pd.read_html(i)[0][['Symbol', 'Name', 'GICS Sector']]
if last_row(df.iloc[-1]['Symbol'], df.iloc[-1]['Name']):
# Delete the last row
df_list.append(df.iloc[:-1])
else:
# Keep last row
df_list.append(df)
return pd.concat(df_list).reset_index(drop=True).rename(columns={'Name': 'Security'})
def open_in_excel(dataframe): # Code to view my df in excel.
xw.view(dataframe)
if __name__ == "__main__":
start = datetime.now()
df = get_foreigncompanies_info()
print(datetime.now() - start)
open_in_excel(get_foreigncompanies_info())
我想让代码运行得更快(在某种程度上,这不会产生太多不必要的请求)。 我的想法是将表格下载为 csv,因为在网站上,有一个“下载 csv”按钮。
我如何用 python 下载 csv?
我检查了按钮,但找不到它的网址。 (如果你能找到它,还请描述你是如何找到它的,也许是“检查”屏幕截图。)
或者有其他更快的方法来下载表格吗?
谢谢你的指点:-)