Question

我要删除此页面以自动下载其中的所有文件： https://spreadstreet.io/database/。

诀窍在于，下拉按钮（）的起始值为'10'。我希望将其设置为“全部”。

但是，尽管在浏览器中看到了结果，但在控制台中打印结果时，却找不到与此下拉列表相关的任何“ div”！

我遵循了一个很好的答案：scraping a response from a selected option in dropdown list

但是我不知道在哪里可以找到这些信息并对其进行更改。

Answer 1

看起来您的表是动态加载其数据的。 BeautifulSoup本身还不够。因此，您将需要使用硒之类的工具。

您还可以通过向源发送http请求来达到目的。

您可以检查浏览器并检查它向何处发送数据并从中创建请求。

这是您问题的解决方案。

from bs4 import BeautifulSoup
import requests
import json

url = "https://spreadstreet.io/wp-admin/admin-ajax.php"

querystring = {"action":"get_wdtable","table_id":"102"}

payload = "draw=4&columns%5B0%5D%5Bdata%5D=0&columns%5B0%5D%5Bname%5D=Cryptocurrency&columns%5B0%5D%5Bsearchable%5D=true&columns%5B0%5D%5Borderable%5D=true&columns%5B0%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B0%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B1%5D%5Bdata%5D=1&columns%5B1%5D%5Bname%5D=frequency&columns%5B1%5D%5Bsearchable%5D=true&columns%5B1%5D%5Borderable%5D=true&columns%5B1%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B1%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B2%5D%5Bdata%5D=2&columns%5B2%5D%5Bname%5D=Exchange&columns%5B2%5D%5Bsearchable%5D=true&columns%5B2%5D%5Borderable%5D=true&columns%5B2%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B2%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B3%5D%5Bdata%5D=3&columns%5B3%5D%5Bname%5D=database_code&columns%5B3%5D%5Bsearchable%5D=true&columns%5B3%5D%5Borderable%5D=true&columns%5B3%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B3%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B4%5D%5Bdata%5D=4&columns%5B4%5D%5Bname%5D=Description&columns%5B4%5D%5Bsearchable%5D=true&columns%5B4%5D%5Borderable%5D=true&columns%5B4%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B4%5D%5Bsearch%5D%5Bregex%5D=false&columns%5B5%5D%5Bdata%5D=5&columns%5B5%5D%5Bname%5D=Download&columns%5B5%5D%5Bsearchable%5D=true&columns%5B5%5D%5Borderable%5D=true&columns%5B5%5D%5Bsearch%5D%5Bvalue%5D=&columns%5B5%5D%5Bsearch%5D%5Bregex%5D=false&order%5B0%5D%5Bcolumn%5D=0&order%5B0%5D%5Bdir%5D=asc&start=0&length=-1"
headers = {
    'Accept': "text/html; charset=UTF-8",
    'Content-Type': "application/x-www-form-urlencoded"
    }

response = requests.request("POST", url, data=payload, headers=headers, params=querystring)

crpytos = json.loads(response.text)['data']

for crpyto in crpytos:
    name = crpyto[0]
    duration = crpyto[1]
    exchange = crpyto[2]
    download_link = BeautifulSoup(crpyto[5],'lxml').find('a')['href']
    print(name,duration,exchange,download_link)

这将为您提供输出

.
.
.
.
Yacoin daily BTER https://spreadstreet.io?nc_download=1&path=BTER/YAC-BTC Exchange Rate.csv
Zcash daily Bitfinex https://spreadstreet.io?nc_download=1&path=Bitfinex/ZEC-BTC Exchange Rate.csv
Zcash daily Bitfinex https://spreadstreet.io?nc_download=1&path=Bitfinex/ZEC-USD Exchange Rate.csv
Zcash daily BTER https://spreadstreet.io?nc_download=1&path=BTER/ZEC-CNY Exchange Rate.csv
Zcash daily BTER https://spreadstreet.io?nc_download=1&path=BTER/ZEC-BTC Exchange Rate.csv
Zcash 5 min Poloniex https://spreadstreet.io?nc_download=1&path=Poloniex/5-Minute/USDT_ZEC 5-Minute.csv
Zcash 15 Min Poloniex https://spreadstreet.io?nc_download=1&path=Poloniex/15-Minute/USDT_ZEC 15-Minute.csv
Zcash 30 Min Poloniex https://spreadstreet.io?nc_download=1&path=Poloniex/30-Minute/USDT_ZEC 30-Minute.csv
Zcash 2 Hour Poloniex https://spreadstreet.io?nc_download=1&path=Poloniex/2-Hour/USDT_ZEC 2-Hour.csv
Zcash 4 Hour Poloniex https://spreadstreet.io?nc_download=1&path=Poloniex/4-Hour/USDT_ZEC 4-Hour.csv
Zcash daily Poloniex https://spreadstreet.io?nc_download=1&path=Poloniex/Daily/USDT_ZEC Daily.csv
Zcash Tick Bitfinex https://spreadstreet.io?nc_download=1&path=Bitfinex/Tick/ByCurrency/ZECBTC.csv
Zcash Tick Bitfinex https://spreadstreet.io?nc_download=1&path=Bitfinex/Tick/ByCurrency/ZECUSD.csv
ZcCoin daily BTER https://spreadstreet.io?nc_download=1&path=BTER/ZCC-CNY Exchange Rate.csv
ZcCoin daily BTER https://spreadstreet.io?nc_download=1&path=BTER/ZCC-BTC Exchange Rate.csv
Zetacoin daily BTER https://spreadstreet.io?nc_download=1&path=BTER/ZET-CNY Exchange Rate.csv
Zetacoin daily BTER https://spreadstreet.io?nc_download=1&path=BTER/ZET-BTC Exchange Rate.csv

找不到html元素以使用python3和Beautifulsoup更改其值

1 个答案: