Question

我正在尝试创建一个简单的脚本，以打印来自网站https://www.bitcoinprice.com/

的当前比特币价格。

#This is a script that returns the price of one Bitcoin in US dollars
import requests
from bs4 import BeautifulSoup

url = "https://www.bitcoinprice.com/"

#Get the url to beautifulSoup format
res = requests.get(url)
res.raise_for_status()
soup = BeautifulSoup(res.text)

#get the price
element = soup.select("#price")

当我尝试打印“ element”变量时，我得到了：

<span id="price" style="font-size: 72px; font-weight: bold;"></span>

我相信这是由页面上的JavaScript引起的。有没有更好的方法可以从网站上获取价格？

Answer 1

是的，您在这里。该值正在由Javascript不断更新，而Scrapper未捕获。
我建议您使用网站用于收集数据的后端API。
有一个开放的API，您可以在一定时间（比如说1分钟）后点击该API，直接获取数据，而不用抓取。

GET https://min-api.cryptocompare.com/data/pricemultifull?fsyms=BTC&tsyms=USD&e=CCCAGG

JSON响应如下，

{
    "RAW": {
        "BTC": {
            "USD": {
                "TYPE": "5",
                "MARKET": "CCCAGG",
                "FROMSYMBOL": "BTC",
                "TOSYMBOL": "USD",
                "FLAGS": "4",
                "PRICE": 6544.82,
                "LASTUPDATE": 1539801288,
                "LASTVOLUME": 0.004,
                "LASTVOLUMETO": 25.74948,
                "LASTTRADEID": "52589962",
                "VOLUMEDAY": 27152.289698759938,
                "VOLUMEDAYTO": 178354574.93607947,
                "VOLUME24HOUR": 38151.0738459291,
                "VOLUME24HOURTO": 250157534.15733647,
                "OPENDAY": 6582.86,
                "HIGHDAY": 6589.2,
                "LOWDAY": 6511.6,
                "OPEN24HOUR": 6541.77,
                "HIGH24HOUR": 6598.58,
                "LOW24HOUR": 6504.37,
                "LASTMARKET": "Coinbase",
                "CHANGE24HOUR": 3.0499999999992724,
                "CHANGEPCT24HOUR": 0.04662346734903967,
                "CHANGEDAY": -38.039999999999964,
                "CHANGEPCTDAY": -0.5778643325241607,
                "SUPPLY": 17327237,
                "MKTCAP": 113403647262.34,
                "TOTALVOLUME24H": 256273.81355038052,
                "TOTALVOLUME24HTO": 1677731603.429824
            }
        }
    },
    "DISPLAY": {
        "BTC": {
            "USD": {
                "FROMSYMBOL": "Ƀ",
                "TOSYMBOL": "$",
                "MARKET": "CryptoCompare Index",
                "PRICE": "$ 6,544.82",
                "LASTUPDATE": "Just now",
                "LASTVOLUME": "Ƀ 0.004000",
                "LASTVOLUMETO": "$ 25.75",
                "LASTTRADEID": "52589962",
                "VOLUMEDAY": "Ƀ 27,152.3",
                "VOLUMEDAYTO": "$ 178,354,574.9",
                "VOLUME24HOUR": "Ƀ 38,151.1",
                "VOLUME24HOURTO": "$ 250,157,534.2",
                "OPENDAY": "$ 6,582.86",
                "HIGHDAY": "$ 6,589.20",
                "LOWDAY": "$ 6,511.60",
                "OPEN24HOUR": "$ 6,541.77",
                "HIGH24HOUR": "$ 6,598.58",
                "LOW24HOUR": "$ 6,504.37",
                "LASTMARKET": "Coinbase",
                "CHANGE24HOUR": "$ 3.05",
                "CHANGEPCT24HOUR": "0.05",
                "CHANGEDAY": "$ -38.04",
                "CHANGEPCTDAY": "-0.58",
                "SUPPLY": "Ƀ 17,327,237.0",
                "MKTCAP": "$ 113.40 B",
                "TOTALVOLUME24H": "Ƀ 256.27 K",
                "TOTALVOLUME24HTO": "$ 1,677.73 M"
            }
        }
    }
}

PRICE键保存所需的核心数据。

尝试使用python3和beautifulSoup抓取网站，但返回一个空列表

1 个答案: