发送具有指定值的下拉表的request.get

时间:2018-12-22 18:48:57

标签: python html beautifulsoup

我需要使用request.get进行网络链接:

https://www.investing.com/crypto/bitcoin/btc-usd-historical-data

默认值为bitfinex,但是我需要Kraken的页面,可以从下拉菜单中进行选择。我在下面包含了html代码。我看不到要选择的选项值,然后将其作为数据参数传递,那么如何使用request.get来设置此下拉列表的值?

<div class="clear"></div><div class="exchangeDropdownContainer             float_lang_base_1 instDropDown" id="DropDownContainer">
<a id="DropdownBtn" class="newBtnDropdown inlineblock ">
                <span class="ceFlags bitcoin">&nbsp;</span>
            <i class="btnTextDropDwn arial_12 bold">Bitfinex</i>
                <i class="bottunImageDoubleArrow buttonWhiteImageDownArrow">&nbsp;</i>
        </a>
        <table class="genTbl closedTbl exchangeDropdownTbl displayNone" id="DropdownSiblingsTable">
        <thead>
        <tr>
            <th class="center">&nbsp;</th>
            <th class="left">Symbol</th>
            <th class="left">Exchange</th>
            <th class="center">Currency</th>
            <th class="left">&nbsp;</th>
        </tr>
        </thead>
        <tbody>
                        <tr data-href="/crypto/bitcoin/btc-usd-historical-data" class="selected">
                <td class="center"><i class="ceFlags bitcoin">&nbsp;</i></td>
                <td class="left bold"><a href="/crypto/bitcoin/btc-usd-historical-data">BTC/USD</a></td>
                <td class="left">Bitfinex</td>
                <td class="center">USD</td>
                <td class="left lightgrayFont bold">Real-time</td>
            </tr>
                        <tr data-href="/indices/investing.com-btc-usd-historical-data" >
                <td class="center"><i class="ceFlags bitcoin">&nbsp;</i></td>
                <td class="left bold"><a href="/indices/investing.com-btc-usd-historical-data">BTC/USD</a></td>
                <td class="left">Investing.com</td>
                <td class="center">USD</td>
                <td class="left lightgrayFont bold">Real-time</td>
            </tr>
                        <tr data-href="/crypto/bitcoin/btc-usd-historical-data?cid=53082" >
                <td class="center"><i class="ceFlags bitcoin">&nbsp;</i></td>
                <td class="left bold"><a href="/crypto/bitcoin/btc-usd-historical-data?cid=53082">BTC/USD</a></td>
                <td class="left">OKCoin</td>
                <td class="center">USD</td>
                <td class="left lightgrayFont bold">Real-time</td>
            </tr>
                        <tr data-href="/crypto/bitcoin/btc-usd-historical-data?cid=1010780" >
                <td class="center"><i class="ceFlags bitcoin">&nbsp;</i></td>
                <td class="left bold"><a href="/crypto/bitcoin/btc-usd-historical-data?cid=1010780">BTC/USD</a></td>
                <td class="left">Poloniex</td>
                <td class="center">USD</td>
                <td class="left lightgrayFont bold">Real-time</td>
            </tr>
                        <tr data-href="/crypto/bitcoin/btc-usd-historical-data?cid=1010796" >
                <td class="center"><i class="ceFlags bitcoin">&nbsp;</i></td>
                <td class="left bold"><a href="/crypto/bitcoin/btc-usd-historical-data?cid=1010796">BTC/USD</a></td>
                <td class="left">GDAX</td>
                <td class="center">USD</td>
                <td class="left lightgrayFont bold">Real-time</td>
            </tr>
                        <tr data-href="/crypto/bitcoin/btc-usd-historical-data?cid=53073" >
                <td class="center"><i class="ceFlags bitcoin">&nbsp;</i></td>
                <td class="left bold"><a href="/crypto/bitcoin/btc-usd-historical-data?cid=53073">BTC/USD</a></td>
                <td class="left">ANX</td>
                <td class="center">USD</td>
                <td class="left lightgrayFont bold">Real-time</td>
            </tr>
                        <tr data-href="/crypto/bitcoin/btc-usd-historical-data?cid=49799" >
                <td class="center"><i class="ceFlags bitcoin">&nbsp;</i></td>
                <td class="left bold"><a href="/crypto/bitcoin/btc-usd-historical-data?cid=49799">BTC/USD</a></td>
                <td class="left">Kraken</td>
                <td class="center">USD</td>
                <td class="left lightgrayFont bold">Real-time</td>
            </tr>

我需要将下拉列表设置为Kraken选项

1 个答案:

答案 0 :(得分:0)

将请求传递到以下网址

https://www.investing.com/crypto/bitcoin/btc-usd-historical-data?cid=49799

只需更改基本URL页面中的下拉列表,您就会看到cid参数已添加到其中。进行网页抓取时,重要的是要与要抓取的网站一起玩耍。

import requests
headers = {'User-Agent' : 'Mozilla/5.0 (Macintosh; Intel Mac OS X x.y; rv:42.0) Gecko/20100101 Firefox/42.' }
url="https://www.investing.com/crypto/bitcoin/btc-usd-historical-data?cid=49799"
response=requests.get(url,headers=headers)
print(response.text)