通过浏览网页中的不同选项卡并将数据提取到Dataframe进行Web抓取
我的链接为：
url = "https://nseindia.com/live_market/dynaContent/live_analysis/top_gainers_losers.htm"
我想像这样导航：
1. Click on Losers Tab
2. Click on F&O Securities Tab
3. Fetch the result table data to Dataframe.
1 个答案:
答案 0 :(得分：1)
查看“浏览器网络”选项卡，清除所有内容并单击1.单击“失败者”选项卡->单击“ F＆O证券”选项卡，您将看到https://nseindia.com/live_market/dynaContent/live_analysis/losers/fnoLosers1.json url请求json响应，这是一个表数据。
import requests
import json
headers = {"Accept-Language": "en-US, en;q=0.5"}
url = 'https://nseindia.com/live_market/dynaContent/live_analysis/losers/fnoLosers1.json'
response = requests.get(url)
print(response.json())
O / P：
{
"data": [
{
"symbol": "RELINFRA",
"series": "EQ",
"openPrice": "83.45",
"highPrice": "86.80",
"lowPrice": "72.70",
"ltp": "73.45",
"previousPrice": "84.90",
"netPrice": "-13.49",
"tradedQuantity": "4,28,81,500",
"turnoverInLakhs": "33,782.05",
"lastCorpAnnouncementDate": "12-Sep-2018",
"lastCorpAnnouncement": "Annual General Meeting\/Dividend Rs 9.50 Per Share"
},
{
"symbol": "DHFL",
"series": "EQ",
"openPrice": "91.10",
"highPrice": "93.20",
"lowPrice": "80.10",
"ltp": "83.25",
"previousPrice": "93.95",
"netPrice": "-11.39",
"tradedQuantity": "8,36,01,446",
"turnoverInLakhs": "71,303.67",
"lastCorpAnnouncementDate": "19-Jun-2018",
"lastCorpAnnouncement": "Annual General Meeting \/ Dividend- Rs 2.50 Per Share"
},
{
"symbol": "RELCAPITAL",
"series": "EQ",
"openPrice": "110.45",
"highPrice": "111.55",
"lowPrice": "97.55",
"ltp": "100.00",
"previousPrice": "111.85",
"netPrice": "-10.59",
"tradedQuantity": "3,62,53,302",
"turnoverInLakhs": "37,616.43",
"lastCorpAnnouncementDate": "12-Sep-2018",
"lastCorpAnnouncement": "Annual General Meeting\/Dividend Rs 11 Per Share"
},
{
"symbol": "PCJEWELLER",
"series": "EQ",
"openPrice": "68.00",
"highPrice": "68.85",
"lowPrice": "60.30",
"ltp": "62.00",
"previousPrice": "67.80",
"netPrice": "-8.55",
"tradedQuantity": "3,27,80,219",
"turnoverInLakhs": "21,051.46",
"lastCorpAnnouncementDate": "19-Sep-2018",
"lastCorpAnnouncement": "Annual General Meeting\/Dividend Re 0.50 Per Share"
},
{
"symbol": "JETAIRWAYS",
"series": "EQ",
"openPrice": "133.00",
"highPrice": "133.00",
"lowPrice": "122.65",
"ltp": "124.90",
"previousPrice": "133.10",
"netPrice": "-6.16",
"tradedQuantity": "50,02,823",
"turnoverInLakhs": "6,367.59",
"lastCorpAnnouncementDate": "01-Aug-2018",
"lastCorpAnnouncement": "Annual General Meeting"
},
{
"symbol": "RPOWER",
"series": "EQ",
"openPrice": "6.70",
"highPrice": "6.70",
"lowPrice": "5.95",
"ltp": "6.20",
"previousPrice": "6.60",
"netPrice": "-6.06",
"tradedQuantity": "9,77,85,958",
"turnoverInLakhs": "6,082.29",
"lastCorpAnnouncementDate": "12-Sep-2018",
"lastCorpAnnouncement": "Annual General Meeting"
},
{
"symbol": "JISLJALEQS",
"series": "EQ",
"openPrice": "45.05",
"highPrice": "46.10",
"lowPrice": "42.50",
"ltp": "42.60",
"previousPrice": "45.05",
"netPrice": "-5.44",
"tradedQuantity": "1,70,42,429",
"turnoverInLakhs": "7,498.67",
"lastCorpAnnouncementDate": "12-Sep-2018",
"lastCorpAnnouncement": "Annual General Meeting\/Dividend- Re 1 Per Share"
},
{
"symbol": "CGPOWER",
"series": "EQ",
"openPrice": "34.20",
"highPrice": "34.95",
"lowPrice": "32.00",
"ltp": "32.65",
"previousPrice": "34.30",
"netPrice": "-4.81",
"tradedQuantity": "70,36,664",
"turnoverInLakhs": "2,347.43",
"lastCorpAnnouncementDate": "16-Aug-2016",
"lastCorpAnnouncement": "Annual General Meeting (Book Closure Dates Revised)"
},
{
"symbol": "MANAPPURAM",
"series": "EQ",
"openPrice": "135.00",
"highPrice": "136.55",
"lowPrice": "122.60",
"ltp": "132.65",
"previousPrice": "137.75",
"netPrice": "-3.70",
"tradedQuantity": "2,59,15,128",
"turnoverInLakhs": "33,881.44",
"lastCorpAnnouncementDate": "27-May-2019",
"lastCorpAnnouncement": "Interim Dividend - Re 0.55 Per Share"
},
{
"symbol": "APOLLOTYRE",
"series": "EQ",
"openPrice": "198.95",
"highPrice": "199.95",
"lowPrice": "191.45",
"ltp": "192.40",
"previousPrice": "199.50",
"netPrice": "-3.56",
"tradedQuantity": "26,45,450",
"turnoverInLakhs": "5,171.33",
"lastCorpAnnouncementDate": "23-Jul-2019",
"lastCorpAnnouncement": "Annual General Meeting\/ Dividend - Rs 3.25 Per Share"
}
],
"time": "Jun 07, 2019 15:59:59"
}
更新：
import requests
import json
from pandas.io.json import json_normalize
headers = {"Accept-Language": "en-US, en;q=0.5"}
url = 'https://nseindia.com/live_market/dynaContent/live_analysis/losers/fnoLosers1.json'
response = requests.get(url)
jsonData = response.json()
print(json_normalize(jsonData['data']))
O / P：
highPrice lastCorpAnnouncement lastCorpAnnouncementDate lowPrice ... series symbol tradedQuantity turnoverInLakhs
0 86.80 Annual General Meeting/Dividend Rs 9.50 Per Share 12-Sep-2018 72.70 ... EQ RELINFRA 4,28,81,500 33,782.05
1 93.20 Annual General Meeting / Dividend- Rs 2.50 Per... 19-Jun-2018 80.10 ... EQ DHFL 8,36,01,446 71,303.67
2 111.55 Annual General Meeting/Dividend Rs 11 Per Share 12-Sep-2018 97.55 ... EQ RELCAPITAL 3,62,53,302 37,616.43
3 68.85 Annual General Meeting/Dividend Re 0.50 Per Share 19-Sep-2018 60.30 ... EQ PCJEWELLER 3,27,80,219 21,051.46
4 133.00 Annual General Meeting 01-Aug-2018 122.65 ... EQ JETAIRWAYS 50,02,823 6,367.59
5 6.70 Annual General Meeting 12-Sep-2018 5.95 ... EQ RPOWER 9,77,85,958 6,082.29
6 46.10 Annual General Meeting/Dividend- Re 1 Per Share 12-Sep-2018 42.50 ... EQ JISLJALEQS 1,70,42,429 7,498.67
7 34.95 Annual General Meeting (Book Closure Dates Rev... 16-Aug-2016 32.00 ... EQ CGPOWER 70,36,664 2,347.43
8 136.55 Interim Dividend - Re 0.55 Per Share 27-May-2019 122.60 ... EQ MANAPPURAM 2,59,15,128 33,881.44
9 199.95 Annual General Meeting/ Dividend - Rs 3.25 Per... 23-Jul-2019 191.45 ... EQ APOLLOTYRE 26,45,450 5,171.33
相关问题
最新问题
- 我写了这段代码，但我无法理解我的错误
- 我无法从一个代码实例的列表中删除 None 值，但我可以在另一个实例中。为什么它适用于一个细分市场而不适用于另一个细分市场？
- 是否有可能使 loadstring 不可能等于打印？卢阿
- java中的random.expovariate()
- Appscript 通过会议在 Google 日历中发送电子邮件和创建活动
- 为什么我的 Onclick 箭头功能在 React 中不起作用？
- 在此代码中是否有使用“this”的替代方法？
- 在 SQL Server 和 PostgreSQL 上查询，我如何从第一个表获得第二个表的可视化
- 每千个数字得到
- 更新了城市边界 KML 文件的来源？