如何从网页上下载股票行情,Beautifulsoup没有获得所有内容

时间:2018-12-31 12:48:49

标签: python html web-scraping

我想从此网页https://www.oslobors.no/markedsaktivitet/#/list/shares/quotelist/ob/all/all/false中获取股票报价值

但是,在使用Beautifulsoup时,我似乎并没有获得所有内容,而且我不太了解如何更改代码以实现目标

import urllib3  
from bs4 import BeautifulSoup  
def oslobors():
http=urllib3.PoolManager()  
url = 'https://www.oslobors.no/markedsaktivitet/#/list/shares/quotelist/ob/all/all/false'  
response = http.request('GET', url)  
soup=BeautifulSoup(response.data, "html.parser")  
print(soup)  
return  
print(oslobors())

1 个答案:

答案 0 :(得分:1)

您要解析的内容是动态生成的。您可以使用任何浏览器模拟器(例如selenium),也可以尝试以下包含json响应的网址。以下是简单的方法。

0

您可能会得到(部分)结果:

import requests

url = 'https://www.oslobors.no/ob/servlets/components?type=table&generators%5B0%5D%5Bsource%5D=feed.ob.quotes.EQUITIES%2BPCC&generators%5B1%5D%5Bsource%5D=feed.merk.quotes.EQUITIES%2BPCC&filter=&view=DELAYED&columns=PERIOD%2C+INSTRUMENT_TYPE%2C+TRADE_TIME%2C+ITEM_SECTOR%2C+ITEM%2C+LONG_NAME%2C+BID%2C+ASK%2C+LASTNZ_DIV%2C+CLOSE_LAST_TRADED%2C+CHANGE_PCT_SLACK%2C+TURNOVER_TOTAL%2C+TRADES_COUNT_TOTAL%2C+MARKET_CAP%2C+HAS_LIQUIDITY_PROVIDER%2C+PERIOD%2C+MIC%2C+GICS_CODE_LEVEL_1%2C+TIME%2C+VOLUME_TOTAL&channel=a66b1ba745886f611af56cec74115a51'

res = requests.get(url)
for ticker in res.json()['rows']:
    ticker_name = ticker['values']['ITEM']
    print(ticker_name)