有没有一种方法可以实时显示变化的数据帧?

时间:2020-06-17 10:38:01

标签: python-3.x pandas user-interface web-scraping

我正在从https://www.bseindia.com/markets/PublicIssues/BSEBidDetails_ofs.aspx?flag=NR&Scripcode=541729抓取数据。数据每10秒更改一次。因此,我为脚本运行设置了一个计时器。在每次迭代的开始,它会重新初始化pandas数据框并抓取网站并将数据添加到该数据框。有什么方法可以在静态窗口中显示此数据吗?

from urllib.request import Request, urlopen
from bs4 import BeautifulSoup as soup
import pandas as pd
import time
url = 'https://www.bseindia.com/markets/PublicIssues/BSEBidDetails_ofs.aspx?flag=NRScripcode=541729'
req = Request(url , headers={'User-Agent': 'Mozilla/5.0'})
bseobj=pd.DataFrame(columns=['Price Interval','qNumber of bids','qConfirmed','qYet to be confirmed','qTotal','Confirmed','Yet to be confirmed','Total'])
while True:
   i=0
   r=[]
   lr=[]    
   webpage = urlopen(req).read()
   page_soup = soup(webpage, "html.parser")
   table=page_soup.find_all("table")

   for row in table[1]:
       for column in row.find_all("td",{"class":"body_text_table"},{"bgcolor":"#FFFFFF"}):
            val=column.get_text()
            val=val.replace(',','')
            chk=val.replace('.','')
            if chk.isdigit():
                r.append(float(val))
            elif val=="":
                r.append(0)
            else:
                r.append(val)

        lr.append(r)
        r=[]
    for i in range(2,len(lr)-2):
        bseobj=bseobj.append({'Price Interval':lr[i][0],'qNumber of bids':lr[i][1],'qConfirmed':lr[i][2],'qYet to be confirmed':lr[i][3],'qTotal':lr[i][4],'Confirmed':lr[i][5],'Yet to be confirmed':lr[i][6],'Total':lr[i][7]},ignore_index=True)
    print(bseobj)
    bseobj=bseobj.iloc[0:0]
    time.sleep(10)

0 个答案:

没有答案