从网站上抓取不断变化的整数值

时间:2018-10-30 05:23:27

标签: python-3.x web-scraping beautifulsoup

from bs4 import BeautifulSoup
import requests

url = "http://www.oanda.com/currency/live-exchange-rates/"
print(url)
response = requests.get(url)
soup = BeautifulSoup(response.content,'html.parser')
print(soup)

data1 = soup.find('span',class_="inline_int")
for data in data1:
    raw_data = data.text
    print(data1)

这种方法通常会提供来自网站的数据。但是,它并没有提供我需要的任何东西。我已经尝试过并返回[]。有谁知道另一种可以解决这个问题的方法?

1 个答案:

答案 0 :(得分:0)

以下方法应获取不断变化的值。由于内容是动态的,因此我使用硒来收集它们。

from selenium import webdriver
from selenium.webdriver.support import ui

driver = webdriver.Chrome()
wait = ui.WebDriverWait(driver, 10)
driver.get("http://www.oanda.com/currency/live-exchange-rates/")
for items in wait.until(lambda driver: driver.find_elements_by_css_selector(".inline_rates_container .rate_row")):
    data = [item.text for item in items.find_elements_by_css_selector(".left a, .right")]
    print(data)
driver.quit()

这是您应该获得的输出的一部分:

['EUR/USD', '1.13656', '1.13643', '', 'USD/EUR', '0.87995', '0.87985', '']
['GBP/USD', '1.27762', '1.27743', '', 'USD/GBP', '0.78282', '0.78271', '']
['USD/CAD', '1.31152', '1.31134', '', 'CAD/USD', '0.76258', '0.76247', '']
['USD/CHF', '1.00257', '1.00243', '', 'CHF/USD', '0.99758', '0.99744', '']
['USD/JPY', '112.798', '112.785', '', 'JPY/USD', '0.00887', '0.00887', '']
['EUR/GBP', '0.88966', '0.88951', '', 'GBP/EUR', '1.12421', '1.12402', '']