使用硒进行网页抓取没有获取任何价值

时间:2021-02-03 12:41:29

标签: python selenium web-scraping

我不知道这里有什么问题,代码似乎可以正常工作,但我没有得到任何文本值,请帮忙

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

PATH = r"C:\Users\KRISHNA\Downloads\chromedriver.exe"

url = "https://cricketexchange.in/scoreboard/8NA/K9/19th-T10/7A/7B/Abu-Dhabi-T10-League-2021-QLD-vs-BGT/live"

driver = webdriver.Chrome(PATH)
driver.get(url)

score = driver.find_elements_by_class_name("row")

try:
    search = WebDriverWait(driver, 40).until(
        EC.presence_of_all_elements_located(
            (By.XPATH, '/html/body/app-root/app-match-details/div/div/div/div[1]/div[1]/div[2]/div[1]/div[1]/div[2]/div[1]'))
    )
except:
    driver.quit()

for s in score:
    batting_team = s.find_element_by_xpath('/html/body/app-root/app-match-details/div/div/div/div[1]/div[1]/div[2]/div[1]/div[1]/div[2]/div[1]').text
    runs = s.find_element_by_xpath('/html/body/app-root/app-match-details/div/div[1]/div/div[1]/div[1]/div[2]/div[1]/div[1]/div[2]/div[2]/span[1]').text
    over = s.find_element_by_xpath('/html/body/app-root/app-match-details/div/div/div/div[1]/div[1]/div[2]/div[1]/div[1]/div[2]/div[2]/span[2]').text
    print(batting_team, runs, over)

1 个答案:

答案 0 :(得分:1)

你的问题是变量score是在页面加载之前声明的,之前

try:
   search = WebDriverWait(driver, 40).until(
       EC.presence_of_all_elements_located(
           (By.XPATH, '/html/body/app-root/app-match-details/div/div/div/div[1]/div[1]/div[2]/div[1]/div[1]/div[2]/div[1]'))
   )
except:
   driver.quit()

那段代码。我在 try-except 语句运行后声明了 score,它对我有用。

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

PATH = r"C:\Users\KRISHNA\Downloads\chromedriver.exe"

url = "https://cricketexchange.in/scoreboard/8NA/K9/19th-T10/7A/7B/Abu-Dhabi-T10-League-2021-QLD-vs-BGT/live"

driver = webdriver.Chrome(PATH)
driver.get(url)

try:
    search = WebDriverWait(driver, 40).until(
        EC.presence_of_all_elements_located(
            (By.XPATH, '/html/body/app-root/app-match-details/div/div/div/div[1]/div[1]/div[2]/div[1]/div[1]/div[2]/div[1]'))
    )
except:
    driver.quit()

score = driver.find_elements_by_class_name("row") # move this below the try-except statement

for s in score:
    batting_team = s.find_element_by_xpath('/html/body/app-root/app-match-details/div/div/div/div[1]/div[1]/div[2]/div[1]/div[1]/div[2]/div[1]').text
    runs = s.find_element_by_xpath('/html/body/app-root/app-match-details/div/div[1]/div/div[1]/div[1]/div[2]/div[1]/div[1]/div[2]/div[2]/span[1]').text
    over = s.find_element_by_xpath('/html/body/app-root/app-match-details/div/div/div/div[1]/div[1]/div[2]/div[1]/div[1]/div[2]/div[2]/span[2]').text
    print(batting_team, runs, over)

如果这不起作用,请在评论中告诉我。