从相应的div获取所有网址

时间:2019-10-31 22:13:13

标签: python selenium web-scraping

我正在尝试从每个div获取数据URL。我在另一个页面上使用了class标签,但是这些元素在div上没有播放器类。我将如何刮擦这样的桌子?

from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.common.exceptions import TimeoutException

browser = webdriver.Chrome("/ProgramData/chocolatey/bin/chromedriver.exe")

browser.get("https://rotogrinders.com/game-stats?site=fanduel&sport=nba")
# Wait 20 seconds for page to load
timeout = 20
try:
    WebDriverWait(browser, timeout).until(
        EC.visibility_of_element_located((By.CLASS_NAME, 'player-popup')))
except TimeoutException:
    print("Timed out waiting for page to load")
    browser.quit()

# find_elements_by_xpath returns an array of selenium objects.

players_info = []

players = browser.find_elements_by_tag_name('a.player-popup')

for player in players:
    link = player.find_element_by_class_name('player-popup')
    players_info.append(player.get_attribute('href'))

players_name = [element.split("/", maxsplit=4)[-1] for element in players_info]
print(players_name)
browser.quit()

[![enter image description here][1]][1]


  [1]: https://i.stack.imgur.com/kN6sN.png

1 个答案:

答案 0 :(得分:0)

function Options({ votes, originalPost, upVote, downVote }) {
  let styling = '#696969';

  if (count > 0) {
    styling = '#008000';
  } else if (count < 0) {
    styling = '#B22222';
  } else {
    styling = '#696969';
  }

  return (
    <div>
      <button title="Down vote" onClick={downVote} className="buttons">
        -
      </button>

      <span title="Vote counter" style={{ color: styling }} className="counter">
        {votes}
      </span>

      <button title="Up vote" onClick={upVote} className="buttons">
        +
      </button>
      <br></br>

      <a
        href={originalPost}
        target="_blank"
        title="Click to check the original post"
        rel="noopener noreferrer"
        className="link"
      >
        Original post
      </a>
    </div>
  );
}