Selenium,与Section标签有关

时间:2017-03-28 11:02:21

标签: python selenium

我一直试图抓住网站Fanduel,但我一直遇到'section'标签的问题。我想访问每个玩家的数据。到目前为止,这是我的代码:

from selenium import webdriver
from pandas.io.html import read_html

driver = webdriver.Firefox()
driver.get(https://www.fanduel.co.uk/fixtures/211/lineups/create?contestId=211-4550213')

table = driver.find_element_by_xpath('//*[@id="root"]/div/main/div/div')

<section class="lineup__main lineup__main--show_fixturelist_players">
<section class="lineup__pitch_view">
<div class="lineup__pitch" style="position: relative;">
<div style="overflow: visible; height: 0px; width: 0px;">
<div class="pitch_view pitch_view--3d" style="left: 58.5px; top: 0px; width: 700px; height: 535px;"><div class="pitch_view__wrapper" style="transform: rotateX(90deg) translateY(-20px);"><div class="pitch_view__pitch"><div class="pitch_view__position"><div class="lineup_position lineup_position--3d lineup_position--G lineup_position--active"><div class="lineup_position__label">GK</div><ul class="lineup_position__players"></ul></div></div><div class="pitch_view__position"><div class="lineup_position lineup_position--3d lineup_position--D"><div class="lineup_position__label">DEF</div><ul class="lineup_position__players"></ul></div></div><div class="pitch_view__position"><div class="lineup_position lineup_position--3d lineup_position--C"><div class="lineup_position__label">MID</div><ul class="lineup_position__players"></ul></div></div><div class="pitch_view__position"><div class="lineup_position lineup_position--3d lineup_position--F"><div class="lineup_position__label">FWD</div><ul class="lineup_position__players"></ul></div></div></div></div></div></div><div class="resize-triggers"><div class="expand-trigger"><div style="width: 848px; height: 566px;"></div></div><div class="contract-trigger"></div></div></div></section><section class="lineup__fixturelist_players" style="transform: translateY(0%);"><header class="fixturelist_players__filters"><ul class="position_filter"><li class="position_filter__item"><button type="button" class="position_filter__button position_filter__button--selected"><span class="position_filter__label">GK</span><small class="position_filter__count">0</small></button></li><li class="position_filter__item"><button type="button" class="position_filter__button"><span class="position_filter__label">DEF</span><small class="position_filter__count">0</small></button></li><li class="position_filter__item"><button type="button" class="position_filter__button"><span class="position_filter__label">MID</span><small class="position_filter__count">0</small></button></li><li class="position_filter__item"><button type="button" class="position_filter__button"><span class="position_filter__label">FWD</span><small class="position_filter__count">0</small></button></li></ul><div class="player_filters"><div class="player_filters__search"><div class="player_search search_field"><span class="search_field__icon"><svg class="icon" x="0px" y="0px" viewBox="0 0 17 17" role="img" aria-labelledby="icon-aria-label-player_search-search" style="fill: currentcolor; vertical-align: middle; width: 17px; height: 17px;"><title id="icon-aria-label-player_search-search"></title><g><path d="M16.854,16.149l-6.274-6.274c0.885-1.045,1.421-2.396,1.421-3.872c0-3.314-2.686-6-6-6c-3.314,0-6,2.686-6,6s2.686,6,6,6c1.477,0,2.827-0.536,3.872-1.421l6.274,6.274c0.098,0.098,0.226,0.146,0.354,0.146c0.128,0,0.256-0.049,0.354-0.146C17.049,16.661,17.049,16.345,16.854,16.149zM1,6.003c0-2.757,2.243-5,5-5s5,2.243,5,5s-2.243,5-5,5S1,8.76,1,6.003z"></path></g></svg></span><input type="text" role="search" class="search_field__input" value="" placeholder="Search player or team"></div></div><div class="player_filters__sort"><section class="player_sort"><button type="button" class="player_sort__button pointer"><!-- react-text: 173 -->Price hi-lo<!-- /react-text --><span class="player_sort__caret"><svg class="icon" x="0px" y="0px" viewBox="0 0 17 17" role="img" aria-labelledby="icon-aria-label-player-sort-caret" style="fill: currentcolor; vertical-align: middle; width: 1.7rem; height: 1.7rem;"><title id="icon-aria-label-player-sort-caret"></title><g><path d="M3,6 L8.5,11 L14,6 L3,6 Z"></path></g></svg></span></button></section></div></div></header><div class="fixturelist_players__list" style="position: relative;"><div style="overflow: visible; height: 0px; width: 0px;"><div aria-label="grid" class="ReactVirtualized__Grid ReactVirtualized__List player_list__virtual_scroll with-data" role="grid" tabindex="0" style="box-sizing: border-box; direction: ltr; height: 242.203px; position: relative; width: 847px; will-change: transform; overflow: auto;"><div class="ReactVirtualized__Grid__innerScrollContainer" style="width: auto; height: 4970px; max-width: 847px; max-height: 4970px; overflow: hidden;"><div id="player_211-67921" class="player_list__item" data-position="G" style="height: 71px; left: 0px; position: absolute; top: 0px; width: 100%;"><div class="player_card player_card--compact"><header class="player_card__header"><button type="button" class="pointer player_card__fullname">Heaton</button></header><div class="player_card__body"><div class="player_card__aside"><div class="player_shirt player_card__image"><img class="player_shirt__image image image--visible" src="https://cdn.fanduel.co.uk/playerimages/soc/9521.svg" alt="Burnley" title="Burnley"></div></div><div class="player_card_data"><div class="player_card_data__item player_card_data__item--fixture"><div class="data_chunk data_chunk--zeta" id="211-67921_fixture"><div class="data_chunk__value" aria-labelledby="211-67921_fixture_label"><div class="fixture_details__teams"><span class="fixture_details__team fixture_details__team--home fixture_details__team--player" title="Burnley"><span class="fixture_details__team_code">BRN</span></span><span class="fixture_details__separator condensed"> v </span><span class="fixture_details__team fixture_details__team--away" title="Tottenham Hotspur"><span class="fixture_details__team_code">TOT</span></span></div></div><div class="data_chunk__label condensed" id="211-67921_fixture_label"><time class="fixture_details__start_time" datetime="2017-04-01T14:00:00Z">Sat 3:00pm</time></div></div></div><div class="player_card_data__item player_card_data__item--played"><div class="data_chunk data_chunk--zeta" id="211-67921_matches"><div class="data_chunk__value" aria-labelledby="211-67921_matches_label">26</div><div class="data_chunk__label condensed" id="211-67921_matches_label">Matches</div></div></div><div class="player_card_data__item player_card_data__item--fppg"><div class="data_chunk data_chunk--zeta" id="211-67921_fppg"><div class="data_chunk__value" aria-labelledby="211-67921_fppg_label">15.66</div><div class="data_chunk__label condensed" id="211-67921_fppg_label">Avg Pts</div></div></div></div><button class="player_action gamma player_action--add"><div class="data_chunk data_chunk--zeta data_chunk--right"><div class="data_chunk__value">£8M</div><div class="data_chunk__label condensed">Price</div></div><svg class="icon" x="0px" y="0px" viewBox="0 0 17 17" role="img" aria-labelledby="icon-aria-label-player-action-circle-211-67921" style="fill: currentcolor; vertical-align: middle; width: 2.5rem; height: 2.5rem;"><title id="icon-aria-label-player-action-circle-211-67921"></title><g><path d="M8.5,0C3.806,0,0,3.806,0,8.5S3.806,17,8.5,17S17,13.194,17,8.5S13.194,0,8.5,0z M8.5,16C4.365,16,1,12.635,1,8.5S4.365,1,8.5,1S16,4.365,16,8.5S12.635,16,8.5,16z"></path><path d="M12.5,8H9V4.5C9,4.224,8.776,4,8.5,4S8,4.224,8,4.5V8H4.5C4.224,8,4,8.224,4,8.5S4.224,9,4.5,9H8v3.5C8,12.776,8.224,13,8.5,13S9,12.776,9,12.5V9h3.5C12.776,9,13,8.776,13,8.5S12.776,8,12.5,8z"></path></g></svg></button></div></div></div><div id="player_211-68010" class="player_list__item" data-position="G" style="height: 71px; left: 0px; position: absolute; top: 71px; width: 100%;"><div class="player_card player_card--compact"><header class="player_card__header"><button type="button" class="pointer player_card__fullname">Boruc</button></header><div class="player_card__body"><div class="player_card__aside"><div class="player_shirt player_card__image"><img class="player_shirt__image image image--visible" src="https://cdn.fanduel.co.uk/playerimages/soc/9513.svg" alt="Bournemouth" title="Bournemouth"></div></div><div class="player_card_data"><div class="player_card_data__item player_card_data__item--fixture"><div class="data_chunk data_chunk--zeta" id="211-68010_fixture"><div class="data_chunk__value" aria-labelledby="211-68010_fixture_label"><div class="fixture_details__teams"><span class="fixture_details__team fixture_details__team--home" title="Southampton"><span class="fixture_details__team_code">SOU</span></span><span class="fixture_details__separator condensed"> v </span><span class="fixture_details__team fixture_details__team--away fixture_details__team--player" title="Bournemouth"><span class="fixture_details__team_code">BOU</span></span></div></div><div class="data_chunk__label condensed" id="211-68010_fixture_label"><time class="fixture_details__start_time" datetime="2017-04-01T16:30:00Z">Sat 5:30pm</time></div></div></div><div class="player_card_data__item player_card_data__item--played"><div class="data_chunk data_chunk--zeta" id="211-68010_matches"><div class="data_chunk__value" aria-labelledby="211-68010_matches_label">27</div><div class="data_chunk__label condensed" id="211-68010_matches_label">Matches</div></div></div><div class="player_card_data__item player_card_data__item--fppg"><div class="data_chunk data_chunk--zeta" id="211-68010_fppg"><div class="data_chunk__value" aria-labelledby="211-68010_fppg_label">11.40</div><div class="data_chunk__label condensed" id="211-68010_fppg_label">Avg Pts</div></div></div></div><button class="player_action gamma player_action--add"><div class="data_chunk data_chunk--zeta data_chunk--right"><div class="data_chunk__value">£7.8M</div><div class="data_chunk__label condensed">Price</div></div><svg class="icon" x="0px" y="0px" viewBox="0 0 17 17" role="img" aria-labelledby="icon-aria-label-player-action-circle-211-68010" style="fill: currentcolor; vertical-align: middle; width: 2.5rem; height: 2.5rem;"><title id="icon-aria-label-player-action-circle-211-68010"></title><g><path d="M8.5,0C3.806,0,0,3.806,0,8.5S3.806,17,8.5,17S17,13.194,17,8.5S13.194,0,8.5,0z M8.5,16C4.365,16,1,12.635,1,8.5S4.365,1,8.5,1S16,4.365,16,8.5S12.635,16,8.5,16z"></path><path d="M12.5,8H9V4.5C9,4.224,8.776,4,8.5,4S8,4.224,8,4.5V8H4.5C4.224,8,4,8.224,4,8.5S4.224,9,4.5,9H8v3.5C8,12.776,8.224,13,8.5,13S9,12.776,9,12.5V9h3.5C12.776,9,13,8.776,13,8.5S12.776,8,12.5,8z"></path></g></svg></button></div></div></div><div id="player_211-68390" class="player_list__item" data-position="G" style="height: 71px; left: 0px; position: absolute; top: 142px; width: 100%;"><div class="player_card player_card--compact"><header class="player_card__header"><button type="button" class="pointer player_card__fullname">Robles</button></header><div class="player_card__body"><div class="player_card__aside"><div class="player_shirt player_card__image"><img class="player_shirt__image image image--visible" src="https://cdn.fanduel.co.uk/playerimages/soc/9512.svg" alt="Everton" title="Everton"></div></div><div class="player_card_data"><div class="player_card_data__item player_card_data__item--fixture"><div class="data_chunk data_chunk--zeta" id="211-68390_fixture"><div class="data_chunk__value" aria-labelledby="211-68390_fixture_label"><div class="fixture_details__teams"><span class="fixture_details__team fixture_details__team--home" title="Liverpool"><span class="fixture_details__team_code">LIV</span></span><span class="fixture_details__separator condensed"> v </span><span class="fixture_details__team fixture_details__team--away fixture_details__team--player" title="Everton"><span class="fixture_details__team_code">EVE</span></span></div></div><div class="data_chunk__label condensed" id="211-68390_fixture_label"><time class="fixture_details__start_time" datetime="2017-04-01T11:30:00Z">Sat 12:30pm</time></div></div></div><div class="player_card_data__item player_card_data__item--played"><div class="data_chunk data_chunk--zeta" id="211-68390_matches"><div class="data_chunk__value" aria-labelledby="211-68390_matches_label">14</div><div class="data_chunk__label condensed" id="211-68390_matches_label">Matches</div></div></div><div class="player_card_data__item player_card_data__item--fppg"><div class="data_chunk data_chunk--zeta" id="211-68390_fppg"><div class="data_chunk__value" aria-labelledby="211-68390_fppg_label">16.20</div><div class="data_chunk__label condensed" id="211-68390_fppg_label">Avg Pts</div></div></div></div><button class="player_action gamma player_action--add"><div class="data_chunk data_chunk--zeta data_chunk--right"><div class="data_chunk__value">£7.6M</div><div class="data_chunk__label condensed">Price</div></div><svg class="icon" x="0px" y="0px" viewBox="0 0 17 17" role="img" aria-labelledby="icon-aria-label-player-action-circle-211-68390" style="fill: currentcolor; vertical-align: middle; width: 2.5rem; height: 2.5rem;"><title id="icon-aria-label-player-action-circle-211-68390"></title><g><path d="M8.5,0C3.806,0,0,3.806,0,8.5S3.806,17,8.5,17S17,13.194,17,8.5S13.194,0,8.5,0z M8.5,16C4.365,16,1,12.635,1,8.5S4.365,1,8.5,1S16,4.365,16,8.5S12.635,16,8.5,16z"></path><path d="M12.5,8H9V4.5C9,4.224,8.776,4,8.5,4S8,4.224,8,4.5V8H4.5C4.224,8,4,8.224,4,8.5S4.224,9,4.5,9H8v3.5C8,12.776,8.224,13,8.5,13S9,12.776,9,12.5V9h3.5C12.776,9,13,8.776,13,8.5S12.776,8,12.5,8z"></path></g></svg></button></div></div></div><div id="player_211-68261" class="player_list__item" data-position="G" style="height: 71px; left: 0px; position: absolute; top: 213px; width: 100%;"><div class="player_card player_card--compact"><header class="player_card__header"><button type="button" class="pointer player_card__fullname">Pickford</button></header><div class="player_card__body"><div class="player_card__aside"><div class="player_shirt player_card__image"><img class="player_shirt__image image image--visible" src="https://cdn.fanduel.co.uk/playerimages/soc/9507.svg" alt="Sunderland" title="Sunderland"></div></div><div class="player_card_data"><div class="player_card_data__item player_card_data__item--fixture"><div class="data_chunk data_chunk--zeta" id="211-68261_fixture"><div class="data_chunk__value" aria-labelledby="211-68261_fixture_label"><div class="fixture_details__teams"><span class="fixture_details__team fixture_details__team--home" title="Watford"><span class="fixture_details__team_code">WAT</span></span><span class="fixture_details__separator condensed"> v </span><span class="fixture_details__team fixture_details__team--away fixture_details__team--player" title="Sunderland"><span class="fixture_details__team_code">SUN</span></span></div></div><div class="data_chunk__label condensed" id="211-68261_fixture_label"><time class="fixture_details__start_time" datetime="2017-04-01T14:00:00Z">Sat 3:00pm</time></div></div></div><div class="player_card_data__item player_card_data__item--played"><div class="data_chunk data_chunk--zeta" id="211-68261_matches"><div class="data_chunk__value" aria-labelledby="211-68261_matches_label">19</div><div class="data_chunk__label condensed" id="211-68261_matches_label">Matches</div></div></div><div class="player_card_data__item player_card_data__item--fppg"><div class="data_chunk data_chunk--zeta" id="211-68261_fppg"><div class="data_chunk__value" aria-labelledby="211-68261_fppg_label">16.37</div><div class="data_chunk__label condensed" id="211-68261_fppg_label">Avg Pts</div></div></div></div><button class="player_action gamma player_action--add"><div class="data_chunk data_chunk--zeta data_chunk--right"><div class="data_chunk__value">£7.5M</div><div class="data_chunk__label condensed">Price</div></div><svg class="icon" x="0px" y="0px" viewBox="0 0 17 17" role="img" aria-labelledby="icon-aria-label-player-action-circle-211-68261" style="fill: currentcolor; vertical-align: middle; width: 2.5rem; height: 2.5rem;"><title id="icon-aria-label-player-action-circle-211-68261"></title><g><path d="M8.5,0C3.806,0,0,3.806,0,8.5S3.806,17,8.5,17S17,13.194,17,8.5S13.194,0,8.5,0z M8.5,16C4.365,16,1,12.635,1,8.5S4.365,1,8.5,1S16,4.365,16,8.5S12.635,16,8.5,16z"></path><path d="M12.5,8H9V4.5C9,4.224,8.776,4,8.5,4S8,4.224,8,4.5V8H4.5C4.224,8,4,8.224,4,8.5S4.224,9,4.5,9H8v3.5C8,12.776,8.224,13,8.5,13S9,12.776,9,12.5V9h3.5C12.776,9,13,8.776,13,8.5S12.776,8,12.5,8z"></path></g></svg></button></div></div></div></div></div></div><div class="resize-triggers"><div class="expand-trigger"><div style="width: 848px; height: 243px;"></div></div><div class="contract-trigger"></div></div></div></section></section>

1 个答案:

答案 0 :(得分:0)

似乎是动态生成所需内容,因此您可能需要等到执行JavaScript代码。

尝试使用以下代码并告诉我结果:

from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait as wait
from selenium import webdriver
from pandas.io.html import read_html

driver = webdriver.Firefox()
driver.get(https://www.fanduel.co.uk/fixtures/211/lineups/create?contestId=211-4550213')

table = wait(driver, 10).until(EC.presence_of_element_located((By.XPATH, '//section[contains(@class,"lineup__main")]')))

这应该可以让您在section

生成并出现后立即找到目标DOM