结合使用Selenium和Python解析表数据

Question

<table class="table table-striped">
    <thead>
    <tr class="reactable-column-header">
        <th class="reactable-th-status reactable-header-sortable " role="button" tabindex="0"><strong></strong></th>
        <th class="reactable-th-question_id reactable-header-sortable reactable-header-sort-asc" role="button"
            tabindex="0"><strong>#</strong></th>
        <th class="reactable-th-question_title reactable-header-sortable " role="button" tabindex="0">
            <strong>Title</strong></th>
        <th class="reactable-th-editorial reactable-header-sortable " role="button" tabindex="0">
            <strong>Solution</strong></th>
        <th class="reactable-th-acceptance reactable-header-sortable " role="button" tabindex="0">
            <strong>Acceptance</strong></th>
        <th class="reactable-th-difficulty reactable-header-sortable " role="button" tabindex="0">
            <strong>Difficulty</strong></th>
        <th class="reactable-th-frequency reactable-header-sortable " role="button" tabindex="0"><strong>Frequency
            &nbsp;<span id="frequency-tooltip" class="fa fa-lock" data-toggle="tooltip" data-placement="top" title=""
                        data-original-title="Only premium members can see the frequency"></span></strong></th>
    </tr>
    </thead>
    <tbody class="reactable-data">
    <tr>
        <td label="[object Object]"></td>
        <td label="[object Object]">1</td>
        <td value="Two Sum" label="[object Object]">
            <div><a href="/problems/two-sum">Two Sum</a>&nbsp;&nbsp;&nbsp;&nbsp;</div>
        </td>
        <td label="[object Object]"><a href="/articles/two-sum"><i class="fa fa-file-text"></i></a></td>
        <td value="44.23248982536708" label="[object Object]">44.2%</td>
        <td value="[object Object]" label="[object Object]"><span class="label label-success round">Easy</span></td>
        <td label="[object Object]"></td>
    </tr>
    <tr>
        <td label="[object Object]"></td>
        <td label="[object Object]">2</td>
        <td value="Add Two Numbers" label="[object Object]">
            <div><a href="/problems/add-two-numbers">Add Two Numbers</a>&nbsp;&nbsp;&nbsp;&nbsp;</div>
        </td>
        <td label="[object Object]"><a href="/articles/add-two-numbers"><i class="fa fa-file-text"></i></a></td>
        <td value="31.20978757805531" label="[object Object]">31.2%</td>
        <td value="[object Object]" label="[object Object]"><span class="label label-warning round">Medium</span></td>
        <td label="[object Object]"></td>
    </tr>
    <tr>
        <td label="[object Object]"></td>
    </tbody>
</table>

上面的HTML表示图像中显示的两行。 我想遍历表格行，并使用Selenium和Python打印出标题（两个总和并加上两个数字）。

但是，表结构太复杂了，我不确定如何制作一个通用函数，该函数可能适用于具有更多行的更大表。

有帮助吗？

Answer 1

如果您使用selenium并跟随xpath，它将返回表主体中行下方的所有单元格。

//table[@class='table table-striped']/tbody[@class='reactable-data']//tr//td

但是，您需要找出单元格index才能找到单元格内的particular text或particular tag。

(Two Sum Add Two Numbers)

在这种情况下，您的xpath应该是

//table[@class='table table-striped']/tbody[@class='reactable-data']//tr//td[3]//a

要处理动态元素，最好引出WebdriverWait

这是您的完整代码。

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

driver=webdriver.Chrome(executable_path='path/to/chromedriver')
driver.get("url")
elements=WebDriverWait(driver,20).until(EC.visibility_of_all_elements_located((By.XPATH,"  //table[@class='table table-striped']/tbody[@class='reactable-data']//tr//td[3]//a")))
for ele in elements:
    print(ele.text)

输出将打印在控制台上。

Two Sum
Add Two Numbers