Question

我正在尝试使用Python从一行中抓取文本。我能够从同一行中获得class属性，但不能从文本中获得class属性，尝试了.text和.get_text()，但它们都不起作用。

我想念什么？

这是我的Python脚本，用于从该行获取文本：

from selenium import webdriver
from webdriver_manager.chrome import ChromeDriverManager
import time
import datetime
import csv
    
    class toy(object):
    
        browser = webdriver.Chrome(ChromeDriverManager().install())
    
        browser.get('https://continuumgames.com/product/16-tracer-racer-set/')
        time.sleep(2)
    
        try:
            test = browser.find_element_by_xpath('//*[@id="tab-additional_information"]/table/tbody/tr[3]/td').get_attribute('class')
    
        except:
            test = 'NA'
    
        try:
            upcode = browser.find_element_by_xpath('//*[@id="tab-additional_information"]/table/tbody/tr[3]/td').text
    
        except:
            upcode = 'NA'
    
    
        print(test)
        print(upcode)
    
    
        browser.close()

这是页面的HTML：

<div class="woocommerce-Tabs-panel woocommerce-Tabs-panel--additional_information panel entry-content wc-tab" id="tab-additional_information" role="tabpanel" aria-labelledby="tab-title-additional_information" style="display: none;">
 
    <table class="woocommerce-product-attributes shop_attributes">
        <tbody>
            <tr class="woocommerce-product-attributes-item woocommerce-product-attributes-item--weight">
               <th class="woocommerce-product-attributes-item__label">Weight</th>
               <td class="woocommerce-product-attributes-item__value">2.5 oz</td>
            </tr>
            <tr class="woocommerce-product-attributes-item woocommerce-product-attributes-item--dimensions">
               <th class="woocommerce-product-attributes-item__label">Dimensions</th>
               <td class="woocommerce-product-attributes-item__value">24 × 4 × 2 in</td>
            </tr>
            <tr class="woocommerce-product-attributes-item woocommerce-product-attributes-item--attribute_product_upc">
               <th class="woocommerce-product-attributes-item__label">UPC</th>
               <td class="woocommerce-product-attributes-item__value">605444972168</td>
            </tr>
        </tbody>
     </table>
</div>

这是我的奔跑：

C:\Users\Carre\scrape>python test.py

[WDM] - Current google-chrome version is 83.0.4103
[WDM] - Get LATEST driver version for 83.0.4103
[WDM] - Driver [C:\Users\Carre\.wdm\drivers\chromedriver\win32\83.0.4103.39\chromedriver.exe] found in cache

DevTools listening on ws://127.0.0.1:56807/devtools/browser/03318f43-1d26-44c7-8d90-65233969f03b
woocommerce-product-attributes-item__value

Answer 1

您的选择器可能已关闭。尝试使用Xpath。右键单击标签，然后选择复制Xpath。然后，用此替换您的代码。

upcode = browser.find_element_by_xpath('paste XPath here').text

Answer 2

我有您的解决方案，这是我处理硒不一致性时通常的绕行方式：切换到beautifulsoup4

from selenium import webdriver
import bs4
from webdriver_manager.chrome import ChromeDriverManager
import time
import datetime
import csv



class toy(object):

    browser = webdriver.Chrome(ChromeDriverManager().install())

    browser.get('https://continuumgames.com/product/16-tracer-racer-set/')
    time.sleep(2)

    try:
        test = browser.find_element_by_xpath('//*[@id="tab-additional_information"]/table/tbody/tr[3]/td').get_attribute('class')

    except:
        test = 'NA'

    try:
        upcode = browser.find_element_by_xpath('//*[@id="tab-additional_information"]/table/tbody/tr[3]/td')
        upcode = bs4.BeautifulSoup(upcode.get_attribute('outerHTML'))
        upcode = upcode.text

    except:
        upcode = 'NA'


    print(test)
    print(upcode)


    browser.close()

无法使用Python从网站上抓取数据

2 个答案: