Question

BeautifulSoup找不到div标签“ pl-price js-pl-price”。我在inspect元素中看到了它，但是当我运行代码时，我的代码返回“ None”。也可以在HTML中找到Div'product-details'。但是，Beautifulsoup无法找到div标签“ pl-price js-pl-price”。为什么会这样？

我的代码：

import urllib2, sys, requests
from bs4 import BeautifulSoup

site = "https://www.lowes.com/pl/Refrigerators-Appliances/4294857973?goToProdList=true"
hdr = {'User-Agent': 'Mozilla/5.0'}
req = urllib2.Request(site,headers=hdr)
page = urllib2.urlopen(req)
soup = BeautifulSoup(page, 'html.parser')

details = soup.find('div', attrs={'class': 'product-details'})
name = details.find('p', attrs={'class': 'h6 product-title js-product-title met-product-title v-spacing-large ellipsis-three-line art-plp-itemDescription'}) 
price = soup.find('div', attrs={'class': "product-pricing"})
actual_price = price.find('div', attrs={'class': "pl-price js-pl-price"})

print actual_price

来自网站的HTML：

<div class="product-details">
 <div class="product-pricing">
  <div class="pl-price js-pl-price" tabindex="-1">
  <!-- Start of Product Family Pricing -->
  <!-- Map price and savings through date present for product family -->

结果：

scrape_products.py 没有

Answer 1

如果您查看price搜索的结果，则可以在HTML中看到以下文本：

"Since Lowes.com is national in scope, we check inventory at your local store first in an effort to fulfill your order more quickly. You may find product or pricing that differ from that of your local store, but we make every effort to minimize those differences so you can get exactly what you want at the best possible price."

这会让我相信定价信息不会立即加载，因此也不会加载到BeautifulSoup解析的HTML中。您应该尝试使用Selenium的无头浏览器解决方案。

BeautifulSoup找不到标签3

1 个答案: