Question

我正在尝试从每个产品页面中提取每个“综合评分”（强标签中的数字值） https://www.guitarguitar.co.uk/product/12082017334688--epiphone-les-paul-standard-plus-top-pro-translucent-blue 结构如下：

  <div class="col-sm-12"> 
   <h2 class="line-bottom"> Customer Reviews</h2>
   <h4>
   Overall Rating
   <strong>5</strong>
   <span></span>
  </h4>
  </div>

我试图仅提取强值。

 productsRating = soup.find("div", {"class": "col-sm-12"}.h4

这有时可行，但是页面对不同的元素使用相同的类，因此它提取了不需要的html元素。

有什么解决方案只能获得产品的整体评价吗？

已编辑！

这是我程序的整个循环。

for page in range(1, 2):
    guitarPage = requests.get('https://www.guitarguitar.co.uk/guitars/electric/page-{}'.format(page)).text
    soup = BeautifulSoup(guitarPage, 'lxml')
    guitars = soup.find_all(class_='col-xs-6 col-sm-4 col-md-4 col-lg-3')

    for guitar in guitars:

        title_text = guitar.h3.text.strip()
        print('Guitar Name: ', title_text)
        price = guitar.find(class_='price bold small').text.strip()
        trim = re.compile(r'[^\d.,]+')
        int_price = trim.sub('', price)
        print('Guitar Price: ', int_price)

        priceSave = guitar.find('span', {'class': 'price save'})
        if priceSave is not None:
            priceOf = priceSave.text
            trim = re.compile(r'[^\d.,]+')
            int_priceOff = trim.sub('', priceOf)
            print('Save: ', int_priceOff)
        else:
            print("No discount!")

        image = guitar.img.get('src')
        print('Guitar Image: ', image)

        productLink = guitar.find('a').get('href')
        linkProd = url + productLink
        print('Link of product', linkProd)
        productsPage.append(linkProd)

        for products in productsPage:
            response = requests.get(products)
            soup = BeautifulSoup(response.content, "lxml")
            productsDetails = soup.find("div", {"class": "description-preview"})
            if productsDetails is not None:
                description = productsDetails.text
                print('product detail: ', description)
            else:
                print('none')
            time.sleep(0.2)
            productsRating = soup.find_all('strong')[0].text
            print(productsRating)

Answer 1

尝试：

import requests
from bs4 import BeautifulSoup 

url = 'https://www.guitarguitar.co.uk/product/190319340849008--gibson-les-paul-standard-60s-iced-tea'

html = requests.get(url).text

soup = BeautifulSoup(html, "lxml")
try:
    productsRating = soup.find('h2', string=lambda s: "Customer reviews" in s).find_next_siblings()[0].find('strong').text
except:
    productsRating = None

print(productsRating)

Answer 2

评论信息全部在脚本标记中，您可以提取并使用json加载。足够简单地了解如何使其适应循环。

class Comment < ApplicationRecord
  belongs_to :article
  belongs_to :user
end

class User < ApplicationRecord
  has_many :comments, dependent: :destroy
  # Some devise things
end

class Comment < ApplicationRecord
  belongs_to :article
  belongs_to :user
end

输出：

浏览json

要处理任何评论，您可以使用简单的import requests from bs4 import BeautifulSoup as bs import json url = 'https://www.guitarguitar.co.uk/product/12082017334688--epiphone-les-paul-standard-plus-top-pro-translucent-blue' r = requests.get(url) soup = bs(r.content, 'lxml') script = soup.select_one('[type="application/ld+json"]').text data = json.loads(script.strip()) overall_rating = data['@graph'][2]['aggregateRating']['ratingValue'] reviews = [review for review in data['@graph'][2]['review']] #extract what you want：

try except

或者，使用if语句：

import requests
from bs4 import BeautifulSoup as bs
import json

url = 'https://www.guitarguitar.co.uk/product/190319340849008--gibson-les-paul-standard-60s-iced-tea'
r = requests.get(url)
soup = bs(r.content, 'lxml')
script = soup.select_one('[type="application/ld+json"]').text
data = json.loads(script.strip())
try:
    overall_rating = data['@graph'][2]['aggregateRating']['ratingValue']
    reviews = [review for review in data['@graph'][2]['review']] #extract what you want
except: #you might want to use except KeyError
    overall_rating = "None"
    reviews = ['None']

如何在h4内提取文字强？

2 个答案: