Question

我正在尝试使用此代码来刮取网站上的产品名称。我正在尝试使用特定的类来查找p标签。这是代码，运行时它只打印出一个。我要抓取的元素已评论。

#<p class="product-name">Yaesu FT-DX101D HF/50MHz 100W SDR</p>


import requests
import urllib.request
import time
from bs4 import BeautifulSoup

url = "https://www.gigaparts.com/products/radios-amps-and-repeaters#/?Category1=Radios&Category2=Radios%2C+Amps+and+Repeaters&Category3=Radio+Transceivers&search_return=all&Category4=Base+Stations"
response = requests.get(url)
soup = BeautifulSoup(response.text, "html.parser")
for i in range(10):
    a_tags = soup.find("p", {"class": "product-name"})
    print(a_tags)
    time.sleep(2)

Answer 1

您可以模仿jhr xhr，但删除所有多余的东西，这些东西看起来很可能随时间变化。

import requests, re, json, ast
from bs4 import BeautifulSoup

r = requests.get('https://gigaparts-v2.ecomm-nav.com/nav.js?initial_url=https%3A%2F%2Fwww.gigaparts.com%2Fproducts%2Fradios-amps-and-repeaters%23%2F%3FCategory1%3DRadios%26Category2%3DRadios%252C%2BAmps%2Band%2BRepeaters%26Category3%3DRadio%2BTransceivers%26search_return%3Dall%26Category4%3DBase%2BStations&nxt_custom_options=formKey%3D%26groupId%3DNOT%2BLOGGED%2BIN&Category1=Radios&Category2=Radios%2C+Amps+and+Repeaters&Category3=Radio+Transceivers&search_return=all&Category4=Base+Stations&callback=jQuery0')
p = re.compile(r'jQuery0\((.*)\);')
d = ast.literal_eval(p.findall(r.text)[0])
soup = bs(d['content'], 'lxml')
product_names = [i.text for i in soup.select('.product-name')]
print(product_names )

BS4按类别查找p标签

1 个答案: