Question

我正在练习抓取网站，所以我选择了一个网站 https://www.dunzo.com/bangalore/nilgiris-supermarket-koramangala-ejipura

这是我正在使用的代码

url="https://www.dunzo.com/bangalore/nilgiris-supermarket-koramangala-ejipura"
r=requests.get(url)
htmlcontnent=r.content 
soup=BeautifulSoup(htmlcontnent,'html.parser')
elem=soup.select('.hozIhp')
print(elem)

现在我得到的输出为：

[<p class="sc-1gu8y64-0 dlNpIS sc-1twyv6b-1 hozIhp">Britannia Sweet Bread</p>, <p class="sc-1gu8y64-0 dlNpIS sc-1twyv6b-1 hozIhp">Britannia Sweet Bun</p>, <p class="sc-1gu8y64-0 dlNpIS sc-1twyv6b-1 hozIhp">Nilgiris Cheese Garlic Bread</p>, <p class="sc-1gu8y64-0 dlNpIS sc-1twyv6b-1 hozIhp">Nilgiris Fruit Bread</p>, <p class="sc-1gu8y64-0 dlNpIS sc-1twyv6b-1 hozIhp">Nilgiris Pav Bun</p>, <p class="sc-1gu8y64-0 dlNpIS sc-1twyv6b-1 hozIhp">Nilgiri's Broken Wheat Bread</p>, <p class="sc-1gu8y64-0 dlNpIS sc-1twyv6b-1 hozIhp">Nilgiri's Garlic Bread</p>, <p class="sc-1gu8y64-0 dlNpIS sc-1twyv6b-1 hozIhp">Nilgiri's Multi Grain Bread</p>, <p class="sc-1gu8y64-0 dlNpIS sc-1twyv6b-1 hozIhp">Nilgiri's Whole Wheat Bread</p>, <p class="sc-1gu8y64-0 dlNpIS sc-1twyv6b-1 hozIhp">Nilgiri's Whole Wheat Brown Bread</p>]

因此输出以列表的形式出现。现在我想提取项目名称，例如 Britannia Sweet Bread、Britannia Sweet Bun、Nilgiris Cheese Garlic Bread 等。我尝试了一些方法，例如在汤中添加 .text ，但没有用。有人可以帮我怎么做吗？？

Answer 1

试试这个：

url="https://www.dunzo.com/bangalore/nilgiris-supermarket-koramangala-ejipura"
r=requests.get(url)
htmlcontnent=r.content 
soup=BeautifulSoup(htmlcontnent,'html.parser')
elem=soup.select('.hozIhp')
print(*[el.text for el in elem], sep="\n")

输出：

Britannia Sweet Bread
Britannia Sweet Bun
Nilgiris Cheese Garlic Bread
Nilgiris Fruit Bread
Nilgiris Pav Bun
Nilgiri's Broken Wheat Bread
Nilgiri's Garlic Bread
Nilgiri's Multi Grain Bread
Nilgiri's Whole Wheat Bread
Nilgiri's Whole Wheat Brown Bread

Answer 2

url="https://www.dunzo.com/bangalore/nilgiris-supermarket-koramangala-ejipura"
r=requests.get(url)
htmlcontnent=r.content 
soup=BeautifulSoup(htmlcontnent,'html.parser')
elem=soup.select('.hozIhp')
#add to your code
for item in elem:
    print(item.text)

Answer 3

您遇到的问题；页面正在动态加载，requests 无法加载整个页面

要解决此问题，您需要更多代码首先，使用 pip install selenium 安装 selenium 从 https://chromedriver.chromium.org/downloads 下载可压缩的 Google Chrome 网络驱动程序（您的计算机上必须安装 Google chrome）将 Web 驱动程序提取到与您的 python 脚本相同的文件夹中

然后运行这段代码

from bs4 import BeautifulSoup
from selenium import webdriver
import time
 
browser = webdriver.Chrome(executable_path="chromedriver")

url="https://www.dunzo.com/bangalore/nilgiris-supermarket-koramangala-ejipura"
browser.get(url)

#the browser will scroll down for 7 times to load the remaining contents
for i in range (0,6):
    browser.execute_script("window.scrollTo(0, document.body.scrollHeight)")
    #waits for 5 seconds the content to load(You can adjust value depending on your internet speed)
    time.sleep(5)

html = browser.page_source
#r=requests.get(url)
#htmlcontnent=r.content 
soup=BeautifulSoup(html,'html.parser')
elem=soup.select('.hozIhp')
for item in elem:
    print(item.text)
    
browser.close()

输出

Britannia Sweet Bread
Britannia Sweet Bun
Nilgiris Cheese Garlic Bread
Nilgiris Fruit Bread
Nilgiris Pav Bun
Nilgiri's Broken Wheat Bread
Nilgiri's Garlic Bread
Nilgiri's Multi Grain Bread
Nilgiri's Whole Wheat Bread
Nilgiri's Whole Wheat Brown Bread
Nilgiri's Milk Bread
Nilgiri's Sandwich Bread
Bajaj White Eggs Gold Pack
Suguna Healthy Eggs
Eggs
Nandini - Shubham Pasteurized Standardized Milk
Nandini Good Life Slim Milk
Nilgiris Lite Milk
Nilgiris Double Toned Milk
Nilgiris Full Cream Milk
Nilgiri's Rich Milk
Amul Premium Dahi
Amul Cheese Slices A+
Cavin's Curd Pouch
Epigamia Mishti Doi
Id Natural Curd
Milky Mist Mango Yogurt
Nilgiris Curd Lite
Nilgiris Low Fat Probiotic Curd
Nilgiris Paneer
Nestle A+ Nourish Dahi
Nilgiris Natural Curd Set
Nilgiris Butter Milk
Nilgiri's Toned Milk Curd Pouch
Nilgiri's Lite Curd Pouch
Nilgiri's Malai Paneer
Soulfull Choco And Vanilla Fills - Ragi Bites
Soulfull Choco Fills - Ragi Bites
Soulfull Vanilla Fills - Ragi Bites
Soulfull Strawberry Fills - Ragi Bites
Soulfull Diet Millet Muesli
Soulfull Fruit & Nut Millet Muesli
Soulfull Crunchy Millet Muesli
Soulfull Baked Desi Muesli - Chatpata
Soulfull Baked Desi Muesli - Masala
Kellogg's Corn Flakes
Fortune Mini Soya Chunks
Kellogg's Chocos Moon And Stars
Soulfull Millet Smoothix - Cocoa Lite Protein Drink Sachets
Soulfull Millet Smoothix - Almond Protein Drink Sachets
Soulfull Millet Smoothix - Almond Protein Drink Sachets
Soulfull Millet Smoothix - Cocoa Lite Protein Drink Sachets

Answer 4

如 documentation 中所述，您可以使用 get_text() 从文档或标签中提取文本

无法使用 python 抓取网站

4 个答案: