如何使用python抓取嵌套标签元素

时间:2020-10-31 10:22:22

标签: python html json web-scraping

您好,我想获取标记下的一些数据,但是我找不到任何解决方案,任何人都可以了解此报废信息,并且有获取这些信息的任何简便方法< / p>

这是我的python代码

  import requests
  import json
  from bs4 import BeautifulSoup
  
  header = {'User-Agent': 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.7) Gecko/2009021910 Firefox/3.0.7'}
  
  base_url = "https://www.n11.com/super-firsatlar"
  
  r = requests.get(base_url,headers=header)
  
  if r.status_code == 200:
    soup = BeautifulSoup(r.text, 'html.parser')
    books = soup.find_all('li',attrs={"class":"column"})
  
    result=[]
    for book in books:
      title=book.find('h3').text
      link=base_url +book.find('a')['href']
      picture = base_url + book.find('img')['src']
  
      price = soup.find('a',attrs={"class":"ins"})
  
  
  
      single ={'title':title,'link':link,'picture':picture,'price':price}
      result.append(single)
      with open('book.json','w', encoding='utf-8') as f:
        json.dump(result ,f,indent=4,ensure_ascii=False)
  else:
    print(r.status_code)
  

<div class="proDetail">
  <a href="https://test.com"class="oldPrice" title="Premium">  
      
      <del>69,00 TL</del></a>
      
  <a href="https://test.com"class="newPrice" title="Premium">
     
     <ins>14,90</ins>
       
         </a>
</div>

这是我的输出

{
    "title": "Premium",
    "link": "https://test.com",
    "picture": "https://pic.gif",
    "price": null
},

1 个答案:

答案 0 :(得分:0)

您正在搜索错误的班级。首先搜索类'newPrice',以获取 a-block

price = book.find('a', attrs={'class': 'newPrice'}) 

然后,您可以在 a区块中搜索 ins-block ,例如:

price = book.find('a', attrs={'class': 'newPrice'}).find('ins')

然后您的结果将如下所示:

<ins>14,90</ins>

对于最终结果,请删除html标签:

price = book.find('a', attrs={'class': 'newPrice'}).find('ins').text.strip()