Question

我有一个循环遍历网站多个页面的脚本，我想跳过或为某些页面上可能没有的项目添加空格。例如，有些页面不包含许可证。当我遇到其中一个页面时，我收到属性错误。下面我的脚本循环显示前两页没有问题，但当它到达第三页时它停止。我怎样才能解决这个问题？这是我的剧本：

from urllib.request import urlopen
from bs4 import BeautifulSoup as soup
import json

base_url = "https://open.umn.edu/opentextbooks/"

data = []
n = 50
for i in range(4, n+1):
   response = urlopen(base_url + "BookDetail.aspx?bookId=" + str(i))
   page_html = response.read()
   response.close()

   #html parsing
   page_soup = soup(page_html, "html.parser")

   #grabs info for each textbook
   containers = page_soup.findAll("div",{"class":"LongDescription"})
   author = page_soup.select("p")

   for container in containers:
       item = {}
       item['type'] = "Textbook"
       item['title'] = container.find("div",{"class":"twothird"}).h1.text
       item['author'] = author[3].get_text(separator=', ')
       if item['author'] == " ":
          item['author'] = "University of Minnesota Libraries Publishing"
       item['link'] = "https://open.umn.edu/opentextbooks/BookDetail.aspx?bookId=" + str(i)
       item['source'] = "Open Textbook Library"
       item['base_url'] = "https://open.umn.edu/opentextbooks/"
       item['license'] = container.find("p",{"class":"Badge-Condition"}).a.text
       if item['license'] != container.find("p",{"class":"Badge-Condition"}).a.text:
          item['license'] = ""
       item['license_url'] = container.find("p",{"class":"Badge-Condition"}).a["href"]
       data.append(item) # add the item to the list

   with open("./json/noSubject/otl-loop.json", "w") as writeJSON:
      json.dump(data, writeJSON, ensure_ascii=False)

Answer 1

我明白了。我的主要问题是item['license']这是我的修复：

if container.find("p",{"class":"Badge-Condition"}).a:
        item['license'] = container.find("p",{"class":"Badge-Condition"}).a.text
if container.find("img",{"class":"ctl00_maincontent_imgLicence"}):
        item['license'] = ''
if container.find("p",{"class":"Badge-Condition"}).a:
        item['license_url'] = container.find("p",{"class":"Badge-Condition"}).a["href"]
if container.find("img",{"class":"ctl00_maincontent_imgLicence"}):
        item['license_url'] = ''

如果在循环遍历多个页面时元素不存在，则跳过项目 - BeautifulSoup和Python

1 个答案: