我想从python的url用户那里获取输入

Question

我想从用户输入获取网站名称，最大数量为no。他希望抓取网页以抓取网站...但无法获得任何解决方案......我的代码

import requests
from bs4 import *
from urllib import request


url1 = input("Enter url you want to crawl:")
max_pages1 = int(input("Enter no. of pages you want to crawl:"))


def web_crawler(max_pages,url):
   page = 1
   while page <= max_pages:
      url4 = str(url) + str(page)
      url_get = requests.get(url4)
      plain_text = url_get.text
      soup = BeautifulSoup(plain_text,"html.parser")
      for a in soup.findAll('a',{'rel':'bookmark'}):
          href = a.get('href')
          title = a.string
          #print(title)
          print(href)
          #info_about_web_pages(href)
      page +=1

def info_about_web_pages(url):
   url_get = requests.get(url)
   plain_text = url_get.text
   soup = BeautifulSoup(plain_text,"html.parser" )
   links = set()
   for about in soup.findAll('a'):
       href = about.get('href')
       links.update([href])

   print(links)

web_crawler(max_pages1,url1)

它在输出中没有显示任何内容

Answer 1

如果没有你想要在html源代码中找到的属性的锚，那么这将始终不打印任何内容。尝试打印soup.prettify（）并查看您要查找的标签是否存在。通常情况下，当我不打印我期待它的值时，因为该值没有我正在寻找的属性。