Question

我想浏览当前在本地托管的网站。是否有可能爬网本地托管的网站？我收到此错误：

 File "C:/Users/hero/PycharmProjects/project/Crawler.py", line 22, in <module>
    imagefile.write(urllib.request.urlopen("http://192.168.1.1/Webpage.html"+img_src).read())
urllib.error.HTTPError: HTTP Error 404: Not Found

搜寻器的代码：

import urllib.request
from bs4 import BeautifulSoup


def make_soup(url):
    thepage = urllib.request.urlopen(url)
    soupdata = BeautifulSoup(thepage, "html.parser")
    return soupdata


i = 1
soup = make_soup("http://192.168.1.1/Webpage.html")

unique_srcs = []
for img in soup.findAll('img'):
    if img.get('src') not in unique_srcs:
        unique_srcs.append(img.get('src'))
for img_src in unique_srcs:
    filename = str(i)
    i = i + 1
    imagefile = open(filename + '.png', 'wb')
    imagefile.write(urllib.request.urlopen("http://192.168.1.1/Webpage.html"+img_src).read())
    imagefile.close()

Answer 1

您忘记在网址路径中添加斜杠/。

只需将行更改为以下内容：

imagefile.write(urllib.request.urlopen("http://192.168.1.1/Webpage.html/"+img_src).read())

通过本地托管的网站爬网时出错

1 个答案: