Question

我正在尝试从html源下载文件。例如

<a class="el" href="classcs1graphics_1_1Circle.html">Circle</a>
<a class="el" href="classcs1graphics_1_1Polygon.html">Polygon</a>

我得到了所有的href，但是我正在尝试获取href的实际内容。

下面的代码获取了上面的内容（很多内容）并迅速进行了处理。如何获得这些href的内容？预先感谢。

Ed

import urllib.request, urllib.error, urllib.parse
from lxml import html
import requests
from bs4 import BeautifulSoup

#get the data from the URL
udata = requests.get('http://www.cs1graphics.org/doc/1.0/hierarchy.html')

#feed it to BeautifulSoup
soup = BeautifulSoup(udata.text,'html.parser')

#get all the <a table records
number_list_items = soup.find_all('a')

#get the rows in the records
for li_row in number_list_items:
    print(li_row)

Answer 1

您必须循环调用requests.get()才能下载内容。它使用相对URL，因此您必须定义基本URL

base_url = 'http://www.cs1graphics.org/doc/1.0/'
#get the data from the URL
udata = requests.get(base_url + 'hierarchy.html')

#feed it to BeautifulSoup
soup = BeautifulSoup(udata.text,'html.parser')

#get all the <a table records
number_list_items = soup.find_all('a')

# get the rows in the records
for li_row in number_list_items:
    fileName = li_row['href']
    url = base_url + fileName
    print(url)      
    udata = requests.get(url)
    with open(fileName, 'w') as f:
        f.write(udata.text)

使用python3.6和BeautifulSoup。我得到的是hrefs，但无法下载它们

1 个答案: