Question

我想从以下网站抓取全文：https://www.ecb.europa.eu/press/pressconf/2016/html/is161020.en.html。也就是说，从“女士们，先生们，...” 到最后的” ...因此，您实际上可以看到，中小企业贷款与大型企业贷款之间的利差已大大降低”

但是，我的代码只在“我们现在可供您处理问题” （本文的中间）之前才进行抓取。非常感谢您能帮助我解决此问题。

代码如下：

    from bs4 import BeautifulSoup
    import urllib
    import pandas as pd
    import ssl
    import os
    import time
    import string



# function loads html source code of given url
    ssl._create_default_https_context = ssl._create_unverified_context
    user_agent = 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/11.1.2 Safari/605.1.15'
    headers = {'User-Agent':user_agent,}
    url = "https://www.ecb.europa.eu/press/pressconf/2016/html/is161020.en.html"
    req = urllib.request.Request(url, None, headers) 
    response = urllib.request.urlopen(req) 
    html = response.read()
    soup = BeautifulSoup(html, 'html.parser')

    article = soup.find('article')
    paragraphs = article.find_all('p')
    print(article)

Answer 1

全文在以下段落中：

import requests
from bs4 import BeautifulSoup
resp = requests.get('https://www.ecb.europa.eu/press/pressconf/2016/html/is161020.en.html')
soup = BeautifulSoup(resp.content, 'html5lib')
article = soup.find('article')
paragraphs = article.find_all('p')

如何从网站上抓取全文-Python 3.6

1 个答案: