Question

我试图在Python 3中使用BeautifulSoup废弃https://www.crowdcube.com/investments?sector=technology。

Traceback (most recent call last):

      File "D:\DataVisualization\lib\urllib\request.py", line 163, in urlopen
        return opener.open(url, data, timeout)
      File "D:\DataVisualization\lib\urllib\request.py", line 472, in open
        response = meth(req, response)
      File "D:\DataVisualization\lib\urllib\request.py", line 582, in http_response
        'http', request, response, code, msg, hdrs)
      File "D:\DataVisualization\lib\urllib\request.py", line 510, in error
        return self._call_chain(*args)
      File "D:\DataVisualization\lib\urllib\request.py", line 444, in _call_chain
        result = func(*args)
      File "D:\DataVisualization\lib\urllib\request.py", line 590, in http_error_default
        raise HTTPError(req.full_url, code, msg, hdrs, fp)
    urllib.error.HTTPError: HTTP Error 403: Forbidden

Answer 1

使用请求，此网站不需要UA：

In [23]: import requests

In [24]: r = requests.get('https://www.crowdcube.com/investments?sector=technology')

In [25]: r.status_code
Out[25]: 200

我不能用美丽的汤刮一个网页

1 个答案: