Question

我试图使用BeautifulSoup和Requests抓取一个网站。我面临的问题是我得到了一些结果，然后程序无限期挂起。然后，我尝试使用timeout函数，它返回了一个错误。这是我正在使用的代码：

import requests
from bs4 import BeautifulSoup
all_data=[]
for i in range(1950,2020):
    x = requests.get("https://indiankanoon.org/browse/supremecourt/%s/"%(i), timeout = 5)
    soup = BeautifulSoup(x.content, 'html.parser')
    data = [x["href"] for x in soup.find_all('a',{'href': re.compile(r'/search/')})]
    all_data.append(data)

我尝试在headers中使用requests.get()，但这并没有解决。这是我得到的错误：

----------------------
1950
-----------------------
['/search/?formInput=doctypes:supremecourt fromdate:1-1-1950 todate: 31-1-1950', '/search/?formInput=doctypes:supremecourt fromdate:1-2-1950 todate: 28-2-1950', '/search/?formInput=doctypes:supremecourt fromdate:1-3-1950 todate: 31-3-1950', '/search/?formInput=doctypes:supremecourt fromdate:1-4-1950 todate: 30-4-1950', '/search/?formInput=doctypes:supremecourt fromdate:1-5-1950 todate: 31-5-1950', '/search/?formInput=doctypes:supremecourt fromdate:1-6-1950 todate: 30-6-1950', '/search/?formInput=doctypes:supremecourt fromdate:1-7-1950 todate: 31-7-1950', '/search/?formInput=doctypes:supremecourt fromdate:1-8-1950 todate: 31-8-1950', '/search/?formInput=doctypes:supremecourt fromdate:1-9-1950 todate: 30-9-1950', '/search/?formInput=doctypes:supremecourt fromdate:1-10-1950 todate: 31-10-1950', '/search/?formInput=doctypes:supremecourt fromdate:1-11-1950 todate: 30-11-1950', '/search/?formInput=doctypes:supremecourt fromdate:1-12-1950 todate: 31-12-1950']
----------------------
1951
-----------------------
['/search/?formInput=doctypes:supremecourt fromdate:1-1-1951 todate: 31-1-1951', '/search/?formInput=doctypes:supremecourt fromdate:1-2-1951 todate: 28-2-1951', '/search/?formInput=doctypes:supremecourt fromdate:1-3-1951 todate: 31-3-1951', '/search/?formInput=doctypes:supremecourt fromdate:1-4-1951 todate: 30-4-1951', '/search/?formInput=doctypes:supremecourt fromdate:1-5-1951 todate: 31-5-1951', '/search/?formInput=doctypes:supremecourt fromdate:1-6-1951 todate: 30-6-1951', '/search/?formInput=doctypes:supremecourt fromdate:1-7-1951 todate: 31-7-1951', '/search/?formInput=doctypes:supremecourt fromdate:1-8-1951 todate: 31-8-1951', '/search/?formInput=doctypes:supremecourt fromdate:1-9-1951 todate: 30-9-1951', '/search/?formInput=doctypes:supremecourt fromdate:1-10-1951 todate: 31-10-1951', '/search/?formInput=doctypes:supremecourt fromdate:1-11-1951 todate: 30-11-1951', '/search/?formInput=doctypes:supremecourt fromdate:1-12-1951 todate: 31-12-1951']
----------------------
1952
-----------------------
['/search/?formInput=doctypes:supremecourt fromdate:1-1-1952 todate: 31-1-1952', '/search/?formInput=doctypes:supremecourt fromdate:1-2-1952 todate: 29-2-1952', '/search/?formInput=doctypes:supremecourt fromdate:1-3-1952 todate: 31-3-1952', '/search/?formInput=doctypes:supremecourt fromdate:1-4-1952 todate: 30-4-1952', '/search/?formInput=doctypes:supremecourt fromdate:1-5-1952 todate: 31-5-1952', '/search/?formInput=doctypes:supremecourt fromdate:1-6-1952 todate: 30-6-1952', '/search/?formInput=doctypes:supremecourt fromdate:1-7-1952 todate: 31-7-1952', '/search/?formInput=doctypes:supremecourt fromdate:1-8-1952 todate: 31-8-1952', '/search/?formInput=doctypes:supremecourt fromdate:1-9-1952 todate: 30-9-1952', '/search/?formInput=doctypes:supremecourt fromdate:1-10-1952 todate: 31-10-1952', '/search/?formInput=doctypes:supremecourt fromdate:1-11-1952 todate: 30-11-1952', '/search/?formInput=doctypes:supremecourt fromdate:1-12-1952 todate: 31-12-1952']
----------------------
1953
-----------------------
---------------------------------------------------------------------------
timeout                                   Traceback (most recent call last)
~/.pyenv/versions/3.7.3/lib/python3.7/site-packages/urllib3/connectionpool.py in _make_request(self, conn, method, url, timeout, chunked, **httplib_request_kw)
    375         try:
--> 376             self._validate_conn(conn)
    377         except (SocketTimeout, BaseSSLError) as e:

~/.pyenv/versions/3.7.3/lib/python3.7/site-packages/urllib3/connectionpool.py in _validate_conn(self, conn)
    993         if not getattr(conn, "sock", None):  # AppEngine might not have  `.sock`
--> 994             conn.connect()
    995 

~/.pyenv/versions/3.7.3/lib/python3.7/site-packages/urllib3/connection.py in connect(self)
    393             server_hostname=server_hostname,
--> 394             ssl_context=context,
    395         )

~/.pyenv/versions/3.7.3/lib/python3.7/site-packages/urllib3/util/ssl_.py in ssl_wrap_socket(sock, keyfile, certfile, cert_reqs, ca_certs, server_hostname, ssl_version, ciphers, ssl_context, ca_cert_dir, key_password)
    369         if HAS_SNI and server_hostname is not None:
--> 370             return context.wrap_socket(sock, server_hostname=server_hostname)
    371 

~/.pyenv/versions/3.7.3/lib/python3.7/ssl.py in wrap_socket(self, sock, server_side, do_handshake_on_connect, suppress_ragged_eofs, server_hostname, session)
    411             context=self,
--> 412             session=session
    413         )

~/.pyenv/versions/3.7.3/lib/python3.7/ssl.py in _create(cls, sock, server_side, do_handshake_on_connect, suppress_ragged_eofs, server_hostname, context, session)
    852                         raise ValueError("do_handshake_on_connect should not be specified for non-blocking sockets")
--> 853                     self.do_handshake()
    854             except (OSError, ValueError):

~/.pyenv/versions/3.7.3/lib/python3.7/ssl.py in do_handshake(self, block)
   1116                 self.settimeout(None)
-> 1117             self._sslobj.do_handshake()
   1118         finally:

timeout: _ssl.c:1039: The handshake operation timed out

During handling of the above exception, another exception occurred:

ReadTimeoutError                          Traceback (most recent call last)
~/.pyenv/versions/3.7.3/lib/python3.7/site-packages/requests/adapters.py in send(self, request, stream, timeout, verify, cert, proxies)
    448                     retries=self.max_retries,
--> 449                     timeout=timeout
    450                 )

~/.pyenv/versions/3.7.3/lib/python3.7/site-packages/urllib3/connectionpool.py in urlopen(self, method, url, body, headers, retries, redirect, assert_same_host, timeout, pool_timeout, release_conn, chunked, body_pos, **response_kw)
    719             retries = retries.increment(
--> 720                 method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
    721             )

~/.pyenv/versions/3.7.3/lib/python3.7/site-packages/urllib3/util/retry.py in increment(self, method, url, response, error, _pool, _stacktrace)
    399             if read is False or not self._is_method_retryable(method):
--> 400                 raise six.reraise(type(error), error, _stacktrace)
    401             elif read is not None:

~/.pyenv/versions/3.7.3/lib/python3.7/site-packages/urllib3/packages/six.py in reraise(tp, value, tb)
    734                 raise value.with_traceback(tb)
--> 735             raise value
    736         finally:

~/.pyenv/versions/3.7.3/lib/python3.7/site-packages/urllib3/connectionpool.py in urlopen(self, method, url, body, headers, retries, redirect, assert_same_host, timeout, pool_timeout, release_conn, chunked, body_pos, **response_kw)
    671                 headers=headers,
--> 672                 chunked=chunked,
    673             )

~/.pyenv/versions/3.7.3/lib/python3.7/site-packages/urllib3/connectionpool.py in _make_request(self, conn, method, url, timeout, chunked, **httplib_request_kw)
    378             # Py2 raises this as a BaseSSLError, Py3 raises it as socket timeout.
--> 379             self._raise_timeout(err=e, url=url, timeout_value=conn.timeout)
    380             raise

~/.pyenv/versions/3.7.3/lib/python3.7/site-packages/urllib3/connectionpool.py in _raise_timeout(self, err, url, timeout_value)
    330             raise ReadTimeoutError(
--> 331                 self, url, "Read timed out. (read timeout=%s)" % timeout_value
    332             )

ReadTimeoutError: HTTPSConnectionPool(host='indiankanoon.org', port=443): Read timed out. (read timeout=5)

During handling of the above exception, another exception occurred:

ReadTimeout                               Traceback (most recent call last)
<ipython-input-7-ef31c3cbb243> in <module>
      5     print('-----------------------')
      6 #     , headers = headers,timeout = 60headers = {'user-agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36'}
----> 7     x = requests.get("https://indiankanoon.org/browse/supremecourt/%s/"%(i), timeout = 5)
      8     soup = BeautifulSoup(x.content, 'html.parser')
      9     data = [x["href"] for x in soup.find_all('a',{'href': re.compile(r'/search/')})]

~/.pyenv/versions/3.7.3/lib/python3.7/site-packages/requests/api.py in get(url, params, **kwargs)
     73 
     74     kwargs.setdefault('allow_redirects', True)
---> 75     return request('get', url, params=params, **kwargs)
     76 
     77 

~/.pyenv/versions/3.7.3/lib/python3.7/site-packages/requests/api.py in request(method, url, **kwargs)
     58     # cases, and look like a memory leak in others.
     59     with sessions.Session() as session:
---> 60         return session.request(method=method, url=url, **kwargs)
     61 
     62 

~/.pyenv/versions/3.7.3/lib/python3.7/site-packages/requests/sessions.py in request(self, method, url, params, data, headers, cookies, files, auth, timeout, allow_redirects, proxies, hooks, stream, verify, cert, json)
    531         }
    532         send_kwargs.update(settings)
--> 533         resp = self.send(prep, **send_kwargs)
    534 
    535         return resp

~/.pyenv/versions/3.7.3/lib/python3.7/site-packages/requests/sessions.py in send(self, request, **kwargs)
    644 
    645         # Send the request
--> 646         r = adapter.send(request, **kwargs)
    647 
    648         # Total elapsed time of the request (approximately)

~/.pyenv/versions/3.7.3/lib/python3.7/site-packages/requests/adapters.py in send(self, request, stream, timeout, verify, cert, proxies)
    527                 raise SSLError(e, request=request)
    528             elif isinstance(e, ReadTimeoutError):
--> 529                 raise ReadTimeout(e, request=request)
    530             else:
    531                 raise

ReadTimeout: HTTPSConnectionPool(host='indiankanoon.org', port=443): Read timed out. (read timeout=5)

我们可以看到它正确返回了一些结果（在这种情况下为3个），但随后返回错误。我曾尝试将timeout增加到60，但这没有任何效果。谁能告诉我如何解决这个问题？

Answer 1

或者您也可以使用Try..exception块。如果出现超时错误，它将为此打印并移至下一年并继续循环。为避免证书问题，我使用了verify=False

这是代码。

import requests
from bs4 import BeautifulSoup
all_data=[]
for i in range(1950,2020):
  print(i)
  try:
    x = requests.get("https://indiankanoon.org/browse/supremecourt/%s/"%(i), timeout =10 ,verify=False)
    soup = BeautifulSoup(x.content, 'html.parser')
    data = [x["href"] for x in soup.find_all('a',{'href': re.compile(r'/search/')})]
    print(data)
    all_data.append(data)

  except requests.exceptions.Timeout:
      print("Timeout occurred")

print(all_data)

在控制台上输出。

1950
C:\Python35\lib\site-packages\urllib3\connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecureRequestWarning)
['/search/?formInput=doctypes:supremecourt fromdate:1-1-1950 todate: 31-1-1950', '/search/?formInput=doctypes:supremecourt fromdate:1-2-1950 todate: 28-2-1950', '/search/?formInput=doctypes:supremecourt fromdate:1-3-1950 todate: 31-3-1950', '/search/?formInput=doctypes:supremecourt fromdate:1-4-1950 todate: 30-4-1950', '/search/?formInput=doctypes:supremecourt fromdate:1-5-1950 todate: 31-5-1950', '/search/?formInput=doctypes:supremecourt fromdate:1-6-1950 todate: 30-6-1950', '/search/?formInput=doctypes:supremecourt fromdate:1-7-1950 todate: 31-7-1950', '/search/?formInput=doctypes:supremecourt fromdate:1-8-1950 todate: 31-8-1950', '/search/?formInput=doctypes:supremecourt fromdate:1-9-1950 todate: 30-9-1950', '/search/?formInput=doctypes:supremecourt fromdate:1-10-1950 todate: 31-10-1950', '/search/?formInput=doctypes:supremecourt fromdate:1-11-1950 todate: 30-11-1950', '/search/?formInput=doctypes:supremecourt fromdate:1-12-1950 todate: 31-12-1950']
1951
C:\Python35\lib\site-packages\urllib3\connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecureRequestWarning)
['/search/?formInput=doctypes:supremecourt fromdate:1-1-1951 todate: 31-1-1951', '/search/?formInput=doctypes:supremecourt fromdate:1-2-1951 todate: 28-2-1951', '/search/?formInput=doctypes:supremecourt fromdate:1-3-1951 todate: 31-3-1951', '/search/?formInput=doctypes:supremecourt fromdate:1-4-1951 todate: 30-4-1951', '/search/?formInput=doctypes:supremecourt fromdate:1-5-1951 todate: 31-5-1951', '/search/?formInput=doctypes:supremecourt fromdate:1-6-1951 todate: 30-6-1951', '/search/?formInput=doctypes:supremecourt fromdate:1-7-1951 todate: 31-7-1951', '/search/?formInput=doctypes:supremecourt fromdate:1-8-1951 todate: 31-8-1951', '/search/?formInput=doctypes:supremecourt fromdate:1-9-1951 todate: 30-9-1951', '/search/?formInput=doctypes:supremecourt fromdate:1-10-1951 todate: 31-10-1951', '/search/?formInput=doctypes:supremecourt fromdate:1-11-1951 todate: 30-11-1951', '/search/?formInput=doctypes:supremecourt fromdate:1-12-1951 todate: 31-12-1951']
1952
C:\Python35\lib\site-packages\urllib3\connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecureRequestWarning)
['/search/?formInput=doctypes:supremecourt fromdate:1-1-1952 todate: 31-1-1952', '/search/?formInput=doctypes:supremecourt fromdate:1-2-1952 todate: 29-2-1952', '/search/?formInput=doctypes:supremecourt fromdate:1-3-1952 todate: 31-3-1952', '/search/?formInput=doctypes:supremecourt fromdate:1-4-1952 todate: 30-4-1952', '/search/?formInput=doctypes:supremecourt fromdate:1-5-1952 todate: 31-5-1952', '/search/?formInput=doctypes:supremecourt fromdate:1-6-1952 todate: 30-6-1952', '/search/?formInput=doctypes:supremecourt fromdate:1-7-1952 todate: 31-7-1952', '/search/?formInput=doctypes:supremecourt fromdate:1-8-1952 todate: 31-8-1952', '/search/?formInput=doctypes:supremecourt fromdate:1-9-1952 todate: 30-9-1952', '/search/?formInput=doctypes:supremecourt fromdate:1-10-1952 todate: 31-10-1952', '/search/?formInput=doctypes:supremecourt fromdate:1-11-1952 todate: 30-11-1952', '/search/?formInput=doctypes:supremecourt fromdate:1-12-1952 todate: 31-12-1952']
1953
C:\Python35\lib\site-packages\urllib3\connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecureRequestWarning)
['/search/?formInput=doctypes:supremecourt fromdate:1-1-1953 todate: 31-1-1953', '/search/?formInput=doctypes:supremecourt fromdate:1-2-1953 todate: 28-2-1953', '/search/?formInput=doctypes:supremecourt fromdate:1-3-1953 todate: 31-3-1953', '/search/?formInput=doctypes:supremecourt fromdate:1-4-1953 todate: 30-4-1953', '/search/?formInput=doctypes:supremecourt fromdate:1-5-1953 todate: 31-5-1953', '/search/?formInput=doctypes:supremecourt fromdate:1-6-1953 todate: 30-6-1953', '/search/?formInput=doctypes:supremecourt fromdate:1-7-1953 todate: 31-7-1953', '/search/?formInput=doctypes:supremecourt fromdate:1-8-1953 todate: 31-8-1953', '/search/?formInput=doctypes:supremecourt fromdate:1-9-1953 todate: 30-9-1953', '/search/?formInput=doctypes:supremecourt fromdate:1-10-1953 todate: 31-10-1953', '/search/?formInput=doctypes:supremecourt fromdate:1-11-1953 todate: 30-11-1953', '/search/?formInput=doctypes:supremecourt fromdate:1-12-1953 todate: 31-12-1953']
1954
C:\Python35\lib\site-packages\urllib3\connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecureRequestWarning)
['/search/?formInput=doctypes:supremecourt fromdate:1-1-1954 todate: 31-1-1954', '/search/?formInput=doctypes:supremecourt fromdate:1-2-1954 todate: 28-2-1954', '/search/?formInput=doctypes:supremecourt fromdate:1-3-1954 todate: 31-3-1954', '/search/?formInput=doctypes:supremecourt fromdate:1-4-1954 todate: 30-4-1954', '/search/?formInput=doctypes:supremecourt fromdate:1-5-1954 todate: 31-5-1954', '/search/?formInput=doctypes:supremecourt fromdate:1-6-1954 todate: 30-6-1954', '/search/?formInput=doctypes:supremecourt fromdate:1-7-1954 todate: 31-7-1954', '/search/?formInput=doctypes:supremecourt fromdate:1-8-1954 todate: 31-8-1954', '/search/?formInput=doctypes:supremecourt fromdate:1-9-1954 todate: 30-9-1954', '/search/?formInput=doctypes:supremecourt fromdate:1-10-1954 todate: 31-10-1954', '/search/?formInput=doctypes:supremecourt fromdate:1-11-1954 todate: 30-11-1954', '/search/?formInput=doctypes:supremecourt fromdate:1-12-1954 todate: 31-12-1954']
1955
C:\Python35\lib\site-packages\urllib3\connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecureRequestWarning)
['/search/?formInput=doctypes:supremecourt fromdate:1-1-1955 todate: 31-1-1955', '/search/?formInput=doctypes:supremecourt fromdate:1-2-1955 todate: 28-2-1955', '/search/?formInput=doctypes:supremecourt fromdate:1-3-1955 todate: 31-3-1955', '/search/?formInput=doctypes:supremecourt fromdate:1-4-1955 todate: 30-4-1955', '/search/?formInput=doctypes:supremecourt fromdate:1-5-1955 todate: 31-5-1955', '/search/?formInput=doctypes:supremecourt fromdate:1-6-1955 todate: 30-6-1955', '/search/?formInput=doctypes:supremecourt fromdate:1-7-1955 todate: 31-7-1955', '/search/?formInput=doctypes:supremecourt fromdate:1-8-1955 todate: 31-8-1955', '/search/?formInput=doctypes:supremecourt fromdate:1-9-1955 todate: 30-9-1955', '/search/?formInput=doctypes:supremecourt fromdate:1-10-1955 todate: 31-10-1955', '/search/?formInput=doctypes:supremecourt fromdate:1-11-1955 todate: 30-11-1955', '/search/?formInput=doctypes:supremecourt fromdate:1-12-1955 todate: 31-12-1955']
1956
C:\Python35\lib\site-packages\urllib3\connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecureRequestWarning)
['/search/?formInput=doctypes:supremecourt fromdate:1-1-1956 todate: 31-1-1956', '/search/?formInput=doctypes:supremecourt fromdate:1-2-1956 todate: 29-2-1956', '/search/?formInput=doctypes:supremecourt fromdate:1-3-1956 todate: 31-3-1956', '/search/?formInput=doctypes:supremecourt fromdate:1-4-1956 todate: 30-4-1956', '/search/?formInput=doctypes:supremecourt fromdate:1-5-1956 todate: 31-5-1956', '/search/?formInput=doctypes:supremecourt fromdate:1-6-1956 todate: 30-6-1956', '/search/?formInput=doctypes:supremecourt fromdate:1-7-1956 todate: 31-7-1956', '/search/?formInput=doctypes:supremecourt fromdate:1-8-1956 todate: 31-8-1956', '/search/?formInput=doctypes:supremecourt fromdate:1-9-1956 todate: 30-9-1956', '/search/?formInput=doctypes:supremecourt fromdate:1-10-1956 todate: 31-10-1956', '/search/?formInput=doctypes:supremecourt fromdate:1-11-1956 todate: 30-11-1956', '/search/?formInput=doctypes:supremecourt fromdate:1-12-1956 todate: 31-12-1956']
1957
C:\Python35\lib\site-packages\urllib3\connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecureRequestWarning)
['/search/?formInput=doctypes:supremecourt fromdate:1-1-1957 todate: 31-1-1957', '/search/?formInput=doctypes:supremecourt fromdate:1-2-1957 todate: 28-2-1957', '/search/?formInput=doctypes:supremecourt fromdate:1-3-1957 todate: 31-3-1957', '/search/?formInput=doctypes:supremecourt fromdate:1-4-1957 todate: 30-4-1957', '/search/?formInput=doctypes:supremecourt fromdate:1-5-1957 todate: 31-5-1957', '/search/?formInput=doctypes:supremecourt fromdate:1-6-1957 todate: 30-6-1957', '/search/?formInput=doctypes:supremecourt fromdate:1-7-1957 todate: 31-7-1957', '/search/?formInput=doctypes:supremecourt fromdate:1-8-1957 todate: 31-8-1957', '/search/?formInput=doctypes:supremecourt fromdate:1-9-1957 todate: 30-9-1957', '/search/?formInput=doctypes:supremecourt fromdate:1-10-1957 todate: 31-10-1957', '/search/?formInput=doctypes:supremecourt fromdate:1-11-1957 todate: 30-11-1957', '/search/?formInput=doctypes:supremecourt fromdate:1-12-1957 todate: 31-12-1957']
1958
C:\Python35\lib\site-packages\urllib3\connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecureRequestWarning)
['/search/?formInput=doctypes:supremecourt fromdate:1-1-1958 todate: 31-1-1958', '/search/?formInput=doctypes:supremecourt fromdate:1-2-1958 todate: 28-2-1958', '/search/?formInput=doctypes:supremecourt fromdate:1-3-1958 todate: 31-3-1958', '/search/?formInput=doctypes:supremecourt fromdate:1-4-1958 todate: 30-4-1958', '/search/?formInput=doctypes:supremecourt fromdate:1-5-1958 todate: 31-5-1958', '/search/?formInput=doctypes:supremecourt fromdate:1-6-1958 todate: 30-6-1958', '/search/?formInput=doctypes:supremecourt fromdate:1-7-1958 todate: 31-7-1958', '/search/?formInput=doctypes:supremecourt fromdate:1-8-1958 todate: 31-8-1958', '/search/?formInput=doctypes:supremecourt fromdate:1-9-1958 todate: 30-9-1958', '/search/?formInput=doctypes:supremecourt fromdate:1-10-1958 todate: 31-10-1958', '/search/?formInput=doctypes:supremecourt fromdate:1-11-1958 todate: 30-11-1958', '/search/?formInput=doctypes:supremecourt fromdate:1-12-1958 todate: 31-12-1958']
1959
C:\Python35\lib\site-packages\urllib3\connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecureRequestWarning)
['/search/?formInput=doctypes:supremecourt fromdate:1-1-1959 todate: 31-1-1959', '/search/?formInput=doctypes:supremecourt fromdate:1-2-1959 todate: 28-2-1959', '/search/?formInput=doctypes:supremecourt fromdate:1-3-1959 todate: 31-3-1959', '/search/?formInput=doctypes:supremecourt fromdate:1-4-1959 todate: 30-4-1959', '/search/?formInput=doctypes:supremecourt fromdate:1-5-1959 todate: 31-5-1959', '/search/?formInput=doctypes:supremecourt fromdate:1-6-1959 todate: 30-6-1959', '/search/?formInput=doctypes:supremecourt fromdate:1-7-1959 todate: 31-7-1959', '/search/?formInput=doctypes:supremecourt fromdate:1-8-1959 todate: 31-8-1959', '/search/?formInput=doctypes:supremecourt fromdate:1-9-1959 todate: 30-9-1959', '/search/?formInput=doctypes:supremecourt fromdate:1-10-1959 todate: 31-10-1959', '/search/?formInput=doctypes:supremecourt fromdate:1-11-1959 todate: 30-11-1959', '/search/?formInput=doctypes:supremecourt fromdate:1-12-1959 todate: 31-12-1959']
1960
C:\Python35\lib\site-packages\urllib3\connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecureRequestWarning)
['/search/?formInput=doctypes:supremecourt fromdate:1-1-1960 todate: 31-1-1960', '/search/?formInput=doctypes:supremecourt fromdate:1-2-1960 todate: 29-2-1960', '/search/?formInput=doctypes:supremecourt fromdate:1-3-1960 todate: 31-3-1960', '/search/?formInput=doctypes:supremecourt fromdate:1-4-1960 todate: 30-4-1960', '/search/?formInput=doctypes:supremecourt fromdate:1-5-1960 todate: 31-5-1960', '/search/?formInput=doctypes:supremecourt fromdate:1-6-1960 todate: 30-6-1960', '/search/?formInput=doctypes:supremecourt fromdate:1-7-1960 todate: 31-7-1960', '/search/?formInput=doctypes:supremecourt fromdate:1-8-1960 todate: 31-8-1960', '/search/?formInput=doctypes:supremecourt fromdate:1-9-1960 todate: 30-9-1960', '/search/?formInput=doctypes:supremecourt fromdate:1-10-1960 todate: 31-10-1960', '/search/?formInput=doctypes:supremecourt fromdate:1-11-1960 todate: 30-11-1960', '/search/?formInput=doctypes:supremecourt fromdate:1-12-1960 todate: 31-12-1960']
1961
C:\Python35\lib\site-packages\urllib3\connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecureRequestWarning)
['/search/?formInput=doctypes:supremecourt fromdate:1-1-1961 todate: 31-1-1961', '/search/?formInput=doctypes:supremecourt fromdate:1-2-1961 todate: 28-2-1961', '/search/?formInput=doctypes:supremecourt fromdate:1-3-1961 todate: 31-3-1961', '/search/?formInput=doctypes:supremecourt fromdate:1-4-1961 todate: 30-4-1961', '/search/?formInput=doctypes:supremecourt fromdate:1-5-1961 todate: 31-5-1961', '/search/?formInput=doctypes:supremecourt fromdate:1-6-1961 todate: 30-6-1961', '/search/?formInput=doctypes:supremecourt fromdate:1-7-1961 todate: 31-7-1961', '/search/?formInput=doctypes:supremecourt fromdate:1-8-1961 todate: 31-8-1961', '/search/?formInput=doctypes:supremecourt fromdate:1-9-1961 todate: 30-9-1961', '/search/?formInput=doctypes:supremecourt fromdate:1-10-1961 todate: 31-10-1961', '/search/?formInput=doctypes:supremecourt fromdate:1-11-1961 todate: 30-11-1961', '/search/?formInput=doctypes:supremecourt fromdate:1-12-1961 todate: 31-12-1961']
1962
C:\Python35\lib\site-packages\urllib3\connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecureRequestWarning)
['/search/?formInput=doctypes:supremecourt fromdate:1-1-1962 todate: 31-1-1962', '/search/?formInput=doctypes:supremecourt fromdate:1-2-1962 todate: 28-2-1962', '/search/?formInput=doctypes:supremecourt fromdate:1-3-1962 todate: 31-3-1962', '/search/?formInput=doctypes:supremecourt fromdate:1-4-1962 todate: 30-4-1962', '/search/?formInput=doctypes:supremecourt fromdate:1-5-1962 todate: 31-5-1962', '/search/?formInput=doctypes:supremecourt fromdate:1-6-1962 todate: 30-6-1962', '/search/?formInput=doctypes:supremecourt fromdate:1-7-1962 todate: 31-7-1962', '/search/?formInput=doctypes:supremecourt fromdate:1-8-1962 todate: 31-8-1962', '/search/?formInput=doctypes:supremecourt fromdate:1-9-1962 todate: 30-9-1962', '/search/?formInput=doctypes:supremecourt fromdate:1-10-1962 todate: 31-10-1962', '/search/?formInput=doctypes:supremecourt fromdate:1-11-1962 todate: 30-11-1962', '/search/?formInput=doctypes:supremecourt fromdate:1-12-1962 todate: 31-12-1962']
1963
C:\Python35\lib\site-packages\urllib3\connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecureRequestWarning)
['/search/?formInput=doctypes:supremecourt fromdate:1-1-1963 todate: 31-1-1963', '/search/?formInput=doctypes:supremecourt fromdate:1-2-1963 todate: 28-2-1963', '/search/?formInput=doctypes:supremecourt fromdate:1-3-1963 todate: 31-3-1963', '/search/?formInput=doctypes:supremecourt fromdate:1-4-1963 todate: 30-4-1963', '/search/?formInput=doctypes:supremecourt fromdate:1-5-1963 todate: 31-5-1963', '/search/?formInput=doctypes:supremecourt fromdate:1-6-1963 todate: 30-6-1963', '/search/?formInput=doctypes:supremecourt fromdate:1-7-1963 todate: 31-7-1963', '/search/?formInput=doctypes:supremecourt fromdate:1-8-1963 todate: 31-8-1963', '/search/?formInput=doctypes:supremecourt fromdate:1-9-1963 todate: 30-9-1963', '/search/?formInput=doctypes:supremecourt fromdate:1-10-1963 todate: 31-10-1963', '/search/?formInput=doctypes:supremecourt fromdate:1-11-1963 todate: 30-11-1963', '/search/?formInput=doctypes:supremecourt fromdate:1-12-1963 todate: 31-12-1963']
1964
C:\Python35\lib\site-packages\urllib3\connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecureRequestWarning)
['/search/?formInput=doctypes:supremecourt fromdate:1-1-1964 todate: 31-1-1964', '/search/?formInput=doctypes:supremecourt fromdate:1-2-1964 todate: 29-2-1964', '/search/?formInput=doctypes:supremecourt fromdate:1-3-1964 todate: 31-3-1964', '/search/?formInput=doctypes:supremecourt fromdate:1-4-1964 todate: 30-4-1964', '/search/?formInput=doctypes:supremecourt fromdate:1-5-1964 todate: 31-5-1964', '/search/?formInput=doctypes:supremecourt fromdate:1-6-1964 todate: 30-6-1964', '/search/?formInput=doctypes:supremecourt fromdate:1-7-1964 todate: 31-7-1964', '/search/?formInput=doctypes:supremecourt fromdate:1-8-1964 todate: 31-8-1964', '/search/?formInput=doctypes:supremecourt fromdate:1-9-1964 todate: 30-9-1964', '/search/?formInput=doctypes:supremecourt fromdate:1-10-1964 todate: 31-10-1964', '/search/?formInput=doctypes:supremecourt fromdate:1-11-1964 todate: 30-11-1964', '/search/?formInput=doctypes:supremecourt fromdate:1-12-1964 todate: 31-12-1964']
1965
C:\Python35\lib\site-packages\urllib3\connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecureRequestWarning)
['/search/?formInput=doctypes:supremecourt fromdate:1-1-1965 todate: 31-1-1965', '/search/?formInput=doctypes:supremecourt fromdate:1-2-1965 todate: 28-2-1965', '/search/?formInput=doctypes:supremecourt fromdate:1-3-1965 todate: 31-3-1965', '/search/?formInput=doctypes:supremecourt fromdate:1-4-1965 todate: 30-4-1965', '/search/?formInput=doctypes:supremecourt fromdate:1-5-1965 todate: 31-5-1965', '/search/?formInput=doctypes:supremecourt fromdate:1-6-1965 todate: 30-6-1965', '/search/?formInput=doctypes:supremecourt fromdate:1-7-1965 todate: 31-7-1965', '/search/?formInput=doctypes:supremecourt fromdate:1-8-1965 todate: 31-8-1965', '/search/?formInput=doctypes:supremecourt fromdate:1-9-1965 todate: 30-9-1965', '/search/?formInput=doctypes:supremecourt fromdate:1-10-1965 todate: 31-10-1965', '/search/?formInput=doctypes:supremecourt fromdate:1-11-1965 todate: 30-11-1965', '/search/?formInput=doctypes:supremecourt fromdate:1-12-1965 todate: 31-12-1965']
1966
C:\Python35\lib\site-packages\urllib3\connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecureRequestWarning)
['/search/?formInput=doctypes:supremecourt fromdate:1-1-1966 todate: 31-1-1966', '/search/?formInput=doctypes:supremecourt fromdate:1-2-1966 todate: 28-2-1966', '/search/?formInput=doctypes:supremecourt fromdate:1-3-1966 todate: 31-3-1966', '/search/?formInput=doctypes:supremecourt fromdate:1-4-1966 todate: 30-4-1966', '/search/?formInput=doctypes:supremecourt fromdate:1-5-1966 todate: 31-5-1966', '/search/?formInput=doctypes:supremecourt fromdate:1-6-1966 todate: 30-6-1966', '/search/?formInput=doctypes:supremecourt fromdate:1-7-1966 todate: 31-7-1966', '/search/?formInput=doctypes:supremecourt fromdate:1-8-1966 todate: 31-8-1966', '/search/?formInput=doctypes:supremecourt fromdate:1-9-1966 todate: 30-9-1966', '/search/?formInput=doctypes:supremecourt fromdate:1-10-1966 todate: 31-10-1966', '/search/?formInput=doctypes:supremecourt fromdate:1-11-1966 todate: 30-11-1966', '/search/?formInput=doctypes:supremecourt fromdate:1-12-1966 todate: 31-12-1966']
1967
C:\Python35\lib\site-packages\urllib3\connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecureRequestWarning)
['/search/?formInput=doctypes:supremecourt fromdate:1-1-1967 todate: 31-1-1967', '/search/?formInput=doctypes:supremecourt fromdate:1-2-1967 todate: 28-2-1967', '/search/?formInput=doctypes:supremecourt fromdate:1-3-1967 todate: 31-3-1967', '/search/?formInput=doctypes:supremecourt fromdate:1-4-1967 todate: 30-4-1967', '/search/?formInput=doctypes:supremecourt fromdate:1-5-1967 todate: 31-5-1967', '/search/?formInput=doctypes:supremecourt fromdate:1-6-1967 todate: 30-6-1967', '/search/?formInput=doctypes:supremecourt fromdate:1-7-1967 todate: 31-7-1967', '/search/?formInput=doctypes:supremecourt fromdate:1-8-1967 todate: 31-8-1967', '/search/?formInput=doctypes:supremecourt fromdate:1-9-1967 todate: 30-9-1967', '/search/?formInput=doctypes:supremecourt fromdate:1-10-1967 todate: 31-10-1967', '/search/?formInput=doctypes:supremecourt fromdate:1-11-1967 todate: 30-11-1967', '/search/?formInput=doctypes:supremecourt fromdate:1-12-1967 todate: 31-12-1967']
1968
C:\Python35\lib\site-packages\urllib3\connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecureRequestWarning)
['/search/?formInput=doctypes:supremecourt fromdate:1-1-1968 todate: 31-1-1968', '/search/?formInput=doctypes:supremecourt fromdate:1-2-1968 todate: 29-2-1968', '/search/?formInput=doctypes:supremecourt fromdate:1-3-1968 todate: 31-3-1968', '/search/?formInput=doctypes:supremecourt fromdate:1-4-1968 todate: 30-4-1968', '/search/?formInput=doctypes:supremecourt fromdate:1-5-1968 todate: 31-5-1968', '/search/?formInput=doctypes:supremecourt fromdate:1-6-1968 todate: 30-6-1968', '/search/?formInput=doctypes:supremecourt fromdate:1-7-1968 todate: 31-7-1968', '/search/?formInput=doctypes:supremecourt fromdate:1-8-1968 todate: 31-8-1968', '/search/?formInput=doctypes:supremecourt fromdate:1-9-1968 todate: 30-9-1968', '/search/?formInput=doctypes:supremecourt fromdate:1-10-1968 todate: 31-10-1968', '/search/?formInput=doctypes:supremecourt fromdate:1-11-1968 todate: 30-11-1968', '/search/?formInput=doctypes:supremecourt fromdate:1-12-1968 todate: 31-12-1968']
1969
C:\Python35\lib\site-packages\urllib3\connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecureRequestWarning)
['/search/?formInput=doctypes:supremecourt fromdate:1-1-1969 todate: 31-1-1969', '/search/?formInput=doctypes:supremecourt fromdate:1-2-1969 todate: 28-2-1969', '/search/?formInput=doctypes:supremecourt fromdate:1-3-1969 todate: 31-3-1969', '/search/?formInput=doctypes:supremecourt fromdate:1-4-1969 todate: 30-4-1969', '/search/?formInput=doctypes:supremecourt fromdate:1-5-1969 todate: 31-5-1969', '/search/?formInput=doctypes:supremecourt fromdate:1-6-1969 todate: 30-6-1969', '/search/?formInput=doctypes:supremecourt fromdate:1-7-1969 todate: 31-7-1969', '/search/?formInput=doctypes:supremecourt fromdate:1-8-1969 todate: 31-8-1969', '/search/?formInput=doctypes:supremecourt fromdate:1-9-1969 todate: 30-9-1969', '/search/?formInput=doctypes:supremecourt fromdate:1-10-1969 todate: 31-10-1969', '/search/?formInput=doctypes:supremecourt fromdate:1-11-1969 todate: 30-11-1969', '/search/?formInput=doctypes:supremecourt fromdate:1-12-1969 todate: 31-12-1969']
1970
C:\Python35\lib\site-packages\urllib3\connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecureRequestWarning)
['/search/?formInput=doctypes:supremecourt fromdate:1-1-1970 todate: 31-1-1970', '/search/?formInput=doctypes:supremecourt fromdate:1-2-1970 todate: 28-2-1970', '/search/?formInput=doctypes:supremecourt fromdate:1-3-1970 todate: 31-3-1970', '/search/?formInput=doctypes:supremecourt fromdate:1-4-1970 todate: 30-4-1970', '/search/?formInput=doctypes:supremecourt fromdate:1-5-1970 todate: 31-5-1970', '/search/?formInput=doctypes:supremecourt fromdate:1-6-1970 todate: 30-6-1970', '/search/?formInput=doctypes:supremecourt fromdate:1-7-1970 todate: 31-7-1970', '/search/?formInput=doctypes:supremecourt fromdate:1-8-1970 todate: 31-8-1970', '/search/?formInput=doctypes:supremecourt fromdate:1-9-1970 todate: 30-9-1970', '/search/?formInput=doctypes:supremecourt fromdate:1-10-1970 todate: 31-10-1970', '/search/?formInput=doctypes:supremecourt fromdate:1-11-1970 todate: 30-11-1970', '/search/?formInput=doctypes:supremecourt fromdate:1-12-1970 todate: 31-12-1970']
1971
C:\Python35\lib\site-packages\urllib3\connectionpool.py:851: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecureRequestWarning)
['/search/?formInput=doctypes:supremecourt fromdate:1-1-1971 todate: 31-1-1971', '/search/?formInput=doctypes:supremecourt fromdate:1-2-1971 todate: 28-2-1971', '/search/?formInput=doctypes:supremecourt fromdate:1-3-1971 todate: 31-3-1971', '/search/?formInput=doctypes:supremecourt fromdate:1-4-1971 todate: 30-4-1971', '/search/?formInput=doctypes:supremecourt fromdate:1-5-1971 todate: 31-5-1971', '/search/?formInput=doctypes:supremecourt fromdate:1-6-1971 todate: 30-6-1971', '/search/?formInput=doctypes:supremecourt fromdate:1-7-1971 todate: 31-7-1971', '/search/?formInput=doctypes:supremecourt fromdate:1-8-1971 todate: 31-8-1971', '/search/?formInput=doctypes:supremecourt fromdate:1-9-1971 todate: 30-9-1971', '/search/?formInput=doctypes:supremecourt fromdate:1-10-1971 todate: 31-10-1971', '/search/?formInput=doctypes:supremecourt fromdate:1-11-1971 todate: 30-11-1971', '/search/?formInput=doctypes:supremecourt fromdate:1-12-1971 todate: 31-12-1971']

以此类推。...

Answer 2

在请求之间尝试time.sleep(2)，似乎确实可以解决

在BeautifulSoup中使用请求时，如何解决超时错误？

2 个答案: