Question

知道如何用Python 2.7.8来解决这类问题吗？如图所示，使用了一些站点包，例如gevent，但AFAIK的问题是限制error: IOError: cannot watch more than 1024 sockets

  File "C:\Python27\lib\httplib.py", line 806, in send
    self.connect()
  File "C:\Python27\lib\site-packages\requests-2.3.0-py2.7.egg\requests\packages\urllib3\connection.py", line 106, in connect
    conn = self._new_conn()
  File "C:\Python27\lib\site-packages\requests-2.3.0-py2.7.egg\requests\packages\urllib3\connection.py", line 90, in _new_conn
    (self.host, self.port), self.timeout, *extra_args)
  File "C:\Python27\lib\site-packages\gevent\socket.py", line 574, in create_connection
    sock = socket(af, socktype, proto)
  File "C:\Python27\lib\site-packages\gevent\socket.py", line 242, in __init__
    self._read_event = io(fileno, 1)
  File "core.pyx", line 466, in gevent.core.loop.io (gevent/gevent.core.c:8305)
  File "core.pyx", line 804, in gevent.core.io.__init__ (gevent/gevent.core.c:14393)

IOError: cannot watch more than 1024 sockets

有什么想法吗？

Answer 1

您可能只需要增加系统上的打开文件描述符限制。有关更多信息，请参阅此处：increase ulimit for # of file descriptors - Linux上的默认值通常为1024。

Answer 2

如果您使用grequests发出请求，则可以使用会话

sess = requests.Session()
reqs = [grequests.get(u, /* params */, session=sess) for u in urls]
results = grequests.map(reqs)

Python套接字限制（1024）错误

2 个答案: