所以我在我的代码中遇到了这个问题:
worker_threads = 950
threads = []
for thr in range(worker_threads):
thread = threading.Thread(target=worker_function, args=(in_queue, out_queue, status_control))
thread.daemon = True
threads.append(thread)
thread.start()
for thread in threads:
thread.join()
线程正在使用请求模块并执行简单的Web爬网。出于某种原因,当线程数超过650时,我得到了消息:
Traceback (most recent call last):
File "C:/Users/bb/Desktop/wp/Новая папка/upload.py", line 132, in <module>
thread.start()
File "C:\Users\bb\AppData\Local\Programs\Python\Python35-32\lib\threading.py", line 844, in start
_start_new_thread(self._bootstrap, ())
RuntimeError: can't start new thread
我正在使用带有4GB内存的VDS windows 2008 x64服务器。我该如何解决这个问题?我没有找到任何有用的解决方案来解决这类问题。谢谢。
UPD。 这是工人职能的例子:
def worker_function(in_queue, out_queue, status_control):
while not in_queue.empty():
try:
url = in_queue.get_nowait()
except queue.Empty:
break
try:
print('Checking '+url)
except Exception as e:
print(str(e))
现在python崩溃了。