Question

我有一个代码可以从网站检索搜索结果。

ExtractDetails功能是用于搜索网站的大型搜索功能。

LangList = ['91','32','69','A9','48','62','01','75','34','73','B1',
'68','61','20','03','35','36','86','87','AS','63','04','AO','65','57','05','AE','06','AC','07','08','A4','37','72','70',
'38','09','95','10','A1','B0','AL','A8','39','AT','94','11','84','12','59','AG','AB','13','40','14','AW','67','98','AJ',
'41','15','74','16','33','17','AU','AV','A0','42','92','AI','18','AD','A2','43','AM','44','45','A3','85','58','83','71',
'21','22','A6','90','AR','99','46','78','23','24','76','AP','25','26','AN','47','60','97','49','56','27','93','AZ','02',
'28','77','AA','29','50','A5','64','51','52','AQ','AY','A7','80','30','53','54','31','96','79','89','55','66','82','81']
Alphabet = 'ABCDEFGHIJKLMNOPQRSTUVWXYZ'
SearchCriteria=[Alphabet] * len(LangList)
for Lang , alphabets in LostConn.items():
    for item in alphabets:
        t = Thread(target=ExtractDetails,args=(Lang,item + '%',))
        t.start()
        th.append(t)
for thr in th:
    thr.join()

脚本的问题是，如果我运行此代码，笔记本电脑将挂起，因为它将生成3406个线程。我的问题是如何确定特定笔记本电脑上的最佳线程数？

Answer 1

可以同时运行的线程数取决于您的计算机和cpu拥有的内核数。大多数机器每个内核可以运行2个线程。找出适用于您的计算机的简单方法是在Python的multiprocessing模块中：

import multiprocessing
multiprocessing.cpu_count()

此函数将显示可以同时运行的最大线程数。希望这可以帮助！

Python-网络抓取-多线程-线程限制了吗？

1 个答案: