Question

我想知道如何限制这样的东西，一次只使用10个线程

with open("data.txt") as f:
    for line in f:
        lines = line.rstrip("\n\r")
        t1 = Thread(target=Checker, args=("company"))
        t1.start()

Answer 1

使用Python的ThreadPoolExecutor并将max_workers参数设置为10。

像这样：`

pool = ThreadPoolExecutor(max_workers=10)
with open("data.txt") as f:
    for line in f:
        lines = line.rstrip("\n\r")
        pool.submit(Checker,"company")

pool.shutdown(wait=True)

pool将根据需要自动分配线程，将最大分配数限制为10. pool.submit()中的第一个参数是函数名，参数只是作为逗号分隔值传递。 / p>

pool.shutdown(wait=True)等待所有线程完成执行。

Answer 2

使用ThreadPoolExecutor并告诉它你想要10个线程。

def your_function_processing_one_line(line):
    pass  # your computations

with concurrent.futures.ThreadPoolExecutor(10) as executor:
    result = executor.map(your_function_processing_one_line, [line for line in f])

...您将获得result中的所有结果。

Answer 3

（适用于Python 2.6+和Python 3）

使用threadPool模块中的multiprocessing：

from multiprocessing.pool import ThreadPool

唯一的问题是没有详细记录......

Answer 4

我编写了这个嵌套循环来将线程限制为一个变量。此代码依赖于要处理的预设命令数组。我从线程启动的其他答案中借用了一些元素。

import os, sys, datetime, logging, thread, threading, time
from random import randint

# set number of threads
threadcount = 20

# alltests is an array of test data

numbertests = len(alltests)
testcounter = numbertests

# run tests
for test in alltests:
    # launch worker thread
    def worker():
        """thread worker function"""
        os.system(command)
        return
    threads = []
    t = threading.Thread(target=worker)
    threads.append(t)
    t.start()
    testcounter -= 1
    # cap the threads if over limit
    while threading.active_count() >= threadcount:
        threads = threading.active_count()
        string = "Excessive threads, pausing 5 secs - " + str(threads) 
        print (string)
        logging.info(string)
        time.sleep(5)

# monitor for threads winding down
while threading.active_count() != 1:
    threads = threading.active_count()
    string = "Active threads running - " + str(threads) 
    print (string)
    logging.info(string)
    time.sleep(5)

python如何设置线程限制？

4 个答案: