Question

我正在使用线程进行远程API调用，不使用连接，这样程序就可以进行下一次API调用而无需等待最后一次完成。

像这样：

def run_single_thread_no_join(function, args):
thread = Thread(target=function, args=(args,))
thread.start()
return

问题是我需要知道所有API调用何时完成。所以我转向使用提示和放大器的代码。加入。

线程似乎现在连续运行。

我似乎无法弄清楚如何让连接工作以便线程并行执行。

我做错了什么？

def run_que_block(methods_list, num_worker_threads=10):
'''
Runs methods on threads.  Stores method returns in a list.  Then outputs that list
after all methods in the list have been completed.

:param methods_list: example ((method name, args), (method_2, args), (method_3, args)
:param num_worker_threads: The number of threads to use in the block.
:return: The full list of returns from each method.
'''

method_returns = []

# log = StandardLogger(logger_name='run_que_block')

# lock to serialize console output
lock = threading.Lock()

def _output(item):
    # Make sure the whole print completes or threads can mix up output in one line.
    with lock:
        if item:
            print(item)
        msg = threading.current_thread().name, item
        # log.log_debug(msg)

    return

# The worker thread pulls an item from the queue and processes it
def _worker():

    while True:
        item = q.get()
        if item is None:
            break

        method_returns.append(item)
        _output(item)

        q.task_done()

# Create the queue and thread pool.
q = Queue()

threads = []
# starts worker threads.
for i in range(num_worker_threads):
    t = threading.Thread(target=_worker)
    t.daemon = True  # thread dies when main thread (only non-daemon thread) exits.
    t.start()
    threads.append(t)

for method in methods_list:
    q.put(method[0](*method[1]))

# block until all tasks are done
q.join()

# stop workers
for i in range(num_worker_threads):
    q.put(None)
for t in threads:
    t.join()

return method_returns

Answer 1

你正在主线程中完成所有工作：

for method in methods_list:
    q.put(method[0](*method[1]))

假设methods_list中的每个条目都是一个可调用的和一系列参数，你在主线程中完成了所有工作，然后将每个函数调用的结果放入队列中，这是不允许的除print之外的任何并行化（通常不足以证明线程/队列开销的合理性）。

据推测，您希望线程为每个函数执行工作，因此将该循环更改为：

for method in methods_list:
    q.put(method)  # Don't call it, queue it to be called in worker

并更改_worker函数，以便它调用在线程中执行工作的函数：

def _worker():
    while True:
        method, args = q.get()  # Extract and unpack callable and arguments
        item = method(*args)    # Call callable with provided args and store result
        if item is None:
            break

        method_returns.append(item)
        _output(item)

        q.task_done()

线程提示是串行运行，而不是并行运行？

1 个答案: