任何python的内置模块都使用线程吗?

时间:2014-07-18 17:13:02

标签: python multithreading standard-library

导入/使用任何python的内置库会产生线程而不会被明确询问吗?

2 个答案:

答案 0 :(得分:7)

multiprocessing模块和subprocess模块都在内部生成threading.Thead个对象,以帮助处理它们产生的进程的I / O.

具体而言,只要您实例化multiprocessing.Pool spawns three threads

class Pool(object):
    ''' 
    Class which supports an async version of the `apply()` builtin
    '''
    Process = Process

    def __init__(self, processes=None, initializer=None, initargs=(),
                 maxtasksperchild=None):

       ... # Stuff we don't care about
       self._worker_handler = threading.Thread(
            target=Pool._handle_workers,
            args=(self, )
            )   
        self._worker_handler.daemon = True
        self._worker_handler._state = RUN 
        self._worker_handler.start()


        self._task_handler = threading.Thread(
            target=Pool._handle_tasks,
            args=(self._taskqueue, self._quick_put, self._outqueue, self._pool)
            )   
        self._task_handler.daemon = True
        self._task_handler._state = RUN 
        self._task_handler.start()

        self._result_handler = threading.Thread(
            target=Pool._handle_results,
            args=(self._outqueue, self._quick_get, self._cache)
            )   
        self._result_handler.daemon = True
        self._result_handler._state = RUN 
        self._result_handler.start()

subprocess生成线程when you call popen_object.communicate,从正在运行的子进程中读取stdout / stderr。

def _communicate(self, input):
    stdout = None # Return
    stderr = None # Return

    if self.stdout:
        stdout = [] 
        stdout_thread = threading.Thread(target=self._readerthread,
                                         args=(self.stdout, stdout))
        stdout_thread.setDaemon(True)
        stdout_thread.start()
    if self.stderr:
        stderr = [] 
        stderr_thread = threading.Thread(target=self._readerthread,
                                         args=(self.stderr, stderr))
        stderr_thread.setDaemon(True)
        stderr_thread.start()

修改

Mark Dickinson points out concurrent.futures.ProcessPoolExecutor也产生了一个线程,原因与multiprocessing.Pool类似(使用生成进程处理I / O):

def _start_queue_management_thread(self):
    # When the executor gets lost, the weakref callback will wake up
    # the queue management thread.
    def weakref_cb(_, q=self._result_queue):
        q.put(None)
    if self._queue_management_thread is None:
        # Start the processes so that their sentinels are known.
        self._adjust_process_count()
        self._queue_management_thread = threading.Thread(
                target=_queue_management_worker,
                args=(weakref.ref(self, weakref_cb),
                      self._processes,
                      self._pending_work_items,
                      self._work_ids,
                      self._call_queue,
                      self._result_queue))
        self._queue_management_thread.daemon = True
        self._queue_management_thread.start()

答案 1 :(得分:5)

以下模块导入threading,表示可以使用“线程”(无论是隐式还是显式)

  1. asyncio
  2. decimal
  3. functools
  4. cookiejar
  5. multiprocessing
  6. queue
  7. sched
  8. subprocess
  9. telnetlib
  10. tempfile
  11. trace

  12. 方法论

    grep -r "thread" * | grep "import" | grep -v "test" | grep -v "Lib/threading"
    

    {python install path}/Lib目录中。

    使用的方法是查看“线程”是否出现在grep结果中,并使用一系列grep来处理结果。

    所以用一点盐来回答这个问题。