所以我想比较线程是更快还是多处理。理论上,由于 GIL ,多处理应该比多线程更快,因为一次只能运行一个线程。但是我得到了相反的结果,即线程比多处理花费的时间少,我想念的是什么,请帮忙。
下面是线程
的代码import threading
from queue import Queue
import time
print_lock = threading.Lock()
def exampleJob(worker):
time.sleep(10)
with print_lock:
print(threading.current_thread().name,worker)
def threader():
while True:
worker = q.get()
exampleJob(worker)
q.task_done()
q = Queue()
for x in range(4):
t = threading.Thread(target=threader)
print(x)
t.daemon = True
t.start()
start = time.time()
for worker in range(8):
q.put(worker)
q.join()
print('Entire job took:',time.time() - start)
下面是多处理
的代码import multiprocessing as mp
import time
def exampleJob(print_lock,worker): # function simulating some computation
time.sleep(10)
with print_lock:
print(mp.current_process().name,worker)
def processor(print_lock,q): # function where process pick up the job
while True:
worker = q.get()
if worker is None: # flag to exit the process
break
exampleJob(print_lock,worker)
if __name__ == '__main__':
print_lock = mp.Lock()
q = mp.Queue()
processes = [mp.Process(target=processor,args=(print_lock,q)) for _ in range(4)]
for process in processes:
process.start()
start = time.time()
for worker in range(8):
q.put(worker)
for process in processes:
q.put(None) # quit indicator
for process in processes:
process.join()
print('Entire job took:',time.time() - start)
答案 0 :(得分:1)
这不是适当的测试。 time.sleep
可能不会获得GIL,因此您正在运行并发线程与并发进程。线程速度更快,因为没有启动成本。
您应该在线程中执行一些计算,然后您会看到差异。
答案 1 :(得分:0)
由于存在GIL,仅当您在执行计算密集型任务时,添加到@zmbq线程才会变慢。如果您的操作是受I / O约束的,并且很少有其他类似的操作,那么线程化肯定会更快,因为所涉及的开销更少。请参考以下博客,以更好地了解该博客。
Exploiting Multiprocessing and Multithreading in Python as a Data Scientist
希望这会有所帮助!