进行大型计算时的多核效率?

时间:2016-10-07 22:22:01

标签: python multithreading python-2.7

我有一个代码来查找大素数,因为它检查每个奇数,但我想知道我是否可以检查,例如,每隔一个奇数,并且在跳过的数字上检查不同的核心。

1 个答案:

答案 0 :(得分:0)

编辑,添加了多进程,仍然没有看到加速超线程。我不明白为什么多处理没有加速代码,我很感激反馈。也许我没有正确地写出来。您可以尝试自己玩。

代码:

from multiprocessing import Process, Queue
from threading import Thread
import time

#Function to determine if 'a' is prime, used by all methods
def is_prime(a):
    return all(a % i for i in xrange(2, a))

#Look for primes between 'min_prime' and 'max_prime'
min_prime = 3 #must start on odd number!
max_prime = 200000
num_threads = 8
times = {}
primes = {}

#######################
#Multiprocess approach#
#######################
def get_primes(q,start_val,num_threads,max_prime):
    vals = range(start_val,max_prime,num_threads*2)
    q.put([val for val in vals if is_prime(val)])

start_processes_time = time.time()
q = Queue()
processes = []
for i in range(num_threads):
    start = i*2+min_prime
    p = Process(target=get_primes,args=(q,start,num_threads,max_prime+1,))
    processes.append(p)
    p.start()

#Wait for all of them to stop
process_primes = set()
for p in processes:
    p.join()
    process_primes.update(q.get())

times["Processes"] = time.time()-start_processes_time
primes["Processes"] = process_primes

####################
#Threading approach#
####################
class PrimeThread(Thread):
    __slots__ = ["start_val","num_threads","max_prime","vals","primes"]
    def __init__(self,start_val,num_threads,max_prime):
        self.start_val = start_val
        self.num_threads = num_threads
        self.stop = max_prime
        super(PrimeThread, self).__init__()

    def run(self):
        self.vals = range(self.start_val,self.stop,self.num_threads*2)
        self.primes = {val for val in self.vals if is_prime(val)}

threads = []

start_thread_time = time.time()
for i in range(num_threads):
    start = i*2+min_prime
    t = PrimeThread(start,num_threads,max_prime+1)
    threads.append(t)
    t.start()

thread_primes = set()
for t in threads:
    t.join()
    thread_primes.update(t.primes)

times["Threading"] = time.time()-start_thread_time
primes["Threading"] = thread_primes

#######################
#Non-threaded approach#
#######################
start_no_thread_time = time.time()
no_thread_primes = {val for val in range(min_prime,max_prime+1,2) if is_prime(val)}
times["Threadless"] = time.time()-start_no_thread_time
primes["Threadless"] = no_thread_primes

###############
#Compare times#
###############
all_found_same_primes = all([p1 == p2 for p1 in primes.values() for p2 in primes.values()])
print "For min_prime=",min_prime,"max_prime=",max_prime,"and",num_threads,"threads:"
print "All methods found same primes:",all_found_same_primes,"\n"

times = sorted(times.items(), key=lambda x: x[1])
print "\n".join([l+" time: "+str(t) for l,t in times]),"\n"

f_m,f_t = times[0] #get fastest method (f_m) and fastest time (f_t)
print f_m,"was"," and ".join([str(t/f_t)+" times faster than "+l for l,t in times[1:]])

输出:

For min_prime = 3 max_prime = 23 and 8 threads:
All methods found same primes: True

Threadless time: 2.50339508057e-05
Threading time: 0.00144100189209
Processes time: 0.0272209644318

Threadless was 57.5619047619 times faster than Threading and 1087.36190476 times faster than Processes

...

For min_prime = 3 max_prime = 200000 and 8 threads:
All methods found same primes: True

Threadless time: 149.900467873
Processes time: 166.985640049
Threading time: 668.010253906

Threadless was 1.11397677685 times faster than Processes and 4.45635869845 times faster than Threading