Question

我刚刚在AWS上部署了一个m5.4xlarge来测试多处理性能，我得到了奇怪的结果。

multiprocessing.cpu_count()返回16

#home I5-3570K 4cores/4threads, with a pool size of 4 : Computation took 5.15700006485 seconds
#aws m5.4xlarge 16 threads, with a pool size of 4 : Computation took 3.80112195015 seconds
#aws m5.4xlarge 16 threads, with a pool size of 8 : Computation took 3.77861309052 seconds
#aws m5.4xlarge 16 threads, with a pool size of 15 : Computation took 3.26295304298 seconds
#aws m5.4xlarge 16 threads, with a pool size of 16 : Computation took 4.16541814804 seconds

我的剧本中有错误吗？

# coding: utf-8

import hashlib
import time

from multiprocessing import Pool

#on a fresh AWS linux instance run :
#sudo yum groupinstall "Development Tools"
#sudo easy_install hashlib

def compute_hash_256(very_random_string):
    return hashlib.sha256(very_random_string).hexdigest()

if __name__ == '__main__':
    POOL_SIZE = 16 #number of threads of our computer
    pool = Pool(processes=POOL_SIZE)

    ########################### generates strings for hashing
    N_STRINGS = 3000000
    print "Generating {} strings for hashing...".format(N_STRINGS)
    random_strings = []
    padding_size = len(str(N_STRINGS))
    for i in range(N_STRINGS):
        random_strings.append(str(i).zfill(padding_size))

    ############################ hashes the strings using multiprocessing
    print "Computing {} hashes".format(len(random_strings))
    start = time.time()
    hashes = pool.map(compute_hash_256, random_strings)
    end = time.time()
    print "Computation took {} seconds".format(end-start)

由于

Answer 1

当你进行计算密集型工作时，有一个分配线程的规则，线程的数量应该总是少于机器中核心的数量。如果线程数增加，则会出现竞争条件，你的算法将会花更多时间回馈结果

NoOfThreads＆lt; NoOfCores

您可以使用此代码检查核心数

/**
 * This method controls what happens when you move to /admin or /admin/index in your app.
 */
public function index()
{
    $this->Language->load('error/not_found'); 

    $header = $this->Language->get('heading_title');   

    $this->View->render('admin/index','admin', array(
            'users' => UserModel::getPublicProfilesOfAllUsers(),
            'header' => $header
        )
    );
}

增加池大小时，多处理性能会降低

1 个答案: