当使用许多等待响应的CURL请求时,为什么我的CPU很高?

时间:2012-05-09 21:01:03

标签: php curl apache curl-multi

我有一个大型的PHP脚本,它对我自己的服务器和其他服务器使用curl和多卷曲请求。除此之外,它还在我自己的服务器上执行了许多数据库请求和其他操作。

脚本太大而无法放在这里,也不会让事情变得更容易。

我测量了整个脚本的时间。我区分了自己服务器上的操作所需的时间和外部服务器上的操作。

似乎在我自己的服务器上累计所有操作所用的时间总是不会花费超过0.5秒。因此,我希望我自己的脚本对我的服务器来说并不重要。总耗用时间平均为25秒。

大部分时间:24.5秒我的服务器正在等待来自外部服务器的响应。我的服务器确实发出了大约300个卷曲请求,但卷曲请求本身并不重。

问题: 当我运行脚本时,只要此脚本正在运行,cpu就会达到95%并且服务器(几乎)没有响应其他请求。

这是多卷曲请求的示例:

<?PHP
// Multi curl:
$imax=20;
// build the individual requests as above, but do not execute them
for ($i=1; $i<$imax; $i ++ ):
${'ch_'.$i} = curl_init('http://www.domain.com/');
//$ch_2 = curl_init('http://www.domain.com/');
curl_setopt(${'ch_'.$i}, CURLOPT_RETURNTRANSFER, true);
//curl_setopt($ch_2, CURLOPT_RETURNTRANSFER, true);
endfor;

// build the multi-curl handle, adding both $ch
$mh = curl_multi_init();
for ($i=1; $i<$imax; $i ++ ):
curl_multi_add_handle($mh, ${'ch_'.$i});
endfor;

// execute all queries simultaneously, and continue when all are complete
$running = null;
do {
curl_multi_exec($mh, $running);
} while ($running);

// all of our requests are done, we can now access the results
for ($i=1; $i<$imax; $i ++ ):
${'response_'.$i} = curl_multi_getcontent(${'ch_'.$i});
endfor;
//$response_2 = curl_multi_getcontent($ch_2);

for ($i=1; $i<$imax; $i ++ ):
echo "${'response_'.$i}"; // same output as first example
endfor
?>

的httpd.conf:

#
# This is the main Apache HTTP server configuration file.  It contains the
# configuration directives that give the server its instructions.
# See <URL:http://httpd.apache.org/docs/2.2> for detailed information.
# In particular, see 
# <URL:http://httpd.apache.org/docs/2.2/mod/directives.html>
# for a discussion of each configuration directive.
#
# Do NOT simply read the instructions in here without understanding
# what they do.  They're here only as hints or reminders.  If you are unsure
# consult the online docs. You have been warned.  

ServerRoot "/etc/httpd"
Listen 80

#LoadModule dummy_module /usr/lib/apache/mod_dummy.so
#LoadModule php5_module        /usr/lib/apache/libphp5.so
Include /etc/httpd/conf/extra/httpd-phpmodules.conf

User apache
Group apache

KeepAlive On
KeepAliveTimeout 10

ServerAdmin admin@localhost
DocumentRoot "/var/www/html"

MaxClients 10000
MaxRequestsPerChild 50000

<Directory /home/*>
    AllowOverride All
    Options -MultiViews -Indexes FollowSymlinks IncludesNoExec +Includes
<Limit GET POST OPTIONS PROPFIND>
    Order allow,deny
    Allow from all
</Limit>
<LimitExcept GET POST OPTIONS PROPFIND>
    Order deny,allow
    Deny from all
</LimitExcept>
</Directory>

<Directory />
    Options All
    AllowOverride All
</Directory>

<Directory "/var/www/html">
    Options Indexes FollowSymLinks
    AllowOverride All
    Order allow,deny
    Allow from all
   <IfModule mod_suphp.c>
        suPHP_Engine On
        suPHP_UserGroup webapps webapps
    SetEnv PHP_INI_SCAN_DIR
   </IfModule>
</Directory>

<IfModule mod_dir.c>
    DirectoryIndex index.html index.htm index.shtml index.php index.php5 index.php4 index.php3 index.phtml index.cgi
</IfModule>

<FilesMatch "^\.ht">
    Order allow,deny
    Deny from all
    Satisfy All
</FilesMatch>

ErrorLog /var/log/httpd/error_log
LogLevel warn

#replace %b with %O for more accurate logging
<IfModule mod_logio.c>
    LogFormat "%h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" combined
    LogFormat "%h %l %u %t \"%r\" %>s %O" common
    LogFormat "%O %I" bytes
    LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %I %O" combinedio
</IfModule>

    CustomLog /var/log/httpd/access_log common

<IfModule mod_alias.c>
    # Include some DirectAdmin alias
    Include conf/extra/httpd-alias.conf
</IfModule>

<Directory "/var/www/cgi-bin">
    AllowOverride None
    Options None
    Order allow,deny
    Allow from all
</Directory>

DefaultType text/plain

<IfModule mod_mime.c>
    TypesConfig conf/mime.types
    AddType application/x-gzip .tgz
    AddEncoding x-compress .Z
    AddEncoding x-gzip .gz .tgz
    AddType application/x-compress .Z
    AddType application/x-gzip .gz .tgz
    AddHandler cgi-script .cgi
    AddHandler type-map var
    AddType text/html .shtml
    AddOutputFilter INCLUDES .shtml
    AddType video/x-ms-asf .avi
    AddType video/mpeg .mpg
    AddType video/mpeg .mpeg
    AddType video/quicktime .mov
    AddType video/x-ms-wmv .wmv
</IfModule>

#EnableMMAP off
#EnableSendfile off

#######################################################################################
# Do not change anything in included files, because they are rewritten by DirectAdmin #
#######################################################################################

# This is needed for PHP
Include conf/extra/httpd-php-handlers.conf

# Server-pool management (MPM specific)
Include conf/extra/httpd-mpm.conf

# Multi-language error messages
Include conf/extra/httpd-multilang-errordoc.conf

# Fancy directory listings
Include conf/extra/httpd-autoindex.conf

# Language settings
Include conf/extra/httpd-languages.conf

# User home directories
#Include conf/extra/httpd-userdir.conf

# Real-time info on requests and configuration
Include conf/extra/httpd-info.conf

# Virtual hosts
Include conf/extra/httpd-vhosts.conf

# Local access to the Apache HTTP Server Manual
#Include conf/extra/httpd-manual.conf

# Distributed authoring and versioning (WebDAV)
Include conf/extra/httpd-dav.conf

# Various default settings
Include conf/extra/httpd-default.conf

# Secure (SSL/TLS) connections
Include conf/extra/httpd-ssl.conf

# Deflate module settings
Include conf/extra/httpd-deflate.conf

# All the DirectAdmin vhosts
Include conf/extra/directadmin-vhosts.conf

# All suPHP directives
Include conf/extra/httpd-suphp.conf

# For user configurations not maintained by DirectAdmin. Empty by default.
Include conf/extra/httpd-includes.conf

#######################################################################################
# End of included files that are rewritten by DirectAdmin                             #
#######################################################################################

<IfModule mod_ssl.c>
SSLRandomSeed startup builtin
SSLRandomSeed connect builtin
</IfModule>

我的问题: 如何更好地配置我的服务器以防止过载?

1 个答案:

答案 0 :(得分:0)

只是一个猜测:curl-multi-exec返回一个你可以(并且应该)检查while块的值

do {
    $mrc = curl_multi_exec($mh, $active);
} while ($active && $mrc == CURLM_CALL_MULTI_PERFORM);

如PHP文档中所述