使用Nginx,uWSGI和web.py的MemoryError

时间:2016-06-14 16:48:38

标签: python nginx uwsgi web.py mdm

我有一个python Apple MDM服务器,我试图将其从命令行进程转换为使用Nginx和uWSGI进行托管。我使用web.py进行路由。一切顺利,最适合的。我已经得到了我认为是Nginx和uWSGI的良好基本配置,但今天早上运行它不到24小时后,我在服务器收到的每个其他请求上都收到此错误。有足够的内存可用,我使用的是64位版本的python。

EDIT1:SSL正在端口8080上进行通信,因为它前面有一个负载均衡器,可以将SSL流量路由到该端口。

EDIT2:我已将uWSGI中的流程设置从3更改为1以查看是否存在问题。

EDIT3:这是来自AWS的新盒子。安装了我的服务器及其依赖项,nginx和uWSGI。我在运行uWSGI作为服务时遇到了问题,并且在我最终使aptitude版本与pip一起运行之前安装并卸载了pipsystemd版本。这就是它现在如何运行,它正常地停止,启动和重新启动。

[ Tue Jun 14 2016 10:29:22 ] Traceback (most recent call last):
  File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 237, in process
  File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 565, in processor
  File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 77, in reload_mapping
MemoryError
[ Tue Jun 14 2016 10:29:22 ] 
[ Tue Jun 14 2016 10:29:22 ] Traceback (most recent call last):
  File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 237, in process
  File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 566, in processor
  File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 237, in <lambda>
  File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 246, in process
  File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 477, in internalerror
MemoryError
[ Tue Jun 14 2016 10:29:22 ] 
[ Tue Jun 14 2016 10:29:22 ] Traceback (most recent call last):
  File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 237, in process
  File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 581, in processor
  File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 237, in <lambda>
  File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 246, in process
  File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 477, in internalerror
MemoryError
[ Tue Jun 14 2016 10:29:22 ] 
[ Tue Jun 14 2016 10:29:22 ] Traceback (most recent call last):
  File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 237, in process
  File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 566, in processor
  File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 237, in <lambda>
  File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 246, in process
  File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 477, in internalerror
MemoryError
[ Tue Jun 14 2016 10:29:22 ] 
[ Tue Jun 14 2016 10:29:22 ] Traceback (most recent call last):
[ Tue Jun 14 2016 10:29:22 ]   File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 279, in wsgi
[ Tue Jun 14 2016 10:29:22 ]     [ Tue Jun 14 2016 10:29:22 ] result = self.handle_with_processors()
[ Tue Jun 14 2016 10:29:22 ]   File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 249, in handle_with_processors
[ Tue Jun 14 2016 10:29:22 ]     [ Tue Jun 14 2016 10:29:22 ] return process(self.processors)
[ Tue Jun 14 2016 10:29:22 ]   File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 246, in process
[ Tue Jun 14 2016 10:29:22 ]     [ Tue Jun 14 2016 10:29:22 ] raise self.internalerror()
[ Tue Jun 14 2016 10:29:22 ]   File "/path/to/app/virtualenv/local/lib/python2.7/site-packages/web/application.py", line 477, in internalerror
[ Tue Jun 14 2016 10:29:22 ]     [ Tue Jun 14 2016 10:29:22 ] import debugerror
[ Tue Jun 14 2016 10:29:22 ] MemoryError[ Tue Jun 14 2016 10:29:22 ] 

nginx.conf

user www-data;
worker_processes auto;
pid /run/nginx.pid;

events {
        worker_connections 768;
        # multi_accept on;
}

http {

        ##
        # Basic Settings
        ##

        sendfile on;
        tcp_nopush on;
        tcp_nodelay on;
        keepalive_timeout 65;
        types_hash_max_size 2048;
        # server_tokens off;

        # server_names_hash_bucket_size 64;
        # server_name_in_redirect off;

        include /etc/nginx/mime.types;
        default_type application/octet-stream;

        ##
        # SSL Settings
        ##

        ssl_protocols TLSv1 TLSv1.1 TLSv1.2; # Dropping SSLv3, ref: POODLE
        ssl_prefer_server_ciphers on;

        ##
        # Logging Settings
        ##

        access_log /var/log/nginx/access.log;
        error_log /var/log/nginx/error.log;

        ##
        # Gzip Settings
        ##

        gzip on;
        gzip_disable "msie6";

        # gzip_vary on;
        # gzip_proxied any;
        # gzip_comp_level 6;
        # gzip_buffers 16 8k;
        # gzip_http_version 1.1;
        # gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript;

        ##
        # Virtual Host Configs
        ##

        include /etc/nginx/conf.d/*.conf;
        include /etc/nginx/sites-enabled/*;
}
#mail {
#       # See sample authentication script at:
#       # http://wiki.nginx.org/ImapAuthenticateWithApachePhpScript
# 
#       # auth_http localhost/auth.php;
#       # pop3_capabilities "TOP" "USER";
#       # imap_capabilities "IMAP4rev1" "UIDPLUS";
# 
#       server {
#               listen     localhost:110;
#               protocol   pop3;
#               proxy      on;
#       }
# 
#       server {
#               listen     localhost:143;
#               protocol   imap;
#               proxy      on;
#       }
#}

启用位点

server {
    listen 80;

    location ^~ /static/ {
        root /path/to/app/;
    }

    location / {
        include   uwsgi_params;
        uwsgi_pass  127.0.0.1:9000;
    }
}
server {
    listen   8080;

    ssl    on;
    ssl_certificate    /path/to/app/certs/ssl.crt;
    ssl_certificate_key    /path/to/app/certs/ssl.key;

    location ^~ /static/ {
        root /path/to/app/static/;
    }

    location / {
        include   uwsgi_params;
        uwsgi_pass  127.0.0.1:9000;
        uwsgi_param UWSGI_CHDIR /path/to/app/;
        uwsgi_param UWSGI_PYHOME /path/to/app/virtualenv/;
        uwsgi_param UWSGI_SCRIPT server;
    }
}

uWSGI ini

home = /path/to/app/virtualenv/
chdir = /path/to/app/
module = server:application
logto = /var/log/uwsgi/%n.log
rlogdate =  [ %%a %%b %%d %%Y %%H:%%M:%%S ]
logformat = %(method) %(status) %(addr)  %(uri) wid: %(wid) 
log-encoder = format ${strftime:[ %%a %%b %%d %%Y %%H:%%M:%%S ]} ${msgnl}
gid = ubuntu
uid = ubuntu
vhost = false 
socket = 127.0.0.1:9000
master = true
processes = 3
harakiri = 20
limit-as = 128
memory-report
no-orphans

我被告知我的意外SysAdmin综合症病例相当严重,所以我们非常感谢所有的帮助/建议。只是为了衡量proc/meminfo

的输出
MemTotal:        7657772 kB
MemFree:         5971508 kB
MemAvailable:    7145084 kB
Buffers:          225528 kB
Cached:           900756 kB
SwapCached:            0 kB
Active:          1142912 kB
Inactive:         358288 kB
Active(anon):     380568 kB
Inactive(anon):    11956 kB
Active(file):     762344 kB
Inactive(file):   346332 kB
Unevictable:           0 kB
Mlocked:               0 kB
SwapTotal:             0 kB
SwapFree:              0 kB
Dirty:                48 kB
Writeback:             0 kB
AnonPages:        375008 kB
Mapped:            55840 kB
Shmem:             17616 kB
Slab:             129300 kB
SReclaimable:     106084 kB
SUnreclaim:        23216 kB
KernelStack:        3120 kB
PageTables:         6672 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:     3828884 kB
Committed_AS:     602572 kB
VmallocTotal:   34359738367 kB
VmallocUsed:           0 kB
VmallocChunk:          0 kB
HardwareCorrupted:     0 kB
AnonHugePages:    143360 kB
CmaTotal:              0 kB
CmaFree:               0 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:       71680 kB
DirectMap2M:     7923712 kB

1 个答案:

答案 0 :(得分:1)

您的uWSGI .ini中没有设置max-requests。因此,您的python解释器永远存在并可能积累垃圾。尝试将其设置为某些内容,例如5000,希望泄漏会停止。

如果您将来获得更多统计结果,并且他们反对max-requests版本,请分享它们。祝你好运,James Fitter。