Dotcloud supervisord显示错误但进程正在运行

时间:2012-06-03 11:24:21

标签: dotcloud supervisord

我的dotcloud设置(带有rabbitmq的django-celery)一周前工作正常 - 流程开始正常,日志很干净。但是,我最近回复了(没有更新任何代码),现在日志说这些进程即使它们似乎正在运行也无法启动。

Supervisord log

dotcloud@hack-default-www-0:/var/log/supervisor$ more supervisord.log
2012-06-03 10:51:51,836 CRIT Set uid to user 1000
2012-06-03 10:51:51,836 WARN Included extra file "/etc/supervisor/conf.d/uwsgi.c
onf" during parsing
2012-06-03 10:51:51,836 WARN Included extra file "/home/dotcloud/current/supervi
sord.conf" during parsing
2012-06-03 10:51:51,938 INFO RPC interface 'supervisor' initialized
2012-06-03 10:51:51,938 WARN cElementTree not installed, using slower XML parser
 for XML-RPC
2012-06-03 10:51:51,938 CRIT Server 'unix_http_server' running without any HTTP
authentication checking
2012-06-03 10:51:51,946 INFO daemonizing the supervisord process
2012-06-03 10:51:51,947 INFO supervisord started with pid 144
2012-06-03 10:51:53,128 INFO spawned: 'celerycam' with pid 159
2012-06-03 10:51:53,133 INFO spawned: 'apnsd' with pid 161
2012-06-03 10:51:53,148 INFO spawned: 'djcelery' with pid 164
2012-06-03 10:51:53,168 INFO spawned: 'uwsgi' with pid 167
2012-06-03 10:51:53,245 INFO exited: djcelery (exit status 1; not expected)
2012-06-03 10:51:53,247 INFO exited: celerycam (exit status 1; not expected)
2012-06-03 10:51:54,698 INFO spawned: 'celerycam' with pid 176
2012-06-03 10:51:54,698 INFO success: apnsd entered RUNNING state, process has s
tayed up for > than 1 seconds (startsecs)
2012-06-03 10:51:54,705 INFO spawned: 'djcelery' with pid 177
2012-06-03 10:51:54,706 INFO success: uwsgi entered RUNNING state, process has s
tayed up for > than 1 seconds (startsecs)
2012-06-03 10:51:54,731 INFO exited: djcelery (exit status 1; not expected)
2012-06-03 10:51:54,754 INFO exited: celerycam (exit status 1; not expected)
2012-06-03 10:51:56,760 INFO spawned: 'celerycam' with pid 178
2012-06-03 10:51:56,765 INFO spawned: 'djcelery' with pid 179
2012-06-03 10:51:56,790 INFO exited: celerycam (exit status 1; not expected)
2012-06-03 10:51:56,791 INFO exited: djcelery (exit status 1; not expected)
2012-06-03 10:51:59,798 INFO spawned: 'celerycam' with pid 180
2012-06-03 10:52:00,538 INFO spawned: 'djcelery' with pid 181
2012-06-03 10:52:00,565 INFO exited: celerycam (exit status 1; not expected)
2012-06-03 10:52:00,571 INFO gave up: celerycam entered FATAL state, too many st
art retries too quickly
2012-06-03 10:52:00,573 INFO exited: djcelery (exit status 1; not expected)
2012-06-03 10:52:01,575 INFO gave up: djcelery entered FATAL state, too many sta
rt retries too quickly
dotcloud@hack-default-www-0:/var/log/supervisor$

djerror日志:

dotcloud@hack-default-www-0:/var/log/supervisor$ more djcelery_error.log
Traceback (most recent call last):
  File "hack/manage.py", line 2, in 
    from django.core.management import execute_manager
ImportError: No module named django.core.management
Traceback (most recent call last):
  File "hack/manage.py", line 2, in 
    from django.core.management import execute_manager
ImportError: No module named django.core.management
Traceback (most recent call last):
  File "hack/manage.py", line 2, in 
    from django.core.management import execute_manager
ImportError: No module named django.core.management
Traceback (most recent call last):
  File "hack/manage.py", line 2, in 
    from django.core.management import execute_manager
ImportError: No module named django.core.management
dotcloud@hack-default-www-0:/var/log/supervisor$

statusctrl显示某些进程正在运行,但pids不同。此外,芹菜功能似乎工作正常。消息被处理,我可以在django管理界面中看到正在处理的消息(dj celery cam正在运行)。

# supervisorctl status
apnsd                            RUNNING    pid 225, uptime 0:00:44
celerycam                        RUNNING    pid 224, uptime 0:00:44
djcelery                         RUNNING    pid 226, uptime 0:00:44

Supervisord.conf文件:

[program:djcelery]
directory = /home/dotcloud/current/
command = python hack/manage.py celeryd -E -l info -c 2
stderr_logfile = /var/log/supervisor/%(program_name)s_error.log
stdout_logfile = /var/log/supervisor/%(program_name)s.log

[program:celerycam]
directory = /home/dotcloud/current/
command = python hack/manage.py celerycam
stderr_logfile = /var/log/supervisor/%(program_name)s_error.log
stdout_logfile = /var/log/supervisor/%(program_name)s.log

http://jefurii.cafejosti.net/blog/2011/01/26/celery-in-virtualenv-with-supervisord/说问题可能是使用的python不正确,所以我在supervisord文件中明确指定了python。它现在有效,但它没有解释我上面看到的内容,以及为什么我必须在上周工作正常时更改我的配置。

此外,并非所有的pid都在排队:

2012-06-03 11:19:03,045 CRIT Server 'unix_http_server' running without any HTTP
authentication checking
2012-06-03 11:19:03,051 INFO daemonizing the supervisord process
2012-06-03 11:19:03,052 INFO supervisord started with pid 144
2012-06-03 11:19:04,061 INFO spawned: 'celerycam' with pid 151
2012-06-03 11:19:04,066 INFO spawned: 'apnsd' with pid 153
2012-06-03 11:19:04,085 INFO spawned: 'djcelery' with pid 155
2012-06-03 11:19:04,104 INFO spawned: 'uwsgi' with pid 156
2012-06-03 11:19:05,271 INFO success: celerycam entered RUNNING state, process h
as stayed up for > than 1 seconds (startsecs)
2012-06-03 11:19:05,271 INFO success: apnsd entered RUNNING state, process has s
tayed up for > than 1 seconds (startsecs)
2012-06-03 11:19:05,271 INFO success: djcelery entered RUNNING state, process ha
s stayed up for > than 1 seconds (startsecs)
2012-06-03 11:19:05,271 INFO success: uwsgi entered RUNNING state, process has s
tayed up for > than 1 seconds (startsecs)

状态显示芹菜凸轮p不排队:

# supervisorctl status
apnsd                            RUNNING    pid 153, uptime 0:06:17
celerycam                        RUNNING    pid 150, uptime 0:06:17
djcelery                         RUNNING    pid 155, uptime 0:06:17

1 个答案:

答案 0 :(得分:2)

我的第一个猜测是你使用错误的python二进制文件(system python而不是virtualenv python),并且它导致了这个错误(下面),因为system python二进制文件没有安装该软件包。

  Traceback (most recent call last):
  File "hack/manage.py", line 2, in 
    from django.core.management import execute_manager
  ImportError: No module named django.core.management

您应该将supervisord.conf更改为以下内容,以确保指向正确的python版本。

[program:djcelery]
directory = /home/dotcloud/current/
command = /home/dotcloud/env/bin/python hack/manage.py celeryd -E -l info -c 2
stderr_logfile = /var/log/supervisor/%(program_name)s_error.log
stdout_logfile = /var/log/supervisor/%(program_name)s.log

[program:celerycam]
directory = /home/dotcloud/current/
command = /home/dotcloud/env/bin/python hack/manage.py celerycam
stderr_logfile = /var/log/supervisor/%(program_name)s_error.log
stdout_logfile = /var/log/supervisor/%(program_name)s.log

python路径从python转到/home/dotcloud/env/bin/python

我不确定为什么主管说它正在运行但不是,但希望这一点改变有助于消除错误,让一切恢复正常。