我已经使用主管管理我的laravel队列多年没有问题,但突然在11月5日它停止工作并不断退出。我无法在任何地方找到任何错误消息。我已经检查了主管,laravel的日志文件以及我登录到bugsnag的异常,但是没有真正有用的信息。通常我认为这是我的laravel应用程序中的一个问题导致退出,但我发现如果我只是使用所有作业将处理正常:
php artisan queue:listen
但是对于主管,它会运行一段时间然后退出,或者有时会退出spawn。
这是我的程序配置文件
[program:laravel_queue]
command=php artisan queue:work --tries=3
process_name=%(program_name)s_%(process_num)02d
directory=/path/to/my/app
autostart=true
autorestart=true
user=myusername
numprocs=1
stderr_logfile=/path/to/my/logs/supervisor_queue_stderr.log
stdout_logfile=/path/to/my/logs/supervisor_queue_stdout.log
这是我的主管conf文件
; supervisor config file
[unix_http_server]
file=/var/run/supervisor.sock ; (the path to the socket file)
chmod=0700 ; sockef file mode (default 0700)
[supervisord]
#logfile=/var/log/supervisor/supervisord.log ; (main log file;default $CWD/supervisord.log)
logfile=/path/to/logs/supervisord.log
pidfile=/var/run/supervisord.pid ; (supervisord pidfile;default supervisord.pid)
childlogdir=/var/log/supervisor ; ('AUTO' child log dir, default $TEMP)
loglevel=debug
; the below section must remain in the config file for RPC
; (supervisorctl/web interface) to work, additional interfaces may be
; added by defining them in separate rpcinterface: sections
[rpcinterface:supervisor]
supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface
[supervisorctl]
serverurl=unix:///var/run/supervisor.sock ; use a unix:// URL for a unix socket
; The [include] section can just contain the "files" setting. This
; setting can list multiple files (separated by whitespace or
; newlines). It can also contain wildcards. The filenames are
; interpreted as relative to this file. Included files *cannot*
; include files themselves.
[include]
files = /path/to/my/program/confs/*.conf
现在我发现了两个主要错误。
第一次有时会在重新启动进程后直接显示:
laravel_queue:laravel_queue_00:错误(产生错误)
每次都不会出现,有时会说
laravel_queue:laravel_queue_00:已开始
然后退出。
如果我sudo supervisorctl status
它说:
laravel_queue:laravel_queue_00 FATAL退出太快(进程日志可能包含详细信息)
程序的stdout_logfile日志文件只是一直显示:
Processed: Illuminate\Queue\CallQueuedHandler@call
Processed: Illuminate\Queue\CallQueuedHandler@call
Processed: Illuminate\Queue\CallQueuedHandler@call
Processed: Illuminate\Queue\CallQueuedHandler@call
Processed: Illuminate\Queue\CallQueuedHandler@call
哪个没用。
我将loglevel更改为debug,这是我在主supervisord日志文件中得到的结果:
2016-11-14 15:51:30,255 INFO spawned: 'laravel_queue_00' with pid 13969
2016-11-14 15:51:30,445 DEBG 'laravel_queue_00' stdout output:
Processed: Illuminate\Queue\CallQueuedHandler@call
2016-11-14 15:51:30,445 DEBG fd 11 closed, stopped monitoring <POutputDispatcher at 140364068662736 for <Subprocess at 140364059961176 with name laravel_queue_00 in state STARTING> (stdout)>
2016-11-14 15:51:30,446 DEBG fd 13 closed, stopped monitoring <POutputDispatcher at 140364060359496 for <Subprocess at 140364059961176 with name laravel_queue_00 in state STARTING> (stderr)>
2016-11-14 15:51:30,454 INFO exited: laravel_queue_00 (exit status 0; not expected)
2016-11-14 15:51:30,454 DEBG received SIGCLD indicating a child quit
2016-11-14 15:51:31,457 INFO spawned: 'laravel_queue_00' with pid 13973
2016-11-14 15:51:31,636 DEBG 'laravel_queue_00' stdout output:
Processed: Illuminate\Queue\CallQueuedHandler@call
2016-11-14 15:51:31,637 DEBG fd 10 closed, stopped monitoring <POutputDispatcher at 140364060359496 for <Subprocess at 140364059961176 with name laravel_queue_00 in state STARTING> (stdout)>
2016-11-14 15:51:31,637 DEBG fd 12 closed, stopped monitoring <POutputDispatcher at 140364060578304 for <Subprocess at 140364059961176 with name laravel_queue_00 in state STARTING> (stderr)>
2016-11-14 15:51:31,645 INFO exited: laravel_queue_00 (exit status 0; not expected)
2016-11-14 15:51:31,645 DEBG received SIGCLD indicating a child quit
2016-11-14 15:51:33,649 INFO spawned: 'laravel_queue_00' with pid 13977
2016-11-14 15:51:33,852 DEBG 'laravel_queue_00' stdout output:
Processed: Illuminate\Queue\CallQueuedHandler@call
2016-11-14 15:51:33,853 DEBG fd 10 closed, stopped monitoring <POutputDispatcher at 140364060359496 for <Subprocess at 140364059961176 with name laravel_queue_00 in state STARTING> (stdout)>
2016-11-14 15:51:33,853 DEBG fd 12 closed, stopped monitoring <POutputDispatcher at 140364068662736 for <Subprocess at 140364059961176 with name laravel_queue_00 in state STARTING> (stderr)>
2016-11-14 15:51:33,861 INFO exited: laravel_queue_00 (exit status 0; not expected)
2016-11-14 15:51:33,861 DEBG received SIGCLD indicating a child quit
2016-11-14 15:51:36,864 INFO spawned: 'laravel_queue_00' with pid 13981
2016-11-14 15:51:37,655 DEBG 'laravel_queue_00' stdout output:
Processed: Illuminate\Queue\CallQueuedHandler@call
2016-11-14 15:51:37,656 DEBG 'laravel_queue_00' stdout output:
2016-11-14 15:51:37,658 DEBG fd 10 closed, stopped monitoring <POutputDispatcher at 140364069069168 for <Subprocess at 140364059961176 with name laravel_queue_00 in state STARTING> (stdout)>
2016-11-14 15:51:37,659 DEBG fd 12 closed, stopped monitoring <POutputDispatcher at 140364068662736 for <Subprocess at 140364059961176 with name laravel_queue_00 in state STARTING> (stderr)>
2016-11-14 15:51:37,670 INFO exited: laravel_queue_00 (exit status 0; not expected)
2016-11-14 15:51:37,670 DEBG received SIGCLD indicating a child quit
2016-11-14 15:51:37,992 INFO gave up: laravel_queue_00 entered FATAL state, too many start retries too quickly
2016-11-14 15:51:47,713 INFO spawned: 'laravel_queue_00' with pid 13993
2016-11-14 15:51:47,902 DEBG 'laravel_queue_00' stdout output:
Processed: Illuminate\Queue\CallQueuedHandler@call
2016-11-14 15:51:47,903 DEBG fd 11 closed, stopped monitoring <POutputDispatcher at 140364060709376 for <Subprocess at 140364059961176 with name laravel_queue_00 in state STARTING> (stdout)>
2016-11-14 15:51:47,903 DEBG fd 13 closed, stopped monitoring <POutputDispatcher at 140364060709952 for <Subprocess at 140364059961176 with name laravel_queue_00 in state STARTING> (stderr)>
2016-11-14 15:51:47,911 INFO exited: laravel_queue_00 (exit status 0; not expected)
2016-11-14 15:51:47,911 DEBG received SIGCLD indicating a child quit
2016-11-14 15:51:48,914 INFO spawned: 'laravel_queue_00' with pid 13997
2016-11-14 15:51:49,092 DEBG 'laravel_queue_00' stdout output:
Processed: Illuminate\Queue\CallQueuedHandler@call
2016-11-14 15:51:49,093 DEBG fd 11 closed, stopped monitoring <POutputDispatcher at 140364060709952 for <Subprocess at 140364059961176 with name laravel_queue_00 in state STARTING> (stdout)>
2016-11-14 15:51:49,093 DEBG fd 13 closed, stopped monitoring <POutputDispatcher at 140364060710528 for <Subprocess at 140364059961176 with name laravel_queue_00 in state STARTING> (stderr)>
2016-11-14 15:51:49,101 INFO exited: laravel_queue_00 (exit status 0; not expected)
2016-11-14 15:51:49,101 DEBG received SIGCLD indicating a child quit
2016-11-14 15:51:51,106 INFO spawned: 'laravel_queue_00' with pid 14001
2016-11-14 15:51:51,284 DEBG 'laravel_queue_00' stdout output:
Processed: Illuminate\Queue\CallQueuedHandler@call
2016-11-14 15:51:51,284 DEBG fd 10 closed, stopped monitoring <POutputDispatcher at 140364059718216 for <Subprocess at 140364059961176 with name laravel_queue_00 in state STARTING> (stdout)>
2016-11-14 15:51:51,284 DEBG fd 12 closed, stopped monitoring <POutputDispatcher at 140364060709952 for <Subprocess at 140364059961176 with name laravel_queue_00 in state STARTING> (stderr)>
2016-11-14 15:51:51,293 INFO exited: laravel_queue_00 (exit status 0; not expected)
2016-11-14 15:51:51,293 DEBG received SIGCLD indicating a child quit
2016-11-14 15:51:54,298 INFO spawned: 'laravel_queue_00' with pid 14005
2016-11-14 15:51:54,572 DEBG 'laravel_queue_00' stdout output:
Processed: Illuminate\Queue\CallQueuedHandler@call
2016-11-14 15:51:54,573 DEBG fd 10 closed, stopped monitoring <POutputDispatcher at 140364059718216 for <Subprocess at 140364059961176 with name laravel_queue_00 in state STARTING> (stdout)>
2016-11-14 15:51:54,573 DEBG fd 12 closed, stopped monitoring <POutputDispatcher at 140364060710456 for <Subprocess at 140364059961176 with name laravel_queue_00 in state STARTING> (stderr)>
2016-11-14 15:51:54,582 INFO exited: laravel_queue_00 (exit status 0; not expected)
2016-11-14 15:51:54,583 DEBG received SIGCLD indicating a child quit
2016-11-14 15:51:55,584 INFO gave up: laravel_queue_00 entered FATAL state, too many start retries too quickly
2016-11-14 15:51:58,199 INFO spawned: 'laravel_queue_00' with pid 14013
2016-11-14 15:51:58,394 DEBG 'laravel_queue_00' stdout output:
Processed: Illuminate\Queue\CallQueuedHandler@call
2016-11-14 15:51:58,394 DEBG fd 11 closed, stopped monitoring <POutputDispatcher at 140364060138888 for <Subprocess at 140364059961176 with name laravel_queue_00 in state STARTING> (stdout)>
2016-11-14 15:51:58,395 DEBG fd 13 closed, stopped monitoring <POutputDispatcher at 140364060034472 for <Subprocess at 140364059961176 with name laravel_queue_00 in state STARTING> (stderr)>
2016-11-14 15:51:58,403 INFO exited: laravel_queue_00 (exit status 0; not expected)
2016-11-14 15:51:58,404 DEBG received SIGCLD indicating a child quit
2016-11-14 15:51:59,407 INFO spawned: 'laravel_queue_00' with pid 14017
2016-11-14 15:52:00,407 DEBG 'laravel_queue_00' stdout output:
Processed: Illuminate\Queue\CallQueuedHandler@call
2016-11-14 15:52:00,407 INFO success: laravel_queue_00 entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2016-11-14 15:52:00,409 DEBG fd 11 closed, stopped monitoring <POutputDispatcher at 140364060138888 for <Subprocess at 140364059961176 with name laravel_queue_00 in state RUNNING> (stdout)>
2016-11-14 15:52:00,410 DEBG fd 13 closed, stopped monitoring <POutputDispatcher at 140364060034472 for <Subprocess at 140364059961176 with name laravel_queue_00 in state RUNNING> (stderr)>
2016-11-14 15:52:00,422 INFO exited: laravel_queue_00 (exit status 0; expected)
2016-11-14 15:52:00,422 DEBG received SIGCLD indicating a child quit
2016-11-14 15:52:01,426 INFO spawned: 'laravel_queue_00' with pid 14025
2016-11-14 15:52:01,620 DEBG 'laravel_queue_00' stdout output:
Processed: Illuminate\Queue\CallQueuedHandler@call
2016-11-14 15:52:01,620 DEBG fd 10 closed, stopped monitoring <POutputDispatcher at 140364060138888 for <Subprocess at 140364059961176 with name laravel_queue_00 in state STARTING> (stdout)>
2016-11-14 15:52:01,620 DEBG fd 12 closed, stopped monitoring <POutputDispatcher at 140364060034472 for <Subprocess at 140364059961176 with name laravel_queue_00 in state STARTING> (stderr)>
2016-11-14 15:52:01,630 INFO exited: laravel_queue_00 (exit status 0; not expected)
2016-11-14 15:52:01,630 DEBG received SIGCLD indicating a child quit
2016-11-14 15:52:02,633 INFO spawned: 'laravel_queue_00' with pid 14059
2016-11-14 15:52:03,650 DEBG 'laravel_queue_00' stdout output:
Processed: Illuminate\Queue\CallQueuedHandler@call
2016-11-14 15:52:03,650 INFO success: laravel_queue_00 entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2016-11-14 15:52:03,652 DEBG fd 10 closed, stopped monitoring <POutputDispatcher at 140364060138888 for <Subprocess at 140364059961176 with name laravel_queue_00 in state RUNNING> (stdout)>
2016-11-14 15:52:03,652 DEBG fd 12 closed, stopped monitoring <POutputDispatcher at 140364060034472 for <Subprocess at 140364059961176 with name laravel_queue_00 in state RUNNING> (stderr)>
2016-11-14 15:52:03,663 INFO exited: laravel_queue_00 (exit status 0; expected)
2016-11-14 15:52:03,663 DEBG received SIGCLD indicating a child quit
2016-11-14 15:52:04,666 INFO spawned: 'laravel_queue_00' with pid 14068
2016-11-14 15:52:05,411 DEBG 'laravel_queue_00' stdout output:
Processed: Illuminate\Queue\CallQueuedHandler@call
2016-11-14 15:52:05,413 DEBG fd 10 closed, stopped monitoring <POutputDispatcher at 140364060138888 for <Subprocess at 140364059961176 with name laravel_queue_00 in state STARTING> (stdout)>
2016-11-14 15:52:05,414 DEBG fd 12 closed, stopped monitoring <POutputDispatcher at 140364060034472 for <Subprocess at 140364059961176 with name laravel_queue_00 in state STARTING> (stderr)>
2016-11-14 15:52:05,424 INFO exited: laravel_queue_00 (exit status 0; not expected)
2016-11-14 15:52:05,424 DEBG received SIGCLD indicating a child quit
2016-11-14 15:52:06,427 INFO spawned: 'laravel_queue_00' with pid 14073
2016-11-14 15:52:06,612 DEBG 'laravel_queue_00' stdout output:
Processed: Illuminate\Queue\CallQueuedHandler@call
2016-11-14 15:52:06,613 DEBG fd 10 closed, stopped monitoring <POutputDispatcher at 140364060138888 for <Subprocess at 140364059961176 with name laravel_queue_00 in state STARTING> (stdout)>
2016-11-14 15:52:06,613 DEBG fd 12 closed, stopped monitoring <POutputDispatcher at 140364060036704 for <Subprocess at 140364059961176 with name laravel_queue_00 in state STARTING> (stderr)>
2016-11-14 15:52:06,621 INFO exited: laravel_queue_00 (exit status 0; not expected)
2016-11-14 15:52:06,622 DEBG received SIGCLD indicating a child quit
2016-11-14 15:52:08,626 INFO spawned: 'laravel_queue_00' with pid 14079
2016-11-14 15:52:09,458 DEBG 'laravel_queue_00' stdout output:
Processed: Illuminate\Queue\CallQueuedHandler@call
2016-11-14 15:52:09,460 DEBG fd 10 closed, stopped monitoring <POutputDispatcher at 140364060138888 for <Subprocess at 140364059961176 with name laravel_queue_00 in state STARTING> (stdout)>
2016-11-14 15:52:09,460 DEBG fd 12 closed, stopped monitoring <POutputDispatcher at 140364060035120 for <Subprocess at 140364059961176 with name laravel_queue_00 in state STARTING> (stderr)>
2016-11-14 15:52:09,472 INFO exited: laravel_queue_00 (exit status 0; not expected)
2016-11-14 15:52:09,473 DEBG received SIGCLD indicating a child quit
2016-11-14 15:52:12,478 INFO spawned: 'laravel_queue_00' with pid 14084
2016-11-14 15:52:13,325 DEBG 'laravel_queue_00' stdout output:
Processed: Illuminate\Queue\CallQueuedHandler@call
2016-11-14 15:52:13,327 DEBG fd 10 closed, stopped monitoring <POutputDispatcher at 140364060138888 for <Subprocess at 140364059961176 with name laravel_queue_00 in state STARTING> (stdout)>
2016-11-14 15:52:13,328 DEBG fd 12 closed, stopped monitoring <POutputDispatcher at 140364060033968 for <Subprocess at 140364059961176 with name laravel_queue_00 in state STARTING> (stderr)>
2016-11-14 15:52:13,339 INFO exited: laravel_queue_00 (exit status 0; not expected)
2016-11-14 15:52:13,339 DEBG received SIGCLD indicating a child quit
2016-11-14 15:52:14,340 INFO gave up: laravel_queue_00 entered FATAL state, too many start retries too quickly
那里有一些奇怪的东西,比如有时会说:
INFO退出:laravel_queue_00(退出状态0;不预期)
其他时候
INFO退出:laravel_queue_00(退出状态0;预期)
没有迹象表明为什么有时候会这样做,有时候不会。
它可能与&#34; startretries&#34;有关。配置选项。我可以在日志中看到一些模式。前两次有4个实例&#34;退出代码0;不期望&#34;和4&#34;处理&#34;输出之前它表示进入致命状态。但是第三次它说它进入致命状态它需要8个退出代码。但是在那段时间内有两个&#34;退出状态0的实例;预期&#34;并且在&#34;退出状态0的另外4个实例之后发生最终的致命错误;不期望&#34;所以也许这与4次失败有关。但是然后默认的startretries是3,我没有改变那个配置选项,所以肯定它应该在3之后失败?或者它可能会启动3次并且不会在第4次启动,因此4次失败实际上是正确的。如果是这样,它可能与初级阶段有关。
但这并没有太大帮助,因为我仍然不知道为什么它会失败。看起来工作正在被处理,因为输出说它有,我可以看到数据库中的活动与正在处理的作业一致。那么导致主管停止运行的错误是什么?
我不知道,经过一整天的谷歌搜索,我感到非常失落。任何帮助或指示将不胜感激。
答案 0 :(得分:2)
This github issue让我在我的主管conf中尝试startsecs=0
,这似乎解决了这个问题。
目前尚不清楚为什么,我不确定这是否会导致进一步的问题(如果我的应用程序崩溃会怎么样?)但是对于其他人发现这个问题,你可能想尝试startsecs = 0
我认为发生的事情是,有时我的剧本跑得太快以至于主管认为它已经崩溃了
答案 1 :(得分:0)
这可能发生,因为这意味着
程序需要保持运行的总秒数 在创业之后考虑开始成功
所以,如果你的命令要快,那么它将失败