Laravel 5.1:主管队列致命退出但是队列运行:听听就好了

时间:2016-11-14 16:52:21

标签: laravel laravel-5 queue supervisord

我已经使用主管管理我的laravel队列多年没有问题,但突然在11月5日它停止工作并不断退出。我无法在任何地方找到任何错误消息。我已经检查了主管,laravel的日志文件以及我登录到bugsnag的异常,但是没有真正有用的信息。通常我认为这是我的laravel应用程序中的一个问题导致退出,但我发现如果我只是使用所有作业将处理正常:

php artisan queue:listen

但是对于主管,它会运行一段时间然后退出,或者有时会退出spawn。

这是我的程序配置文件

[program:laravel_queue]
command=php artisan queue:work --tries=3
process_name=%(program_name)s_%(process_num)02d
directory=/path/to/my/app
autostart=true
autorestart=true
user=myusername
numprocs=1
stderr_logfile=/path/to/my/logs/supervisor_queue_stderr.log
stdout_logfile=/path/to/my/logs/supervisor_queue_stdout.log

这是我的主管conf文件

; supervisor config file

[unix_http_server]
file=/var/run/supervisor.sock   ; (the path to the socket file)
chmod=0700                       ; sockef file mode (default 0700)

[supervisord]
#logfile=/var/log/supervisor/supervisord.log ; (main log file;default $CWD/supervisord.log)
logfile=/path/to/logs/supervisord.log
pidfile=/var/run/supervisord.pid ; (supervisord pidfile;default supervisord.pid)
childlogdir=/var/log/supervisor            ; ('AUTO' child log dir, default $TEMP)
loglevel=debug

; the below section must remain in the config file for RPC
; (supervisorctl/web interface) to work, additional interfaces may be
; added by defining them in separate rpcinterface: sections
[rpcinterface:supervisor]
supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface

[supervisorctl]
serverurl=unix:///var/run/supervisor.sock ; use a unix:// URL  for a unix socket

; The [include] section can just contain the "files" setting.  This
; setting can list multiple files (separated by whitespace or
; newlines).  It can also contain wildcards.  The filenames are
; interpreted as relative to this file.  Included files *cannot*
; include files themselves.

[include]
files = /path/to/my/program/confs/*.conf

现在我发现了两个主要错误。

第一次有时会在重新启动进程后直接显示:

  

laravel_queue:laravel_queue_00:错误(产生错误)

每次都不会出现,有时会说

  

laravel_queue:laravel_queue_00:已开始

然后退出。

如果我sudo supervisorctl status它说:

  

laravel_queue:laravel_queue_00 FATAL退出太快(进程日志可能包含详细信息)

程序的stdout_logfile日志文件只是一直显示:

Processed: Illuminate\Queue\CallQueuedHandler@call
Processed: Illuminate\Queue\CallQueuedHandler@call
Processed: Illuminate\Queue\CallQueuedHandler@call
Processed: Illuminate\Queue\CallQueuedHandler@call
Processed: Illuminate\Queue\CallQueuedHandler@call

哪个没用。

我将loglevel更改为debug,这是我在主supervisord日志文件中得到的结果:

2016-11-14 15:51:30,255 INFO spawned: 'laravel_queue_00' with pid 13969
2016-11-14 15:51:30,445 DEBG 'laravel_queue_00' stdout output:
Processed: Illuminate\Queue\CallQueuedHandler@call

2016-11-14 15:51:30,445 DEBG fd 11 closed, stopped monitoring <POutputDispatcher at 140364068662736 for <Subprocess at 140364059961176 with name laravel_queue_00 in state STARTING> (stdout)>
2016-11-14 15:51:30,446 DEBG fd 13 closed, stopped monitoring <POutputDispatcher at 140364060359496 for <Subprocess at 140364059961176 with name laravel_queue_00 in state STARTING> (stderr)>
2016-11-14 15:51:30,454 INFO exited: laravel_queue_00 (exit status 0; not expected)
2016-11-14 15:51:30,454 DEBG received SIGCLD indicating a child quit
2016-11-14 15:51:31,457 INFO spawned: 'laravel_queue_00' with pid 13973
2016-11-14 15:51:31,636 DEBG 'laravel_queue_00' stdout output:
Processed: Illuminate\Queue\CallQueuedHandler@call

2016-11-14 15:51:31,637 DEBG fd 10 closed, stopped monitoring <POutputDispatcher at 140364060359496 for <Subprocess at 140364059961176 with name laravel_queue_00 in state STARTING> (stdout)>
2016-11-14 15:51:31,637 DEBG fd 12 closed, stopped monitoring <POutputDispatcher at 140364060578304 for <Subprocess at 140364059961176 with name laravel_queue_00 in state STARTING> (stderr)>
2016-11-14 15:51:31,645 INFO exited: laravel_queue_00 (exit status 0; not expected)
2016-11-14 15:51:31,645 DEBG received SIGCLD indicating a child quit
2016-11-14 15:51:33,649 INFO spawned: 'laravel_queue_00' with pid 13977
2016-11-14 15:51:33,852 DEBG 'laravel_queue_00' stdout output:
Processed: Illuminate\Queue\CallQueuedHandler@call

2016-11-14 15:51:33,853 DEBG fd 10 closed, stopped monitoring <POutputDispatcher at 140364060359496 for <Subprocess at 140364059961176 with name laravel_queue_00 in state STARTING> (stdout)>
2016-11-14 15:51:33,853 DEBG fd 12 closed, stopped monitoring <POutputDispatcher at 140364068662736 for <Subprocess at 140364059961176 with name laravel_queue_00 in state STARTING> (stderr)>
2016-11-14 15:51:33,861 INFO exited: laravel_queue_00 (exit status 0; not expected)
2016-11-14 15:51:33,861 DEBG received SIGCLD indicating a child quit
2016-11-14 15:51:36,864 INFO spawned: 'laravel_queue_00' with pid 13981
2016-11-14 15:51:37,655 DEBG 'laravel_queue_00' stdout output:
Processed: Illuminate\Queue\CallQueuedHandler@call
2016-11-14 15:51:37,656 DEBG 'laravel_queue_00' stdout output:


2016-11-14 15:51:37,658 DEBG fd 10 closed, stopped monitoring <POutputDispatcher at 140364069069168 for <Subprocess at 140364059961176 with name laravel_queue_00 in state STARTING> (stdout)>
2016-11-14 15:51:37,659 DEBG fd 12 closed, stopped monitoring <POutputDispatcher at 140364068662736 for <Subprocess at 140364059961176 with name laravel_queue_00 in state STARTING> (stderr)>
2016-11-14 15:51:37,670 INFO exited: laravel_queue_00 (exit status 0; not expected)
2016-11-14 15:51:37,670 DEBG received SIGCLD indicating a child quit
2016-11-14 15:51:37,992 INFO gave up: laravel_queue_00 entered FATAL state, too many start retries too quickly
2016-11-14 15:51:47,713 INFO spawned: 'laravel_queue_00' with pid 13993
2016-11-14 15:51:47,902 DEBG 'laravel_queue_00' stdout output:
Processed: Illuminate\Queue\CallQueuedHandler@call

2016-11-14 15:51:47,903 DEBG fd 11 closed, stopped monitoring <POutputDispatcher at 140364060709376 for <Subprocess at 140364059961176 with name laravel_queue_00 in state STARTING> (stdout)>
2016-11-14 15:51:47,903 DEBG fd 13 closed, stopped monitoring <POutputDispatcher at 140364060709952 for <Subprocess at 140364059961176 with name laravel_queue_00 in state STARTING> (stderr)>
2016-11-14 15:51:47,911 INFO exited: laravel_queue_00 (exit status 0; not expected)
2016-11-14 15:51:47,911 DEBG received SIGCLD indicating a child quit
2016-11-14 15:51:48,914 INFO spawned: 'laravel_queue_00' with pid 13997
2016-11-14 15:51:49,092 DEBG 'laravel_queue_00' stdout output:
Processed: Illuminate\Queue\CallQueuedHandler@call

2016-11-14 15:51:49,093 DEBG fd 11 closed, stopped monitoring <POutputDispatcher at 140364060709952 for <Subprocess at 140364059961176 with name laravel_queue_00 in state STARTING> (stdout)>
2016-11-14 15:51:49,093 DEBG fd 13 closed, stopped monitoring <POutputDispatcher at 140364060710528 for <Subprocess at 140364059961176 with name laravel_queue_00 in state STARTING> (stderr)>
2016-11-14 15:51:49,101 INFO exited: laravel_queue_00 (exit status 0; not expected)
2016-11-14 15:51:49,101 DEBG received SIGCLD indicating a child quit
2016-11-14 15:51:51,106 INFO spawned: 'laravel_queue_00' with pid 14001
2016-11-14 15:51:51,284 DEBG 'laravel_queue_00' stdout output:
Processed: Illuminate\Queue\CallQueuedHandler@call

2016-11-14 15:51:51,284 DEBG fd 10 closed, stopped monitoring <POutputDispatcher at 140364059718216 for <Subprocess at 140364059961176 with name laravel_queue_00 in state STARTING> (stdout)>
2016-11-14 15:51:51,284 DEBG fd 12 closed, stopped monitoring <POutputDispatcher at 140364060709952 for <Subprocess at 140364059961176 with name laravel_queue_00 in state STARTING> (stderr)>
2016-11-14 15:51:51,293 INFO exited: laravel_queue_00 (exit status 0; not expected)
2016-11-14 15:51:51,293 DEBG received SIGCLD indicating a child quit
2016-11-14 15:51:54,298 INFO spawned: 'laravel_queue_00' with pid 14005
2016-11-14 15:51:54,572 DEBG 'laravel_queue_00' stdout output:
Processed: Illuminate\Queue\CallQueuedHandler@call

2016-11-14 15:51:54,573 DEBG fd 10 closed, stopped monitoring <POutputDispatcher at 140364059718216 for <Subprocess at 140364059961176 with name laravel_queue_00 in state STARTING> (stdout)>
2016-11-14 15:51:54,573 DEBG fd 12 closed, stopped monitoring <POutputDispatcher at 140364060710456 for <Subprocess at 140364059961176 with name laravel_queue_00 in state STARTING> (stderr)>
2016-11-14 15:51:54,582 INFO exited: laravel_queue_00 (exit status 0; not expected)
2016-11-14 15:51:54,583 DEBG received SIGCLD indicating a child quit
2016-11-14 15:51:55,584 INFO gave up: laravel_queue_00 entered FATAL state, too many start retries too quickly
2016-11-14 15:51:58,199 INFO spawned: 'laravel_queue_00' with pid 14013
2016-11-14 15:51:58,394 DEBG 'laravel_queue_00' stdout output:
Processed: Illuminate\Queue\CallQueuedHandler@call

2016-11-14 15:51:58,394 DEBG fd 11 closed, stopped monitoring <POutputDispatcher at 140364060138888 for <Subprocess at 140364059961176 with name laravel_queue_00 in state STARTING> (stdout)>
2016-11-14 15:51:58,395 DEBG fd 13 closed, stopped monitoring <POutputDispatcher at 140364060034472 for <Subprocess at 140364059961176 with name laravel_queue_00 in state STARTING> (stderr)>
2016-11-14 15:51:58,403 INFO exited: laravel_queue_00 (exit status 0; not expected)
2016-11-14 15:51:58,404 DEBG received SIGCLD indicating a child quit
2016-11-14 15:51:59,407 INFO spawned: 'laravel_queue_00' with pid 14017
2016-11-14 15:52:00,407 DEBG 'laravel_queue_00' stdout output:
Processed: Illuminate\Queue\CallQueuedHandler@call

2016-11-14 15:52:00,407 INFO success: laravel_queue_00 entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2016-11-14 15:52:00,409 DEBG fd 11 closed, stopped monitoring <POutputDispatcher at 140364060138888 for <Subprocess at 140364059961176 with name laravel_queue_00 in state RUNNING> (stdout)>
2016-11-14 15:52:00,410 DEBG fd 13 closed, stopped monitoring <POutputDispatcher at 140364060034472 for <Subprocess at 140364059961176 with name laravel_queue_00 in state RUNNING> (stderr)>
2016-11-14 15:52:00,422 INFO exited: laravel_queue_00 (exit status 0; expected)
2016-11-14 15:52:00,422 DEBG received SIGCLD indicating a child quit
2016-11-14 15:52:01,426 INFO spawned: 'laravel_queue_00' with pid 14025
2016-11-14 15:52:01,620 DEBG 'laravel_queue_00' stdout output:
Processed: Illuminate\Queue\CallQueuedHandler@call

2016-11-14 15:52:01,620 DEBG fd 10 closed, stopped monitoring <POutputDispatcher at 140364060138888 for <Subprocess at 140364059961176 with name laravel_queue_00 in state STARTING> (stdout)>
2016-11-14 15:52:01,620 DEBG fd 12 closed, stopped monitoring <POutputDispatcher at 140364060034472 for <Subprocess at 140364059961176 with name laravel_queue_00 in state STARTING> (stderr)>
2016-11-14 15:52:01,630 INFO exited: laravel_queue_00 (exit status 0; not expected)
2016-11-14 15:52:01,630 DEBG received SIGCLD indicating a child quit
2016-11-14 15:52:02,633 INFO spawned: 'laravel_queue_00' with pid 14059
2016-11-14 15:52:03,650 DEBG 'laravel_queue_00' stdout output:
Processed: Illuminate\Queue\CallQueuedHandler@call

2016-11-14 15:52:03,650 INFO success: laravel_queue_00 entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2016-11-14 15:52:03,652 DEBG fd 10 closed, stopped monitoring <POutputDispatcher at 140364060138888 for <Subprocess at 140364059961176 with name laravel_queue_00 in state RUNNING> (stdout)>
2016-11-14 15:52:03,652 DEBG fd 12 closed, stopped monitoring <POutputDispatcher at 140364060034472 for <Subprocess at 140364059961176 with name laravel_queue_00 in state RUNNING> (stderr)>
2016-11-14 15:52:03,663 INFO exited: laravel_queue_00 (exit status 0; expected)
2016-11-14 15:52:03,663 DEBG received SIGCLD indicating a child quit
2016-11-14 15:52:04,666 INFO spawned: 'laravel_queue_00' with pid 14068
2016-11-14 15:52:05,411 DEBG 'laravel_queue_00' stdout output:
Processed: Illuminate\Queue\CallQueuedHandler@call

2016-11-14 15:52:05,413 DEBG fd 10 closed, stopped monitoring <POutputDispatcher at 140364060138888 for <Subprocess at 140364059961176 with name laravel_queue_00 in state STARTING> (stdout)>
2016-11-14 15:52:05,414 DEBG fd 12 closed, stopped monitoring <POutputDispatcher at 140364060034472 for <Subprocess at 140364059961176 with name laravel_queue_00 in state STARTING> (stderr)>
2016-11-14 15:52:05,424 INFO exited: laravel_queue_00 (exit status 0; not expected)
2016-11-14 15:52:05,424 DEBG received SIGCLD indicating a child quit
2016-11-14 15:52:06,427 INFO spawned: 'laravel_queue_00' with pid 14073
2016-11-14 15:52:06,612 DEBG 'laravel_queue_00' stdout output:
Processed: Illuminate\Queue\CallQueuedHandler@call

2016-11-14 15:52:06,613 DEBG fd 10 closed, stopped monitoring <POutputDispatcher at 140364060138888 for <Subprocess at 140364059961176 with name laravel_queue_00 in state STARTING> (stdout)>
2016-11-14 15:52:06,613 DEBG fd 12 closed, stopped monitoring <POutputDispatcher at 140364060036704 for <Subprocess at 140364059961176 with name laravel_queue_00 in state STARTING> (stderr)>
2016-11-14 15:52:06,621 INFO exited: laravel_queue_00 (exit status 0; not expected)
2016-11-14 15:52:06,622 DEBG received SIGCLD indicating a child quit
2016-11-14 15:52:08,626 INFO spawned: 'laravel_queue_00' with pid 14079
2016-11-14 15:52:09,458 DEBG 'laravel_queue_00' stdout output:
Processed: Illuminate\Queue\CallQueuedHandler@call

2016-11-14 15:52:09,460 DEBG fd 10 closed, stopped monitoring <POutputDispatcher at 140364060138888 for <Subprocess at 140364059961176 with name laravel_queue_00 in state STARTING> (stdout)>
2016-11-14 15:52:09,460 DEBG fd 12 closed, stopped monitoring <POutputDispatcher at 140364060035120 for <Subprocess at 140364059961176 with name laravel_queue_00 in state STARTING> (stderr)>
2016-11-14 15:52:09,472 INFO exited: laravel_queue_00 (exit status 0; not expected)
2016-11-14 15:52:09,473 DEBG received SIGCLD indicating a child quit
2016-11-14 15:52:12,478 INFO spawned: 'laravel_queue_00' with pid 14084
2016-11-14 15:52:13,325 DEBG 'laravel_queue_00' stdout output:
Processed: Illuminate\Queue\CallQueuedHandler@call

2016-11-14 15:52:13,327 DEBG fd 10 closed, stopped monitoring <POutputDispatcher at 140364060138888 for <Subprocess at 140364059961176 with name laravel_queue_00 in state STARTING> (stdout)>
2016-11-14 15:52:13,328 DEBG fd 12 closed, stopped monitoring <POutputDispatcher at 140364060033968 for <Subprocess at 140364059961176 with name laravel_queue_00 in state STARTING> (stderr)>
2016-11-14 15:52:13,339 INFO exited: laravel_queue_00 (exit status 0; not expected)
2016-11-14 15:52:13,339 DEBG received SIGCLD indicating a child quit
2016-11-14 15:52:14,340 INFO gave up: laravel_queue_00 entered FATAL state, too many start retries too quickly

那里有一些奇怪的东西,比如有时会说:

  

INFO退出:laravel_queue_00(退出状态0;不预期)

其他时候

  

INFO退出:laravel_queue_00(退出状态0;预期)

没有迹象表明为什么有时候会这样做,有时候不会。

它可能与&#34; startretries&#34;有关。配置选项。我可以在日志中看到一些模式。前两次有4个实例&#34;退出代码0;不期望&#34;和4&#34;处理&#34;输出之前它表示进入致命状态。但是第三次​​它说它进入致命状态它需要8个退出代码。但是在那段时间内有两个&#34;退出状态0的实例;预期&#34;并且在&#34;退出状态0的另外4个实例之后发生最终的致命错误;不期望&#34;所以也许这与4次失败有关。但是然后默认的startretries是3,我没有改变那个配置选项,所以肯定它应该在3之后失败?或者它可能会启动3次并且不会在第4次启动,因此4次失败实际上是正确的。如果是这样,它可能与初级阶段有关。

但这并没有太大帮助,因为我仍然不知道为什么它会失败。看起来工作正在被处理,因为输出说它有,我可以看到数据库中的活动与正在处理的作业一致。那么导致主管停止运行的错误是什么?

我不知道,经过一整天的谷歌搜索,我感到非常失落。任何帮助或指示将不胜感激。

2 个答案:

答案 0 :(得分:2)

This github issue让我在我的主管conf中尝试startsecs=0,这似乎解决了这个问题。

目前尚不清楚为什么,我不确定这是否会导致进一步的问题(如果我的应用程序崩溃会怎么样?)但是对于其他人发现这个问题,你可能想尝试startsecs = 0

我认为发生的事情是,有时我的剧本跑得太快以至于主管认为它已经崩溃了

答案 1 :(得分:0)

这可能发生,因为这意味着

  

程序需要保持运行的总秒数   在创业之后考虑开始成功

所以,如果你的命令要快,那么它将失败