我们在两个EC2实例(负载均衡器后面)的Laravel Queues之上使用了beanstalkd。我们的AWS EC2实例使用forge自动配置,包括确保beanstalk始终在运行的主管。
使用beanstalkd,我使用beanstalk实用程序来监视队列的运行状况;例如,如果队列被阻塞,我可以通过运行如下命令轻松地看到:
./beanstalk-queue-stats.rb localhost:11300
会返回类似这样的内容
2018-01-10 11:08:38 +0000
default
- cmd-delete = 32678 (+32678)
- cmd-pause-tube = 0 (0)
- current-jobs-buried = 0 (0)
- current-jobs-delayed = 0 (0)
- current-jobs-ready = 0 (0)
- current-jobs-reserved = 0 (0)
- current-jobs-urgent = 0 (0)
- current-using = 5 (+5)
- current-waiting = 0 (0)
- current-watching = 2 (+2)
- pause = 0 (0)
- pause-time-left = 0 (0)
- total-jobs = 32678 (+32678)
high
- cmd-delete = 116005 (+116005)
- cmd-pause-tube = 0 (0)
- current-jobs-buried = 0 (0)
- current-jobs-delayed = 9 (+9)
- current-jobs-ready = 1117 (+1117)
- current-jobs-reserved = 0 (0)
- current-jobs-urgent = 0 (0)
- current-using = 0 (0)
- current-waiting = 0 (0)
- current-watching = 1 (+1)
- pause = 0 (0)
- pause-time-left = 0 (0)
- total-jobs = 117131 (+117131)
low
- cmd-delete = 0 (0)
- cmd-pause-tube = 0 (0)
- current-jobs-buried = 0 (0)
- current-jobs-delayed = 0 (0)
- current-jobs-ready = 0 (0)
- current-jobs-reserved = 0 (0)
- current-jobs-urgent = 0 (0)
- current-using = 0 (0)
- current-waiting = 0 (0)
- current-watching = 1 (+1)
- pause = 0 (0)
- pause-time-left = 0 (0)
- total-jobs = 0 (0)
marketing
- cmd-delete = 0 (0)
- cmd-pause-tube = 0 (0)
- current-jobs-buried = 0 (0)
- current-jobs-delayed = 0 (0)
- current-jobs-ready = 0 (0)
- current-jobs-reserved = 0 (0)
- current-jobs-urgent = 0 (0)
- current-using = 0 (0)
- current-waiting = 0 (0)
- current-watching = 1 (+1)
- pause = 0 (0)
- pause-time-left = 0 (0)
- total-jobs = 0 (0)
------------------
等。 (参见相关问题here)
问题是我继续因为这个错误而被踢出Beanstalk
/home/forge/.rvm/gems/ruby-2.4.1/gems/beanstalk-client-1.1.1/lib/beanstalk-client/connection.rb:212:in `check_resp': NOT_FOUND (Beanstalk::NotFoundError)
from /home/forge/.rvm/gems/ruby-2.4.1/gems/beanstalk-client-1.1.1/lib/beanstalk-client/connection.rb:231:in `read_yaml'
from /home/forge/.rvm/gems/ruby-2.4.1/gems/beanstalk-client-1.1.1/lib/beanstalk-client/connection.rb:195:in `interact'
from /home/forge/.rvm/gems/ruby-2.4.1/gems/beanstalk-client-1.1.1/lib/beanstalk-client/connection.rb:173:in `stats_tube'
from ./beanstalk-queue-stats.rb:21:in `block (2 levels) in <main>'
from ./beanstalk-queue-stats.rb:19:in `each'
from ./beanstalk-queue-stats.rb:19:in `block in <main>'
from ./beanstalk-queue-stats.rb:17:in `loop'
from ./beanstalk-queue-stats.rb:17:in `<main>'
这两种情况都会发生这种情况。我检查了Forge的工人,他们似乎很好:
worker-157645:worker-157645_00 RUNNING pid 10110, uptime 0:25:47
worker-157646:worker-157646_00 RUNNING pid 10236, uptime 0:25:44
worker-157647:worker-157647_00 RUNNING pid 10336, uptime 0:25:42
worker-157648:worker-157648_00 RUNNING pid 10472, uptime 0:25:36
更重要的是,pid没有改变(我会假设每次Beanstalk崩溃时pid都会改变..但似乎不是这样)
毋庸置疑,这会弄乱我们的系统,因为应该在队列中处理的作业有时会起作用,有时不起作用,有时需要一段时间才能工作。
有什么想法吗?
如果我grep运行Beanstalk的进程,我会得到所有这些:
forge 1596 0.1 0.2 65648 10632 pts/1 Sl+ 09:11 0:00 ruby ./beanstalk-queue-stats.rb localhost:11300
forge 1956 0.0 0.0 4512 700 ? S 09:12 0:00 sh -c '/usr/bin/php7.1' 'artisan' queue:work 'beanstalkd' --once --queue='default' --delay=0 --memory=128 --sleep=6 --tries=3 --env='production'
forge 1957 2.0 1.2 384800 49532 ? S 09:12 0:00 /usr/bin/php7.1 artisan queue:work beanstalkd --once --queue=default --delay=0 --memory=128 --sleep=6 --tries=3 --env=production
forge 1996 0.0 0.0 4512 788 ? S 09:12 0:00 sh -c '/usr/bin/php7.1' 'artisan' queue:work 'beanstalkd' --once --queue='low' --delay=0 --memory=128 --sleep=9 --tries=3 --env='production'
forge 1997 3.5 1.2 384800 49616 ? S 09:12 0:00 /usr/bin/php7.1 artisan queue:work beanstalkd --once --queue=low --delay=0 --memory=128 --sleep=9 --tries=3 --env=production
forge 2036 0.0 0.0 4512 744 ? S 09:12 0:00 sh -c '/usr/bin/php7.1' 'artisan' queue:work 'beanstalkd' --once --queue='high' --delay=0 --memory=128 --sleep=2 --tries=3 --env='production'
forge 2037 5.5 1.2 384800 49360 ? S 09:12 0:00 /usr/bin/php7.1 artisan queue:work beanstalkd --once --queue=high --delay=0 --memory=128 --sleep=2 --tries=3 --env=production
forge 2041 0.0 0.0 4512 804 ? S 09:12 0:00 sh -c '/usr/bin/php7.1' 'artisan' queue:work 'beanstalkd' --once --queue='marketing' --delay=0 --memory=128 --sleep=3 --tries=3 --env='production'
forge 2042 12.0 1.2 384800 49368 ? S 09:12 0:00 /usr/bin/php7.1 artisan queue:work beanstalkd --once --queue=marketing --delay=0 --memory=128 --sleep=3 --tries=3 --env=production
forge 2059 0.0 0.0 12924 1036 pts/3 S+ 09:12 0:00 grep --color=auto beanstalk
forge 10110 0.1 1.1 374208 46108 ? S 08:18 0:04 php /home/forge/default/envoyer/current/artisan queue:listen beanstalkd --sleep=2 --quiet --tries=3 --queue=high
forge 10236 0.0 1.1 374208 46568 ? S 08:19 0:02 php /home/forge/default/envoyer/current/artisan queue:listen beanstalkd --sleep=6 --quiet --tries=3 --queue=default
forge 10336 0.0 1.1 374208 46588 ? S 08:19 0:01 php /home/forge/default/envoyer/current/artisan queue:listen beanstalkd --sleep=9 --quiet --tries=3 --queue=low
forge 10472 0.0 1.1 374204 46572 ? S 08:19 0:02 php /home/forge/default/envoyer/current/artisan queue:listen beanstalkd --sleep=3 --quiet --timeout=60 --tries=3 --queue=marketing
beansta+ 17275 0.0 0.1 25124 4600 ? Ss 2017 21:49 /usr/bin/beanstalkd -l 0.0.0.0 -p 11300
看起来不对劲,似乎有这么多。