Question

我正在尝试在EB上启动一个芹菜工作者，但遇到的错误并不能解释太多。

.ebextensions dir中配置文件中的命令：

03_celery_worker:
  command: "celery worker --app=config --loglevel=info -E --workdir=/opt/python/current/app/my_project/"

列出的命令在我的本地计算机上工作正常（只需更改workdir参数）。

来自EB的错误：

活动执行失败，因为：/opt/python/run/venv/local/lib/python3.6/site-packages/celery/platforms.py:796：RuntimeWarning：您正在以超级用户权限运行worker：这是绝对不推荐！

和

开始新的HTTPS连接（1）：eu-west-1.queue.amazonaws.com （ElasticBeanstalk :: ExternalInvocationError）

我已使用参数--uid=2更新了celery worker命令，特权错误消失了，但由于以下原因，命令执行仍然失败

ExternalInvocationError

有什么建议我做错了吗？

Answer 1

ExternalInvocationError

据我了解，这意味着无法从EB容器命令运行列出的命令。需要在服务器上创建脚本并从该脚本运行celery。 This post介绍了操作方法。

更新：需要在.ebextensions目录中创建配置文件。我称它为celery.config。上面文章的链接提供了一个几乎可以正常运行的脚本。需要做一些小的补充才能100％正确地工作。我在安排定期任务时遇到问题（芹菜节拍）。以下是有关解决方法的步骤：

安装（添加至要求） django-celery beat pip install django-celery-beat，将其添加到已安装的应用程序中，并在启动celery beat时使用--scheduler参数。说明是here。
在脚本中，您指定运行脚本的用户。对于芹菜工作者，该用户是celery用户，该用户已在脚本的前面添加（如果不存在）。当我尝试开始芹菜拍打时，出现错误 PermissionDenied 。这意味着芹菜用户没有所有必要的权限。使用ssh，我登录了EB，查看了所有用户（cat /etc/passwd）的列表，并决定使用 daemon 用户。

列出的步骤解决了芹菜跳动错误。下面是使用脚本更新的配置文件（celery.config）： ``` 文件： “ /opt/elasticbeanstalk/hooks/appdeploy/post/run_supervised_celeryd.sh”：模式：“ 000755” 所有者：root 组：根内容：＃！/ usr / bin / env bash

  # Create required directories
  sudo mkdir -p /var/log/celery/
  sudo mkdir -p /var/run/celery/

  # Create group called 'celery'
  sudo groupadd -f celery
  # add the user 'celery' if it doesn't exist and add it to the group with same name
  id -u celery &>/dev/null || sudo useradd -g celery celery
  # add permissions to the celery user for r+w to the folders just created
  sudo chown -R celery:celery /var/log/celery/
  sudo chown -R celery:celery /var/run/celery/

  # Get django environment variables
  celeryenv=`cat /opt/python/current/env | tr '\n' ',' | sed 's/%/%%/g' | sed 's/export //g' | sed 's/$PATH/%(ENV_PATH)s/g' | sed 's/$PYTHONPATH//g' | sed 's/$LD_LIBRARY_PATH//g'`
  celeryenv=${celeryenv%?}

  # Create CELERY configuration script
  celeryconf="[program:celeryd]
  directory=/opt/python/current/app
  ; Set full path to celery program if using virtualenv
  command=/opt/python/run/venv/bin/celery worker -A config.celery:app --loglevel=INFO --logfile=\"/var/log/celery/%%n%%I.log\" --pidfile=\"/var/run/celery/%%n.pid\"

  user=celery
  numprocs=1
  stdout_logfile=/var/log/celery-worker.log
  stderr_logfile=/var/log/celery-worker.log
  autostart=true
  autorestart=true
  startsecs=10

  ; Need to wait for currently executing tasks to finish at shutdown.
  ; Increase this if you have very long running tasks.
  stopwaitsecs = 60

  ; When resorting to send SIGKILL to the program to terminate it
  ; send SIGKILL to its whole process group instead,
  ; taking care of its children as well.
  killasgroup=true

  ; if rabbitmq is supervised, set its priority higher
  ; so it starts first
  priority=998

  environment=$celeryenv"


  # Create CELERY BEAT configuraiton script
  celerybeatconf="[program:celerybeat]
  ; Set full path to celery program if using virtualenv
  command=/opt/python/run/venv/bin/celery beat -A config.celery:app --loglevel=INFO --scheduler django_celery_beat.schedulers:DatabaseScheduler --logfile=\"/var/log/celery/celery-beat.log\" --pidfile=\"/var/run/celery/celery-beat.pid\"

  directory=/opt/python/current/app
  user=daemon
  numprocs=1
  stdout_logfile=/var/log/celerybeat.log
  stderr_logfile=/var/log/celerybeat.log
  autostart=true
  autorestart=true
  startsecs=10

  ; Need to wait for currently executing tasks to finish at shutdown.
  ; Increase this if you have very long running tasks.
  stopwaitsecs = 60

  ; When resorting to send SIGKILL to the program to terminate it
  ; send SIGKILL to its whole process group instead,
  ; taking care of its children as well.
  killasgroup=true

  ; if rabbitmq is supervised, set its priority higher
  ; so it starts first
  priority=999

  environment=$celeryenv"

  # Create the celery supervisord conf script
  echo "$celeryconf" | tee /opt/python/etc/celery.conf
  echo "$celerybeatconf" | tee /opt/python/etc/celerybeat.conf

  # Add configuration script to supervisord conf (if not there already)
  if ! grep -Fxq "celery.conf" /opt/python/etc/supervisord.conf
    then
      echo "[include]" | tee -a /opt/python/etc/supervisord.conf
      echo "files: uwsgi.conf celery.conf celerybeat.conf" | tee -a /opt/python/etc/supervisord.conf
  fi

  # Enable supervisor to listen for HTTP/XML-RPC requests.
  # supervisorctl will use XML-RPC to communicate with supervisord over port 9001.
  # Source: https://askubuntu.com/questions/911994/supervisorctl-3-3-1-http-localhost9001-refused-connection
  if ! grep -Fxq "[inet_http_server]" /opt/python/etc/supervisord.conf
    then
      echo "[inet_http_server]" | tee -a /opt/python/etc/supervisord.conf
      echo "port = 127.0.0.1:9001" | tee -a /opt/python/etc/supervisord.conf
  fi

  # Reread the supervisord config
  supervisorctl -c /opt/python/etc/supervisord.conf reread

  # Update supervisord in cache without restarting all services
  supervisorctl -c /opt/python/etc/supervisord.conf update

  # Start/Restart celeryd through supervisord
  supervisorctl -c /opt/python/etc/supervisord.conf restart celeryd
  supervisorctl -c /opt/python/etc/supervisord.conf restart celerybeat

命令： 01_killotherbeats：命令：“ ps auxww | grep'芹菜节拍'| awk'{print $ 2}'| sudo xargs kill -9 || true” ignoreErrors：是 02_restartbeat：命令：“ supervisorctl -c /opt/python/etc/supervisord.conf重新启动celerybeat” leader_only：是 ``` 需要注意的一件事：我的项目celery.py文件在config目录中，这就是为什么我在启动 celery worker 和芹菜节拍

在Elastic Beanstalk上启动SQS芹菜工人

1 个答案: