重新启动运行supervisord程序的docker会保留pid文件并在重新启动时导致错误

时间:2015-02-13 18:57:22

标签: docker celery supervisord fig

我有一个通过supervisord运行django celery worker的docker,程序设置非常简单

[program:celery_priority]
command=python manage.py celery worker -E -Q priority --concurrency=2 --loglevel=ERROR
directory=/var/lib/app
stdout_events_enabled = true
stderr_events_enabled = true
stopwaitsecs = 600

[program:celery_medium]
command=python manage.py celery worker -E -Q medium --concurrency=2 --loglevel=ERROR
directory=/var/lib/app
stdout_events_enabled = true
stderr_events_enabled = true
stopwaitsecs = 600

[program:celerycam]
command=python manage.py celerycam
directory=/var/lib/app
stdout_events_enabled = true
stderr_events_enabled = true
stopwaitsecs = 600

我们的部署周期使用fig来管理docker,这是我们的fig.yml文件对于worker来说的样子

worker:
  build: .docker/worker
  command: normal
  volumes_from:
    - appdata
  hostname: workerprod
  domainname: project.internal
  links:
    - redis
    - rabbit
    - appdata
    - mail

我们面临的问题是,当我们尝试使用fig restart worker时,supervisord程序失败,因为它在pid中发现冲突并伴有以下错误

[130.211.XX.XX] out: worker_1     | celery_medium stderr | [2015-02-13 13:40:54,271: WARNING/MainProcess] ERROR: Pidfile (/tmp/med_celery.pid) already exists.
[130.211.XX.XX] out: worker_1     | Seems we're already running? (pid: 17)
[130.211.XX.XX] out: worker_1     | celery_priority stderr | [2015-02-13 13:40:54,272: WARNING/MainProcess] ERROR: Pidfile (/tmp/priority_celery.pid) already exists.
[130.211.XX.XX] out: worker_1     | Seems we're already running? (pid: 16)
[130.211.XX.XX] out: worker_1     | 2015-02-13 18:40:54,359 INFO exited: celery_medium (exit status 0; expected)
[130.211.XX.XX] out: worker_1     | 2015-02-13 18:40:54,359 INFO exited: celery_priority (exit status 0; expected)
[130.211.XX.XX] out: worker_1     | 2015-02-13 18:40:55,360 INFO success: celerycam entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)

然而,当我们使用fig -d up worker时,它起作用,因为显然up无花果试图重新创建容器而不使用现有容器。但这会导致所有链接的服务也重新创建,从而丢失RabbitMQ数据和Redis缓存。

有没有办法使用简单的fig restart worker重新启动docker并确保重启时pid清除?请指教

1 个答案:

答案 0 :(得分:5)

创建ENTRYPOINT脚本,在运行CMD之前清除所有状态数据。 E.g。

FROM someotherimage
COPY entrypoint.sh /entrypoint.sh
ENTRYPOINT ["/entrypoint.sh"]

entrypoint.sh

#!/bin/sh
rm -f /tmp/*.pid
exec "$@"

每次容器启动时都会运行ENTRYPOINT脚本,并确保在运行容器命令之前清除/tmp中的所有pid文件。