我正在运行一个基于Web的项目,该项目具有以数据科学为重点的后端,该后端托管在Microsoft Azure云服务器上。 Django启动项目时,需要加载并初始化大型深度学习模型。我正在通过Nginx服务器运行项目,该服务器通过gunicorn与Django服务器通信。
如果我仅通过Django在同一Azure群集上运行项目(不使用nginx或gunicorn),它将初始化并正常启动。但是,当我通过gunicorn在nginx上运行它时,其中一名工作人员在项目启动时不断重新启动(初始化可能需要2分钟左右的时间。
当我尝试启动Django项目时,这是gunicorn命令的输出:
user@host:/home/libor/libor_transition/libor_transition_api$ sudo gunicorn --pythonpath="$(pwd)" --log-level=DEBUG -t 1000 libor_transition_api.wsgi:appli>
[2019-07-29 08:43:44 +0000] [33808] [DEBUG] Current configuration:
config: None
bind: ['127.0.0.1:8000']
backlog: 2048
workers: 1
worker_class: sync
threads: 1
worker_connections: 1000
max_requests: 0
max_requests_jitter: 0
timeout: 1000
graceful_timeout: 30
keepalive: 2
limit_request_line: 4094
limit_request_fields: 100
limit_request_field_size: 8190
reload: False
reload_engine: auto
reload_extra_files: []
spew: False
check_config: False
preload_app: False
sendfile: None
reuse_port: False
chdir: /home/libor/libor_transition/libor_transition_api
daemon: False
raw_env: []
pidfile: None
worker_tmp_dir: None
user: 0
group: 0
umask: 0
initgroups: False
tmp_upload_dir: None
secure_scheme_headers: {'X-FORWARDED-PROTOCOL': 'ssl', 'X-FORWARDED-PROTO': 'https', 'X-FORWARDED-SSL': 'on'}
forwarded_allow_ips: ['127.0.0.1']
accesslog: None
disable_redirect_access_to_syslog: False
access_log_format: %(h)s %(l)s %(u)s %(t)s "%(r)s" %(s)s %(b)s "%(f)s" "%(a)s"
errorlog: -
loglevel: DEBUG
capture_output: False
logger_class: gunicorn.glogging.Logger
logconfig: None
logconfig_dict: {}
syslog_addr: udp://localhost:514
syslog: False
syslog_prefix: None
syslog_facility: user
enable_stdio_inheritance: False
statsd_host: None
statsd_prefix:
proc_name: None
default_proc_name: libor_transition_api.wsgi:application
pythonpath: /home/libor/libor_transition/libor_transition_api
paste: None
on_starting: <function OnStarting.on_starting at 0x7fb48ec46048>
on_reload: <function OnReload.on_reload at 0x7fb48ec46158>
when_ready: <function WhenReady.when_ready at 0x7fb48ec46268>
pre_fork: <function Prefork.pre_fork at 0x7fb48ec46378>
post_fork: <function Postfork.post_fork at 0x7fb48ec46488>
post_worker_init: <function PostWorkerInit.post_worker_init at 0x7fb48ec46598>
worker_int: <function WorkerInt.worker_int at 0x7fb48ec466a8>
worker_abort: <function WorkerAbort.worker_abort at 0x7fb48ec467b8>
pre_exec: <function PreExec.pre_exec at 0x7fb48ec468c8>
pre_request: <function PreRequest.pre_request at 0x7fb48ec469d8>
post_request: <function PostRequest.post_request at 0x7fb48ec46a60>
child_exit: <function ChildExit.child_exit at 0x7fb48ec46b70>
worker_exit: <function WorkerExit.worker_exit at 0x7fb48ec46c80>
nworkers_changed: <function NumWorkersChanged.nworkers_changed at 0x7fb48ec46d90>
on_exit: <function OnExit.on_exit at 0x7fb48ec46ea0>
proxy_protocol: False
proxy_allow_ips: ['127.0.0.1']
keyfile: None
certfile: None
ssl_version: 2
cert_reqs: 0
ca_certs: None
suppress_ragged_eofs: True
do_handshake_on_connect: False
ciphers: TLSv1
raw_paste_global_conf: []
[2019-07-29 08:43:44 +0000] [33808] [INFO] Starting gunicorn 19.9.0
[2019-07-29 08:43:44 +0000] [33808] [DEBUG] Arbiter booted
[2019-07-29 08:43:44 +0000] [33808] [INFO] Listening at: http://127.0.0.1:8000 (33808)
[2019-07-29 08:43:44 +0000] [33808] [INFO] Using worker: sync
[2019-07-29 08:43:44 +0000] [33812] [INFO] Booting worker with pid: 33812
[2019-07-29 08:43:44 +0000] [33808] [DEBUG] 1 workers
[2019-07-29 08:43:45 +0000] [33812] [DEBUG] worker: SIGWINCH ignored.
[2019-07-29 08:43:45 +0000] [33808] [INFO] Handling signal: winch
[2019-07-29 08:43:45 +0000] [33808] [DEBUG] SIGWINCH ignored. Not daemonized
[2019-07-29 08:44:04 +0000] [33812] [DEBUG] GET /dashboard/overview
[nltk_data] Downloading package punkt to
[nltk_data] /home/kevin.obrien/nltk_data...
[nltk_data] Package punkt is already up-to-date!
[nltk_data] Downloading package stopwords to
[nltk_data] /home/kevin.obrien/nltk_data...
[nltk_data] Package stopwords is already up-to-date!
[nltk_data] Downloading package perluniprops to
[nltk_data] /home/kevin.obrien/nltk_data...
[nltk_data] Package perluniprops is already up-to-date!
[nltk_data] Downloading package nonbreaking_prefixes to
[nltk_data] /home/kevin.obrien/nltk_data...
[nltk_data] Package nonbreaking_prefixes is already up-to-date!
2019-07-29 08:44:56.800 INFO in 'deeppavlov.core.data.simple_vocab'['simple_vocab'] at line 103: [loading vocabulary from /home/kevin.obrien/.deeppavlov/models/ner_ontonotes/tag.dict]
2019-07-29 08:44:56.802 INFO in 'deeppavlov.core.data.simple_vocab'['simple_vocab'] at line 103: [loading vocabulary from /home/kevin.obrien/.deeppavlov/models/ner_ontonotes/char.dict]
2019-07-29 08:44:56.988 INFO in 'deeppavlov.models.embedders.glove_embedder'['glove_embedder'] at line 52: [loading GloVe embeddings from `/home/kevin.obrien/.deeppavlov/downloads/embeddings/glove.6B.100d.txt`]
[2019-07-29 08:45:31 +0000] [34437] [INFO] Booting worker with pid: 34437
[2019-07-29 08:45:32 +0000] [34437] [DEBUG] GET /favicon.ico
如您所见,在加载deeppavlov
模型时,将再次引导工作程序。我已经在线阅读了这可能是内存问题,但是我将内存从8GB增加了一倍,达到16GB,据我所知,这些型号都不应该如此。同样,由于模型在不使用Gunicorn时加载良好,我相信这也意味着在使用Gunicorn时内存也足够。
让我知道是否需要其他详细信息来解决此问题(例如nginx配置文件)。我是一名数据科学家,所以对前端开发,Web服务器,网络等了解甚少。