Question

我有一个相当简单的Flask应用程序（使用fastAPI）来加载numpy数组并定义一些API端点。

import numpy as np
import pandas as pd
import logging

from fastapi import FastAPI

app = FastAPI()
logging.basicConfig(level=logging.DEBUG)

logging.info('Loading texts')
texts = pd.read_csv('cleaned.csv')
logging.info('Loading embeddings')
embeddings = np.load('laser-2020-04-30.npy') # 3.7G
logging.info('Loading completed!')

# some API endpoints below...

我可以使用纯python3.7运行此应用，而不会出现任何问题。同样，在运行香草味甘蓝糖时，它运行良好。在docker容器中运行所有内容（并使用gunicorn）时，会出现问题。似乎在装入大型numpy数组并引导新工作程序时陷入困境。

[2020-05-11 08:33:20 +0000] [1] [INFO] Starting gunicorn 20.0.4
[2020-05-11 08:33:20 +0000] [1] [DEBUG] Arbiter booted
[2020-05-11 08:33:20 +0000] [1] [INFO] Listening at: http://0.0.0.0:80 (1)
[2020-05-11 08:33:20 +0000] [1] [INFO] Using worker: sync
[2020-05-11 08:33:20 +0000] [7] [INFO] Booting worker with pid: 7
[2020-05-11 08:33:20 +0000] [1] [DEBUG] 1 workers
INFO:root:Loading texts
INFO:root:Loading embeddings
[2020-05-11 08:33:35 +0000] [18] [INFO] Booting worker with pid: 18
INFO:root:Loading texts
INFO:root:Loading embeddings
[2020-05-11 08:33:51 +0000] [29] [INFO] Booting worker with pid: 29
INFO:root:Loading texts
INFO:root:Loading embeddings
[2020-05-11 08:34:05 +0000] [40] [INFO] Booting worker with pid: 40
INFO:root:Loading texts
INFO:root:Loading embeddings
[2020-05-11 08:34:19 +0000] [51] [INFO] Booting worker with pid: 51
INFO:root:Loading texts
INFO:root:Loading embeddings
[2020-05-11 08:34:36 +0000] [62] [INFO] Booting worker with pid: 62

我将工人数设置为1，并将超时时间增加到900秒。但是，它每10到15秒就会引导新员工。

在我的Dockerfile中运行应用程序的命令如下

CMD ["gunicorn","-b 0.0.0.0:8080", "main:app", "--timeout 900", "--log-level", "debug", "--workers", "1", "--graceful-timeout", "900"]

Answer 1

要解决此问题，我只需使用increased the available amount of RAM that the docker container。我的MacBook 2019安装的docker的默认值为2G。由于numpy数组为3.7G，这就是为什么它无法加载的原因。

docker run -m=8g -t my_docker

独角兽：坚持引导新员工

1 个答案: