Question

我有一个运行在烧瓶中的应用程序，称为Modeling-manager，它正在接收来自另一个烧瓶应用程序的发布请求。直到那里一切正常。（所有文件都位于同一文件夹中）

通过后请求触发时，建模管理器会在Redis中注册一个作业，以供工作人员执行。要执行的作业称为run_model（），它存储在main.py（烧瓶应用程序）中的不同文件和导入器中

worker与modelling-manager应用程序在同一docker上启动。使用脚本：worker.py

main.py ->烧瓶应用程序

from flask import Flask, request, jsonify
import logging as lg
from datetime import datetime as dt
import os
import boto3
import redis
import json
from rq import Worker, Queue, Connection
from .model import run_model

app = Flask(__name__)

@app.route('/ModelManager/register',  methods=['POST'])
def register_philjob_to_queue():
    lg.info("triggered register cahce on model manager")
    job_data = request.get_json()
    lg.info("json data " + job_data["InstanceNumber"])
    try:
        with Connection(redis.Redis(host="redis-caching", port="6379")):
            lg.info("adding job to queue")
            q = Queue()
            task = q.enqueue(run_model, job_data)
        response_object = {
            'status': 'pending',
            'data': {
                'task_id': task.get_id()
            }
        }
        return jsonify(response_object), 202
        except Exception as e:
          lg.error(e)

worker.py

import redis
from rq import Worker, Queue, Connection

conn = redis.Redis(host="redis-caching", port="6379")
listen = ['default']

if __name__ == '__main__':
    with Connection(conn):
        worker = Worker(list(map(Queue, listen)))
        worker.work()

最终 model.py

import logging as lg
def run_model(job_data):

    # Run phil on started instance
    try:

        lg.info("executing the model") 
        return 'Model Executed'

    except Exception as e:
        print(e)
        print("RUN_MODEL:")
        lg.error(e)

我从工作人员处收到的错误消息，其中“应用程序”可能是指我的烧瓶应用程序（名为应用程序）的根文件夹：

10:35:15 RQ worker 'rq:worker:7555cdf9c1ad.9' started, version 0.12.0
10:35:15 *** Listening on default...
10:35:15 Cleaning registries for queue: default
10:35:15 default: app.model.run_model({'InstanceNumber': '002365984', 
'CallBackUrl': 'http://blabla.be'}) (5e83d96b-de11-4ddb-9c8f- 
b7b256580bfc)
10:35:15 ModuleNotFoundError: No module named 'app'
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/rq/worker.py", line 
793, in perform_job
    rv = job.perform()
  File "/usr/local/lib/python3.7/site-packages/rq/job.py", line 599, in perform
    self._result = self._execute()
  File "/usr/local/lib/python3.7/site-packages/rq/job.py", line 605, in _execute
return self.func(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.7/site-packages/rq/job.py", line 213, 
in func
return import_attribute(self.func_name)
  File "/usr/local/lib/python3.7/site-packages/rq/utils.py", line 152, in import_attribute
    module = importlib.import_module(module_name)
  File "/usr/local/lib/python3.7/importlib/__init__.py", line 127, in 
import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 953, in 
_find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 965, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'app'
Traceback (most recent call last):
 File "/usr/local/lib/python3.7/site-packages/rq/worker.py", line 793, in perform_job
rv = job.perform()
 File "/usr/local/lib/python3.7/site-packages/rq/job.py", line 599, in perform
self._result = self._execute()
  File "/usr/local/lib/python3.7/site-packages/rq/job.py", line 605, in _execute
return self.func(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.7/site-packages/rq/job.py", line 213, in func 
return import_attribute(self.func_name)
  File "/usr/local/lib/python3.7/site-packages/rq/utils.py", line 152, in import_attribute
module = importlib.import_module(module_name)
  File "/usr/local/lib/python3.7/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 953, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 965, in _find_and_load_unlocked
  ModuleNotFoundError: No module named 'app'   
10:35:15 Moving job to 'failed' queue

这都是在docker容器上执行的，redis在另一个容器上执行。这两个容器之间的通信正常。

flask应用程序使用以下命令在docker上启动：烧瓶运行--host = 0.0.0.0 --port = 8081

Answer 1

我正在运行flask应用程序，并在此处遇到了相同的问题。上面的解决方案对我不起作用。但是我的问题与错误的导入有关。

在终端中的Windows上调试时，我的Redis服务器在cygwin中运行。我刚刚从我的静态项目路径开始了我的工作人员。现在，我的导入错误已解决。之前：

rq worker flask-task

之后：

rq worker flask-task --path /mnt/c/Users/<USER>/Desktop/projects/flask_app

Answer 2

如果您将工作目录路径设置在worker.py模块的顶部，则它应该可以工作。

import sys
sys.path.insert(0, '/path/to/working/directory/')

Answer 3

找到了解决方案，问题是由于我用于模块“模型”（.model）的相对路径由于根目录中的 init .py文件，它只能以这种方式工作。因此，我删除了 init .py文件，将导入更改为

from model import run_model

现在一切正常，工作人员可以找到该函数并正确执行该函数！

Answer 4

我也面临着同样的问题。在我的情况下，flask应用可以使用pip进行安装，并且该应用的较旧版本已安装，但没有我要导入的模块。我使用pip卸载命令删除了较旧的模块，现在一切正常。

可能是rq worker试图从已安装的版本而不是相对路径中导入模块。

希望这会有所帮助。

Flask Redis Queue（RQ）worker无法导入名为app的模块

4 个答案: