Question

我正在尝试将长时间的处理任务分配给后台，以便使UI保持响应状态。而不是使用不是真正的并发进程的多线程，我将后台任务实现为多处理。但是我一直遇到错误提示

Traceback (most recent call last):
  File "C:\source\MyApp\MyApp\env\lib\site-packages\engineio\server.py", line 411, in _trigger_event
    return self.handlers[event](*args)
  File "C:\source\MyApp\MyApp\env\lib\site-packages\socketio\server.py", line 522, in _handle_eio_message
    self._handle_event(sid, pkt.namespace, pkt.id, pkt.data)
  File "C:\source\MyApp\MyApp\env\lib\site-packages\socketio\server.py", line 458, in _handle_event
    self._handle_event_internal(self, sid, data, namespace, id)
  File "C:\source\MyApp\MyApp\env\lib\site-packages\socketio\server.py", line 461, in _handle_event_internal
    r = server._trigger_event(data[0], namespace, sid, *data[1:])
  File "C:source\MyApp\MyApp\env\lib\site-packages\socketio\server.py", line 490, in _trigger_event
    return self.handlers[namespace][event](*args)
  File "C:\source\MyApp\MyApp\env\lib\site-packages\flask_socketio\__init__.py", line 251, in _handler
    *args)
  File "C:\source\MyApp\MyApp\env\lib\site-packages\flask_socketio\__init__.py", line 634, in _handle_event
    ret = handler(*args)
  File "C:\source\MyApp\MyApp\MyApp\routes.py", line 171, in StartProcess
    myClassObj.Start()
  File "C:\source\MyApp\MyAppv\src\Controller.py", line 121, in Start
    p.start()
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\multiprocessing\process.py", line 105, in start
    self._popen = self._Popen(self)
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\multiprocessing\context.py", line 223, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\multiprocessing\context.py", line 322, in _Popen
    return Popen(process_obj)
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\multiprocessing\popen_spawn_win32.py", line 65, in __init__
    reduction.dump(process_obj, to_child)
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\multiprocessing\reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)
TypeError: can't pickle _thread.RLock objects

我想知道在使用多处理程序时哪里做错了。

class MyClass(Object):
    def __init__(self):
       self._logger= logging.getLogger('MyClassObj')
       pass
    def Start(self):
       p = Process(self.Test, args(1,))
       p.start()
    def Test(self, number):
       print(number)

我从烧瓶route.py调用此类方法，其中

@socketio.on('StartProcess')
def StartProcess(msg):
    """"Do Processing"""
    myClassObj.Start()
    return 'OK'

此route.py不是我的应用程序的入口点，它由runserver.py调用

from os import environ
from MyApp import routes
from MyApp import app
from flask_socketio import SocketIO
from MyApp.routes import socketio

if __name__ == '__main__':
   HOST = environ.get('SERVER_HOST', 'localhost')
try:
    PORT = int(environ.get('SERVER_PORT', '5555'))
except ValueError:
    PORT = 5555
socketio.run(app, host='0.0.0.0', port=PORT, debug=False)

我确实不断看到人们提到在声明下运行多处理需求

if __name__ == '__main__':

但是我不确定如何正确使用多重处理，因为在入口点（runserver.py），我不需要后台进程。

===编辑====

我创建了一个非常简单的示例来进一步阐述这个问题。我的应用程序具有一个简单的flask应用程序结构。

入口点runserver.py：

from os import environ
from MultiprocessDemo import app

if __name__ == '__main__':
    HOST = environ.get('SERVER_HOST', 'localhost')
    try:
        PORT = int(environ.get('SERVER_PORT', '5100'))
    except ValueError:
        PORT = 5100
    app.run(HOST, PORT,threaded=False)

路由级views.py：

from datetime import datetime
from flask import render_template
from multiprocessing import Process
from MultiprocessDemo import app
from flask_socketio import SocketIO, emit
from MultiprocessDemo.src.MultiprocessClass import MyClass

app.config['SECRET_KEY'] = 'secret!'
#socketio = SocketIO(app)

obj = MyClass()

@app.route('/')
def index():
    """Request To Fire Multiprocess."""
    return render_template(
        'Demo.html'
    )

@app.route('/TriggerMultiprocess', methods=['GET','POST'])
def TriggerMultiprocess():
    print('request to trigger')
    obj .Start()
    return render_template(
        'Demo.html'
    )

创建并执行多进程的MyClass对象

#import queue #Queue as of python 2.x
from multiprocessing import Queue
import threading
import cv2
import os, errno, sys
import logging
import datetime
import time
from multiprocessing import Process

class MyClass(object):

    def __init__(self):
        # ==Where Issue happens ==========================
        self._logger= logging.getLogger('MyClassObj')
        self._logger.setLevel(logging.DEBUG)
        format = logging.Formatter('%(levelname)s - %(asctime)s - (module)s - %(thread)d - %(message)s')
        pass
    def Start(self):
        self.jobs = []
        self._nThread = 3
        for i in range (0, self._nThread):
            thread = Process(target=self.Test, args=('classobj',))
            self.jobs.append(thread)
        # Start the threads (i.e. calculate the random number lists)
        for j in self.jobs:
            j.start()

#bk thread to consume tasks
    def Test(self, name):
        i = 0
        while i < 20:
            print('hello, {}'.format(name))

我发现，如果MyClass不包含python logger的成员，则将无问题地执行多进程，否则，它将引发与我先前遇到的错误相同的错误。 TypeError：无法腌制_thread.RLock对象

但是，对于同一个类，如果我从以下脚本中调用它们，而没有烧瓶，则无论记录器是否是该类成员的一部分，我都不会遇到酸洗问题。

from MultiprocessClass import MyClass

import time

if __name__ == '__main__':

    a = MyClass()
    a.Start()
    print("done")

Python多处理抛出无法腌制_thread.RLock对象

0 个答案: