gRPC +线程本地问题

时间:2018-11-26 19:10:30

标签: python-multithreading werkzeug grpc-python

我正在使用python构建grpc服务器,并尝试使用werkzeug Local和LocalProxy处理一些线程本地存储,与flask相似。

我面临的问题是,当我从服务器拦截器中向本地存储一些数据,然后尝试从服务程序中检索数据时,本地为空。真正的问题是,由于某种原因,拦截器在与服务程序不同的Greenlet中运行,因此由于werkzeug.local。存储最终以不同的密钥结尾,因此无法跨请求共享数据。应该属于同一请求的数据。

使用python线程库也会发生同样的情况,看起来拦截器是从主线程运行的,或者是与服务程序运行的线程不同的。有没有解决方法?我希望拦截器在同一线程中运行,从而允许发生这种情况。

# Define a global somewhere
from werkzeug.local import Local
local = Local()

# from an interceptor save something
local.message = "test msg"

# from the service access it
local.service_var = "test"
print local.message  # this throw a AttributeError

# print the content of local
print local.__storage__  # we have 2 entries in the storage, 2 different greenlets, but we are in the same request.

1 个答案:

答案 0 :(得分:1)

拦截器确实在与处理线程不同的服务线程上运行。服务线程负责服务服务者和拦截服务者处理程序。拦截器返回servicer方法处理程序后,服务线程会将其提交到_server.py#L525处的thread_pool

# Take unary unary call as an example.
# The method_handler is the returned object from interceptor.
def _handle_unary_unary(rpc_event, state, method_handler, thread_pool):
    unary_request = _unary_request(rpc_event, state,
                                   method_handler.request_deserializer)
    return thread_pool.submit(_unary_response_in_pool, rpc_event, state,
                              method_handler.unary_unary, unary_request,
                              method_handler.request_deserializer,
                              method_handler.response_serializer)

关于解决方法,我只能想象在初始化期间将存储实例同时传递给拦截器和服务者。之后,该存储可以用作成员变量。

class StorageServerInterceptor(grpc.ServerInterceptor):

    def __init__(self, storage):
        self._storage = storage

    def intercept_service(self, continuation, handler_call_details):
        key = ...
        value = ...
        self._storage.set(key, value)
        ...
        return continuation(handler_call_details)

class Storage(...StorageServicer):

    def __init__(self, storage):
        self._storage = storage

    ...Servicer Handlers...