Question

我正在尝试使用grpc提供火花酒分类模型。

我已经训练了模型。我已经编写了使用flask在http服务器上服务的代码。

现在，我正在尝试使用grpc来尝试并了解性能提升。

但是我收到以下错误 -

服务器端 -

ERROR:root:Exception calling application: classify() missing 1 required positional argument: 'context'
Traceback (most recent call last):
  File "/media/dosi/Data/mlframework/venv/grpc/lib/python3.5/site-packages/grpc/_server.py", line 375, in _call_behavior
    return behavior(argument, context), True
TypeError: classify() missing 1 required positional argument: 'context'

客户端 -

Traceback (most recent call last):
  File "client.py", line 20, in <module>
    run()
  File "client.py", line 16, in run
    proline=23.12))
  File "/media/dosi/Data/mlframework/venv/grpc/lib/python3.5/site-packages/grpc/_channel.py", line 507, in __call__
    return _end_unary_response_blocking(state, call, False, deadline)
  File "/media/dosi/Data/mlframework/venv/grpc/lib/python3.5/site-packages/grpc/_channel.py", line 455, in _end_unary_response_blocking
    raise _Rendezvous(state, None, None, deadline)
grpc._channel._Rendezvous: <_Rendezvous of RPC that terminated with (StatusCode.UNKNOWN, Exception calling application: classify() missing 1 required positional argument: 'context')>

我有以下疑问，但我认为这不应该是错误的原因 -

grpc安装在python虚拟环境中。也许，在做spark-submit时，我没有使用那个虚拟环境。我认为这不是问题，因为在这种情况下我不能导入grpc，更不用说启动服务器并运行客户端了。

服务器代码 -

import time, sys,  os
from pyspark import SparkContext, SparkConf
from pyspark.sql import SparkSession, SQLContext
import config
import grpc
import wine_pb2
import wine_pb2_grpc
from concurrent import futures

_ONE_DAY_IN_SECONDS = 60 * 60 * 24

class WineClassifier(wine_pb2_grpc.WineClassifierServicer):

  def classify(self, request, context):
    return wine_pb2.WineClass(request.mg)

def init_spark_context():
    # load spark context
    conf = SparkConf().setAppName("wine-app-server")
    # IMPORTANT: pass aditional Python modules to each worker
    sc = SparkContext(conf=conf)
    spark = SparkSession \
    .builder \
    .appName("Python Spark SQL basic example") \
    .config("spark.some.config.option", "some-value") \
    .getOrCreate()

    return sc, spark


def run_server():
    server = grpc.server(futures.ThreadPoolExecutor(max_workers=10))
    wine_pb2_grpc.add_WineClassifierServicer_to_server(WineClassifier, server)
    server.add_insecure_port('[::]:50051')
    server.start()
    try:
        while True:
            time.sleep(_ONE_DAY_IN_SECONDS)
    except KeyboardInterrupt:
        server.stop(0)

if __name__ == "__main__":
    # Init spark context and load libraries
    sc, spark = init_spark_context()

    # start web server
    run_server()

原型文件 -

syntax="proto3";

service WineClassifier {
  rpc classify (WineAttributes) returns (WineClass) {}
}

message WineAttributes {
    double alcohol = 1;
    double malic_acid = 2;
    double ash = 3;
    double alcalinity = 4;
    int32 mg = 5;
    double phenols = 6;
    double flavanoids = 7;
    double flavaniods = 8;
    double nf_phenols = 9;
    double proanthocyanins = 10;
    double color_intensity = 11;
    double hue = 12;
    double diluted = 13;
    double proline = 14;
}

message WineClass {
  int32 class = 1;
}

知道这里出了什么问题吗？

Answer 1

我认为问题是在wine_pb2_grpc.add_WineClassifierServicer_to_server(WineClassifier, server)中你传递的是WineClassifier类对象，而不是WineClassifier类的实例。尝试将该行更改为wine_pb2_grpc.add_WineClassifierServicer_to_server(WineClassifier(), server)。

无法在pyspark中加载模型并使用grpc进行服务

1 个答案: