Question

我已经在Keras（按照本文撰写的Keras和TF的最新版本）中训练了分类模型，该模型的输入和输出与CIFAR10类似。为了提供此模型，我使用以下代码将其导出到分类模型（请参见类型）：

def keras_model_to_tf_serve(saved_keras_model,
                        local_version_dir,
                        type='classification',
                        save_model_version=1):

sess = tf.Session()
K.set_session(sess)
K.set_learning_phase(0)

old_model = load_model(saved_keras_model)
config = old_model.get_config()
weights = old_model.get_weights()

new_model = Sequential.from_config(config)
new_model.set_weights(weights)

classification_inputs = utils.build_tensor_info(new_model.input)
classification_outputs_classes = utils.build_tensor_info(new_model.output)

# The classification signature
classification_signature = tf.saved_model.signature_def_utils.build_signature_def(
    inputs={signature_constants.CLASSIFY_INPUTS: classification_inputs},
    outputs={
        signature_constants.CLASSIFY_OUTPUT_CLASSES:
            classification_outputs_classes
    },
    method_name=signature_constants.CLASSIFY_METHOD_NAME)
#print(classification_signature)
# The prediction signature
tensor_info_x = utils.build_tensor_info(new_model.input)
tensor_info_y = utils.build_tensor_info(new_model.output)
prediction_signature = tf.saved_model.signature_def_utils.build_signature_def(
    inputs={'inputs': tensor_info_x},
    outputs={'outputs': tensor_info_y},
    method_name=signature_constants.PREDICT_METHOD_NAME)
legacy_init_op = tf.group(tf.tables_initializer(), name='legacy_init_op')

print(prediction_signature)
save_model_dir = os.path.join(local_version_dir,str(save_model_version))
if os.path.exists(save_model_dir) and os.path.isdir(save_model_dir):
    shutil.rmtree(save_model_dir)

builder = saved_model_builder.SavedModelBuilder(save_model_dir)
with K.get_session() as sess:
    if type == 'classification':
        builder.add_meta_graph_and_variables(
            sess, [tag_constants.SERVING],
            signature_def_map={
                signature_constants.DEFAULT_SERVING_SIGNATURE_DEF_KEY:
                    classification_signature,
            },
            clear_devices=True, legacy_init_op=legacy_init_op)
    elif type == 'prediction':
        builder.add_meta_graph_and_variables(
            sess, [tag_constants.SERVING],
            signature_def_map={
                # Uncomment the first two lines below and comment out the subsequent four to reset.
                # signature_constants.DEFAULT_SERVING_SIGNATURE_DEF_KEY:
                #    classification_signature,
                'predict_results':
                    prediction_signature,
            },
            clear_devices=True, legacy_init_op=legacy_init_op)
    else:
        builder.add_meta_graph_and_variables(
            sess, [tag_constants.SERVING],
            signature_def_map={
                # Uncomment the first two lines below and comment out the subsequent four to reset.
                # signature_constants.DEFAULT_SERVING_SIGNATURE_DEF_KEY:
                #    classification_signature,
                'predict_results':
                    prediction_signature,
                signature_constants.DEFAULT_SERVING_SIGNATURE_DEF_KEY:
                    classification_signature
            },
            clear_devices=True, legacy_init_op=legacy_init_op)
    builder.save()

这可以很好地导出，并使用saved_model_cli获得以下输出：

saved_model_cli show --dir /develop/1/ --tag_set serve -- 
signature_def serving_default
The given SavedModel SignatureDef contains the following input(s):
  inputs['inputs'] tensor_info:
     dtype: DT_FLOAT
     shape: (-1, 32, 32, 3)
     name: conv2d_1_input_1:0
The given SavedModel SignatureDef contains the following output(s):
  outputs['classes'] tensor_info:
     dtype: DT_FLOAT
     shape: (-1, 10)
     name: activation_6_1/Softmax:0
Method name is: tensorflow/serving/classify

因此，该模型期望获得形状为（-1,32,32,3）的DT_Float。由于这是一个分类模型（由于某种原因，它与预测模型在使用上非常不同），因此我使用了@sdcbr代码（TensorFlow Serving: Pass image to classifier）并做了一些细微的修改：

import tensorflow as tf
import numpy as np
from tensorflow_serving.apis import classification_pb2, input_pb2
from grpc.beta import implementations
from tensorflow_serving.apis import prediction_service_pb2

image = np.random.rand(32,32,3)

def _float_feature(value):
    return tf.train.Feature(float_list=tf.train.FloatList(value=value))

request = classification_pb2.ClassificationRequest()
request.model_spec.name = 'model'
request.model_spec.signature_name = signature_constants.DEFAULT_SERVING_SIGNATURE_DEF_KEY

image = image.flatten().tolist()
image = [float(x) for x in image]
example = tf.train.Example(features=tf.train.Features(feature={'image': _float_feature(image)}))

inp = input_pb2.Input()
inp.example_list.examples.extend([example])

request.input.CopyFrom(inp)

channel = implementations.insecure_channel('localhost', 5005)
stub = prediction_service_pb2.beta_create_PredictionService_stub(channel)
response = stub.Classify(request, 10.0)

其中TF-Serve在我的计算机上的端口上本地运行，并且在启动时被指定为spec_name。据我所知，这应该可行，但是当我运行它时，会出现以下错误（为简便起见，在此简称）：

grpc._channel._Rendezvous: <_Rendezvous of RPC that terminated with:
   status = StatusCode.INVALID_ARGUMENT
   details = "Expects arg[0] to be float but string is provided"
   debug_error_string = " 
      {"created":"@1533046733.211573219","description":"Error received from peer","file":"src/core/lib/surface/call.cc","file_line":1083,"grpc_message":"Expects arg[0] to be float but string is provided","grpc_status":3}"

有什么想法吗？经过数小时的搜索，这是唯一的方法，我可以将任何种类的图像数据放入分类请求中。

Answer 1

尝试使用prediction_service_pb2_grpc.PredictionServiceStub(channel)代替prediction_service_pb2.beta_create_PredictionService_stub(channel)。显然，这是最近从beta版移走的。您可以参考this示例。

TF Server提供的导出的Keras分类模型提供：期望arg [0]为浮点型，但提供了字符串

1 个答案: