Question

我有tf.example格式的数据，并试图以预测格式（使用gRPC）向保存的模型发出请求。我无法确定实现此目的的方法调用。

我从众所周知的汽车定价DNN回归模型（https://github.com/tensorflow/models/blob/master/samples/cookbook/regression/dnn_regression.py）开始，该模型已经通过TF Serving docker容器导出并安装了

import grpc
import numpy as np
import tensorflow as tf
from tensorflow_serving.apis import predict_pb2, prediction_service_pb2_grpc

stub = prediction_service_pb2_grpc.PredictionServiceStub(grpc.insecure_channel("localhost:8500"))

tf_ex = tf.train.Example(
    features=tf.train.Features(
        feature={
            'curb-weight': tf.train.Feature(float_list=tf.train.FloatList(value=[5.1])),
            'highway-mpg': tf.train.Feature(float_list=tf.train.FloatList(value=[3.3])),
            'body-style': tf.train.Feature(bytes_list=tf.train.BytesList(value=[b"wagon"])),
            'make': tf.train.Feature(bytes_list=tf.train.BytesList(value=[b"Honda"])),
        }
    )
)

request = predict_pb2.PredictRequest()
request.model_spec.name = "regressor_test"

# Tried this:
request.inputs['inputs'].CopyFrom(tf_ex)

# Also tried this:
request.inputs['inputs'].CopyFrom(tf.contrib.util.make_tensor_proto(tf_ex))

# This doesn't work either:
request.input.example_list.examples.extend(tf_ex)

# If it did work, I would like to inference on it like this:
result = self.stub.Predict(request, 10.0)

谢谢您的建议

Answer 1

我假设您的saveModel有一个serving_input_receiver_fn，以string作为输入并解析为tf.Example。 Using SavedModel with Estimators

def serving_example_input_receiver_fn():
    serialized_tf_example = tf.placeholder(dtype=tf.string)
    receiver_tensors = {'inputs': serialized_tf_example}   
    features = tf.parse_example(serialized_tf_example, YOUR_EXAMPLE_SCHEMA)
    return tf.estimator.export.ServingInputReceiver(features, receiver_tensors)

因此，serving_input_receiver_fn接受一个字符串，因此您必须SerializeToString tf.Example()。此外，serving_input_receiver_fn的工作方式类似于input_fn进行训练，将数据以批处理方式转储到模型中。

代码可能会更改为：

request = predict_pb2.PredictRequest()
request.model_spec.name = "regressor_test"
request.model_spec.signature_name = 'your method signature, check use saved_model_cli'
request.inputs['inputs'].CopyFrom(tf.make_tensor_proto([tf_ex.SerializeToString()], dtype=types_pb2.DT_STRING))

Answer 2

@hakunami的答案对我不起作用。但是当我将最后一行修改为

时

for (size_t i = 0; i < 3; ++i)
    std::cout << tmp[i];

有效。如果“ shape”为“ None”，则生成的张量原型将精确地表示numpy数组。enter link description here

如何将tf.example发送到TensorFlow服务gRPC预测请求中

2 个答案: