Question

我正在尝试使用保存的tensorflow模型在tensorflow服务中部署我的tensorflow模型。我的tf模型的输入是一个字符串值，并且我已经如下定义了我的签名

        prediction_signature = (
            tf.saved_model.signature_def_utils.build_signature_def(
                inputs={'input_path': tensor_info_input},
                outputs={'output_prediction': tensor_info_output},
                method_name=tf.saved_model.signature_constants.PREDICT_METHOD_NAME))

        builder.add_meta_graph_and_variables(
            sess, [tf.saved_model.tag_constants.SERVING],
            signature_def_map={
                'predict_images':
                    prediction_signature,
            })

我的目标是从签名定义中指定的路径读取图像。如何从输入定义中获取input_path并将张量转换为实际的字符串值以读取路径

Answer 1

我认为您需要在模型文件中添加一些“读取操作”。

赞：

def inference(image_path_string):
    # read image from a image path string
    image_file = tf.read_file(image_path_string)
    image_raw = tf.cond(tf.image.is_jpeg(image_file),
                        lambda: tf.image.decode_jpeg(image_file, channels= 3 if rgb, else 1),
                        lambda: tf.image.decode_bmp(image_file))
    # image preprocessing code
    ...
    ...
    # your prediction model code
    ...
    ...

Answer 2

您可以将下面提到的代码放在Python文件Client.py中。为了进行推断，您可以在终端机

中运行以下提到的命令

python Client.py --image abc/0.png --model mnist --signature_name predict

这将采用图像的路径“ abc / 0.png”，并将其转换为数值，并将运行推断。

Client.py的代码如下所述。以下代码适用于MNIST图片。您可以相应地调整图像的形状：

from __future__ import print_function

import argparse
import time
import numpy as np
from scipy.misc import imread

import grpc
from tensorflow.contrib.util import make_tensor_proto

from tensorflow_serving.apis import predict_pb2
from tensorflow_serving.apis import prediction_service_pb2_grpc


def run(host, port, image, model, signature_name):

    channel = grpc.insecure_channel('{host}:{port}'.format(host=host, port=port))
    stub = prediction_service_pb2_grpc.PredictionServiceStub(channel)

    # Read an image
    data = imread(image)
    data = data.astype(np.float32)
    print(data)

    start = time.time()

    # Call classification model to make prediction on the image
    request = predict_pb2.PredictRequest()
    request.model_spec.name = model
    request.model_spec.signature_name = signature_name
    request.inputs['image'].CopyFrom(make_tensor_proto(data, shape=[1, 28, 28, 1]))

    result = stub.Predict(request, 10.0)

    end = time.time()
    time_diff = end - start

    # Reference:
    # How to access nested values
    # https://stackoverflow.com/questions/44785847/how-to-retrieve-float-val-from-a-predictresponse-object
    print(result)
    print('time elapased: {}'.format(time_diff))


if __name__ == '__main__':
    parser = argparse.ArgumentParser()
    parser.add_argument('--host', help='Tensorflow server host name', default='localhost', type=str)
    parser.add_argument('--port', help='Tensorflow server port number', default=8500, type=int)
    parser.add_argument('--image', help='input image', type=str)
    parser.add_argument('--model', help='model name', type=str)
    parser.add_argument('--signature_name', help='Signature name of saved TF model',
                        default='serving_default', type=str)

    args = parser.parse_args()
    run(args.host, args.port, args.image, args.model, args.signature_name)

有关更多信息，您可以参考这篇有关Tensorflow Serving的漂亮文章， https://medium.com/@yuu.ishikawa/serving-pre-modeled-and-custom-tensorflow-estimator-with-tensorflow-serving-12833b4be421

无法从Tensorflow服务客户端请求中获取输入值

2 个答案: