Question

为什么TensorFlow服务无法使用简单的LSTM keras层提供服务，而使用saved_model_cli run成功运行了呢？而我该如何解决？

TensorFlow 2.0版Alpha

$  pip install tensorflow==2.0.0-alpha0

SavedModel复制：

import tensorflow as tf
from tensorflow.keras.models import Model
from tensorflow.keras.layers import Input, LSTM

img_width= 10
img_height= 10
def build_model():
    input_img = Input(shape=(img_width*img_height, 1), name='input_data', dtype='float32')
    x = LSTM(12, return_sequences=False,  name='lstm_1')(input_img)
    model = Model(input_img, x)
    return model

def save():
    model = build_model()
    tf.saved_model.save(model, "./test_keras_serving/1/")

if __name__ == '__main__':
    save()

TensorFlow服务安装：

$ docker pull tensorflow/serving

$ docker images -a
REPOSITORY                 TAG                 IMAGE ID            CREATED             SIZE
tensorflow/serving         latest              38bee21b2ca0        2 months ago        229MB

提供SavedModel

$ docker run -p 8501:8501 --mount type=bind,source=/the/path/of/dir/test_keras_serving,target=/models/my_model -e MODEL_NAME=my_model -t tensorflow/serving

使用TensorFlow服务预测python代码。

import json
import requests
import numpy as np

def pred():
    inp_value = np.zeros((2,100,1))
    _url = 'http://localhost:8501/v1/models/my_model:predict'
    headers = {"cache-control": "no-cache", "content-type": "application/json"}
    data = json.dumps({"signature_name": "serving_default","instances": inp_value.tolist()})

    json_response = requests.post(url=_url, data=data, headers=headers)
    print(json_response)

if __name__ == '__main__':
    pred()

在客户端中，结果是

<Response [400]>

代替[200]。

服务器显示：

2019-05-12 13:21:49.370594: W external/org_tensorflow/tensorflow/core/framework/op_kernel.cc:1401] OP_REQUIRES failed at partitioned_function_ops.cc:118 : Invalid argument: Expected input[1] == 'TensorArrayV2Stack/TensorListStack/element_shape:output:0' to be a control input.
        In {{node TensorArrayV2Stack/TensorListStack}}

但是，SavedModel在saved_model_cli上可以正常工作：

$ saved_model_cli run --dir ./test_keras_serving/1 --tag_set serve --signature_def serving_default --input_exprs input_data=np.zeros((2,100,1))

输出：

Result for output key lstm_1:
[[0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]
 [0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]]

如何使用saved_model_cli获得与TensorFlow Serving相同的结果？

Answer 1

使用 docker pull tensorflow / serving：nightly

解决了此问题

TensorFlow服务未能提供TensorFlow 2.0 keras.layers.LSTM

1 个答案: