更新

Question

我正在导出一个savedModel，它将字符串占位符作为输入张量。我注入了一个图形来预处理这个字符串张量，以便它可以传递给模型。但是，我使用py_func在张量上执行我的python字符串操作。

此处input_text是savedModel签名中的输入张量。我创建了另一个默认input_ints的占位符，该占位符已初始化为py_func上执行input_text的结果。我最初将input_text作为一个操作（input_ints =tf.py_func(preprocess, [input_text], tf.int64)）但是tf.nn.dynamic_rnn不接受具有未指定形状的张量。

    # Create the graph object
with tf.name_scope('inputs'):
    input_text = tf.placeholder(tf.string, name="input_text")
    input_ints = tf.placeholder_with_default(
        tf.py_func(preprocess, [input_text], tf.int64), shape=[None, None])

def lstm_cell():
    # Your basic LSTM cell
    lstm = tf.contrib.rnn.BasicLSTMCell(lstm_size, reuse=tf.get_variable_scope().reuse)
    # Add dropout to the cell
    return tf.contrib.rnn.DropoutWrapper(lstm, output_keep_prob=keep_prob)


# def create_rnn():
with tf.name_scope("Embeddings"):
    embedding = tf.Variable(tf.random_uniform((vocab_size, embed_size), -1, 1))
    embed = tf.nn.embedding_lookup(embedding, input_ints)
with tf.name_scope("RNN_layers"):
    cell = tf.contrib.rnn.MultiRNNCell([lstm_cell() for _ in range(lstm_layers)])
initial_state = cell.zero_state(batch_size, tf.float32)
with tf.name_scope("RNN_forward"):
    outputs, final_state = tf.nn.dynamic_rnn(cell, embed, initial_state=initial_state)
with tf.name_scope('predictions'):
    predictions = tf.contrib.layers.fully_connected(outputs[:, -1], 1, activation_fn=tf.sigmoid)

现在使用上面的实现，我可以正确导出模型但是在恢复模型时，我收到以下错误：

2017-11-23 17:29:14.600184: W tensorflow/core/framework/op_kernel.cc:1192] Unknown: KeyError: 'pyfunc_0'
Traceback (most recent call last):
  File "/Users/sakibarrahman/anaconda/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1327, in _do_call
    return fn(*args)
  File "/Users/sakibarrahman/anaconda/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1306, in _run_fn
    status, run_metadata)
  File "/Users/sakibarrahman/anaconda/lib/python3.6/contextlib.py", line 89, in __exit__
    next(self.gen)
  File "/Users/sakibarrahman/anaconda/lib/python3.6/site-packages/tensorflow/python/framework/errors_impl.py", line 466, in raise_exception_on_not_ok_status
    pywrap_tensorflow.TF_GetCode(status))
tensorflow.python.framework.errors_impl.UnknownError: KeyError: 'pyfunc_0'
     [[Node: inputs/PyFunc = PyFunc[Tin=[DT_STRING], Tout=[DT_INT64], token="pyfunc_0", _device="/job:localhost/replica:0/task:0/cpu:0"](_arg_inputs/input_text_0_0)]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "neural_load_model.py", line 85, in <module>
    result = sess.run(output_tensor, {input_tensor: "Charter Communications, Inc. (CHTR) Stock Rating Reaffirmed by Goldman Sachs Group, Inc. (The)"})
  File "/Users/sakibarrahman/anaconda/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 895, in run
    run_metadata_ptr)
  File "/Users/sakibarrahman/anaconda/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1124, in _run
    feed_dict_tensor, options, run_metadata)
  File "/Users/sakibarrahman/anaconda/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1321, in _do_run
    options, run_metadata)
  File "/Users/sakibarrahman/anaconda/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1340, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.UnknownError: KeyError: 'pyfunc_0'
     [[Node: inputs/PyFunc = PyFunc[Tin=[DT_STRING], Tout=[DT_INT64], token="pyfunc_0", _device="/job:localhost/replica:0/task:0/cpu:0"](_arg_inputs/input_text_0_0)]]

Caused by op 'inputs/PyFunc', defined at:
  File "neural_load_model.py", line 74, in <module>
    model = tf.saved_model.loader.load(sess, [tf.saved_model.tag_constants.SERVING], import_path)
  File "/Users/sakibarrahman/anaconda/lib/python3.6/site-packages/tensorflow/python/saved_model/loader_impl.py", line 216, in load
    saver = tf_saver.import_meta_graph(meta_graph_def_to_load, **saver_kwargs)
  File "/Users/sakibarrahman/anaconda/lib/python3.6/site-packages/tensorflow/python/training/saver.py", line 1698, in import_meta_graph
    **kwargs)
  File "/Users/sakibarrahman/anaconda/lib/python3.6/site-packages/tensorflow/python/framework/meta_graph.py", line 656, in import_scoped_meta_graph
    producer_op_list=producer_op_list)
  File "/Users/sakibarrahman/anaconda/lib/python3.6/site-packages/tensorflow/python/framework/importer.py", line 313, in import_graph_def
    op_def=op_def)
  File "/Users/sakibarrahman/anaconda/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 2630, in create_op
    original_op=self._default_original_op, op_def=op_def)
  File "/Users/sakibarrahman/anaconda/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1204, in __init__
    self._traceback = self._graph._extract_stack()  # pylint: disable=protected-access

UnknownError (see above for traceback): KeyError: 'pyfunc_0'
     [[Node: inputs/PyFunc = PyFunc[Tin=[DT_STRING], Tout=[DT_INT64], token="pyfunc_0", _device="/job:localhost/replica:0/task:0/cpu:0"](_arg_inputs/input_text_0_0)]]

我已经看过在Github上发布的这个issue，但我不确定如何实现这个。另外，我只是加载模型并传入一个字符串进行输入而不使用'freeze_graph'。

我保存模型的代码：

saver = tf.train.Saver()

#Define new functions
def preprocess(text):
.
.
.
tf.reset_default_graph()
.
.
.
#Define new placeholder that was not in the original model graph
#Define new placeholder with default value initialized with py_func that was not in the original model graph
with tf.name_scope('inputs'):
    input_text = tf.placeholder(tf.string, name="input_text")
    input_ints = tf.placeholder_with_default(
        tf.py_func(preprocess, [input_text], tf.int64), shape=[None, None])
.
.
.
#Define placeholders and ops that I need and were in the original graph



saver = tf.train.Saver()
#Serving the model
with tf.Session() as sess:


#Restore from old checkpoint
saver.restore(sess, import_path)

print ('Exporting trained model to %s'%(export_path))

builder = saved_model_builder.SavedModelBuilder(export_path)

original_assets_directory = export_path + '/assets'
original_assets_filename = "vocabulary.pickle"
original_assets_filepath = write_vocab(original_assets_directory,
                                         original_assets_filename)

# Set up the assets collection.
assets_filepath = tf.constant(original_assets_filepath)
tf.add_to_collection(tf.GraphKeys.ASSET_FILEPATHS, assets_filepath)
filename_tensor = tf.Variable(
    original_assets_filename,
    name="vocab_tensor",
    trainable=False,
    collections=[])
assign_filename_op = filename_tensor.assign(original_assets_filename)


# Build the signature_def_map.
classification_inputs = utils.build_tensor_info(input_text)
classification_outputs_classes = utils.build_tensor_info(predictions)
classification_signature = signature_def_utils.build_signature_def(
    inputs={signature_constants.CLASSIFY_INPUTS: classification_inputs},
    outputs={
      signature_constants.CLASSIFY_OUTPUT_CLASSES:
          classification_outputs_classes,
    },
  method_name=signature_constants.CLASSIFY_METHOD_NAME)

legacy_init_op = tf.group(
    tf.tables_initializer(), name='legacy_init_op')
#add the sigs to the servable
builder.add_meta_graph_and_variables(
    sess, [tag_constants.SERVING],
    signature_def_map={
        signature_constants.DEFAULT_SERVING_SIGNATURE_DEF_KEY:
            classification_signature
    },
    assets_collection=tf.get_collection(tf.GraphKeys.ASSET_FILEPATHS),
    legacy_init_op=tf.group(assign_filename_op))
print ("added meta graph and variables")

builder.save()
print("model saved")

我加载模型的代码。未定义函数或占位符会导致'pyfunc_0'错误：

#Define preprocess function
def preprocess(text_bin):

#Define new placeholders
with tf.name_scope('inputs'):
    input_text = tf.placeholder(tf.string, name="input_text")
    input_ints = tf.placeholder_with_default(
        tf.py_func(preprocess, [input_text], tf.int64), shape=[None, None])

with tf.Session(graph=tf.Graph()) as sess:
    # restore save model
    model = tf.saved_model.loader.load(sess, [tf.saved_model.tag_constants.SERVING], import_path)
    print("model restored")
    loaded_graph = tf.get_default_graph()

    # get necessary tensors by name
    input_tensor_name = model.signature_def[signature_constants.DEFAULT_SERVING_SIGNATURE_DEF_KEY].inputs[signature_constants.CLASSIFY_INPUTS].name
    input_tensor = loaded_graph.get_tensor_by_name(input_tensor_name)
    output_tensor_name = model.signature_def[signature_constants.DEFAULT_SERVING_SIGNATURE_DEF_KEY].outputs[signature_constants.CLASSIFY_OUTPUT_CLASSES].name
    output_tensor = loaded_graph.get_tensor_by_name(output_tensor_name)

    result = sess.run(output_tensor, {input_tensor: "Some String"})
    print (result)

更新

加载savedModel时定义函数和占位符似乎有效。但是，我不知道为什么在使用构建器保存模型之前没有将它们添加到图形中

Answer 1

您的模型看起来像是自定义图层。您可以按照模型代码找到它。因此，您可以在图形加载之前定义该函数。此外，函数定义顺序很重要。

Answer 2

正在使用的预处理函数实际上并不是图的一部分，因此py_func（）在加载savedModel时不知道要使用哪个函数。目前在Tensorflow服务流程中没有简单的方法来进行预处理。必须在使用模型之前在客户端完成，或者可能必须创建自定义op以使其可以成为模型的一部分。另一种选择可能是创建自定义服务。

未知：KeyError：'pyfunc_0'

更新

2 个答案: