Question

我有一个保存的模型，当我在主线程上运行它时可以生成一个预测。当我尝试在线程中使用相同的函数时，出现以下错误。

请注意，我的代码摘要将在错误后粘贴。

Exception in thread Thread-6:
Traceback (most recent call last):
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\threading.py", line 916, in _bootstrap_inner
    self.run()
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\threading.py", line 864, in run
    self._target(*self._args, **self._kwargs)
  File "D:\PiChess\Core\ComModule.py", line 49, in threader
    b = Globals.moduleManager.GetPrediction(predictionSet)
  File "D:\PiChess\Core\ManagementModel.py", line 56, in GetPrediction
    return self.Model.predict(data, steps=10)
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\site-packages\tensorflow\python\keras\engine\training.py", line 1878, in predict
    self, x, batch_size=batch_size, verbose=verbose, steps=steps)
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\site-packages\tensorflow\python\keras\engine\training_arrays.py", line 295, in predict_loop
    batch_outs = f(ins)
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\site-packages\tensorflow\python\keras\backend.py", line 2983, in __call__
    self._make_callable(feed_arrays, feed_symbols, symbol_vals, session)
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\site-packages\tensorflow\python\keras\backend.py", line 2928, in _make_callable
    callable_fn = session._make_callable_from_options(callable_opts)
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\site-packages\tensorflow\python\client\session.py", line 1471, in _make_callable_from_options
    return BaseSession._Callable(self, callable_options)
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\site-packages\tensorflow\python\client\session.py", line 1425, in __init__
    session._session, options_ptr, status)
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\site-packages\tensorflow\python\framework\errors_impl.py", line 528, in __exit__
    c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.InvalidArgumentError: Requested tensor connection from unknown node: "dense_input:0".

Exception ignored in: <bound method BaseSession._Callable.__del__ of <tensorflow.python.client.session.BaseSession._Callable object at 0x000001E7749C5438>>
Traceback (most recent call last):
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\site-packages\tensorflow\python\client\session.py", line 1455, in __del__
    self._session._session, self._handle, status)
  File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\site-packages\tensorflow\python\framework\errors_impl.py", line 528, in __exit__
    c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.InvalidArgumentError: No such callable handle: 2093487919952

当我开始运行代码时，模型已加载到内存中。为此，我创建了一个全局对象，该对象通过以下方式加载模型。

self.ModelName = ModelName
self.Model = load_model("Models/" + self.ModelName + '.mdl')
self.Model._make_predict_function()

要使用预测函数，它必须在线程中运行

predict = self.Model.predict(data, steps=10)

我知道所提供的数据正确无误，因为我只是从代码中切出了线程方面，然后将代码分成一个线程就可以得到结果。

我将大部分代码分割成多个文件，因此我将其剪切掉了。当我尝试将其粘贴到此处时，堆栈溢出告诉我我必须编写许多代码。

如果需要的话，请告诉我哪一部分代码最适合作为编辑发布。

Answer 1

您可能需要同时存储用于加载模型的图形（有时是会话），并将它们传递到单独的线程。

如果您使用的是Keras：

import keras.backend as K

# Load model
self.sess = tf.Session()
K.set_session(sess)
self.ModelName = ModelName
self.Model = load_model("Models/" + self.ModelName + '.mdl')
self.Model._make_predict_function()
self.graph = tf.get_default_graph()

# Predict
K.set_session(sess)
with self.sess.as_default():
    with self.graph.as_default():
        predict = self.Model.predict(data, steps=10)

如果您正在使用TensorFlow：

# Load model
self.sess = tf.Session()
self.sess.run(tf.global_variables_initializer())
self.Model.load(self.sess, "Models/" + self.ModelName + '.mdl')
self.graph = tf.get_default_graph()

# Predict
with self.graph.as_default():
    predict = self.Model.predict(self.sess, ...)

一个相关的答案：The problem you are running into is because the tensorflow model is not loaded and used in the same thread.

Answer 2

不确定你们是否找到了解决方案，我只是遇到了类似的问题，并且已经解决了，请在这里分享。

简而言之，修复非常简单：在调用了K.clear_session()之后，即使在同一过程中，您也必须重新加载模型！

我之前有问题的代码：

model = create_input_selection_model(config.input_shape)

for i in range(batches):
  # Do a big batch prediction (100K or more depends on ur physical memory size).
  sub_preds = model.predict_generator()
  # For solving the memory problem, call 'K.clear_session()' to release GPU memory.
  K.clear_session()

上面的代码将遇到OP发布的问题。

修复非常简单：在每个for循环迭代中重新加载模型：

for i in range(batches):

  model = create_input_selection_model(config.input_shape) # <- reload model here.

  # Do a big batch prediction (100K or more depends on ur physical memory size).
  sub_preds = model.predict_generator()
  # For solving the memory problem, call 'K.clear_session()' to release GPU memory.
  K.clear_session()

在单独的线程中时模型的预测功能出现问题

2 个答案: