保存一个包含通用句子编码器作为嵌入模型的模型

时间:2019-10-10 01:26:59

标签: tensorflow tensorflow2.0 tf.keras tf-hub

我正在尝试保存一个模型,该模型使用tf-hub中的USE作为其嵌入层,并在上面堆叠了一些FFN。该模型似乎运行良好,但是我在保存和加载模型时遇到了问题。

disable_eager_execution()

embed = hub.Module(module_url)

def UniversalEmbedding(x):
    return embed(tf.squeeze(tf.cast(x, tf.string)))

input_text = Input(shape=[], dtype=tf.string)
response_text = Input(shape=[], dtype=tf.string)

text_embedding = Lambda(UniversalEmbedding, output_shape=(512, ))(input_text)

response_embedding = Lambda(UniversalEmbedding, output_shape=(512, ))(response_text)

response_embedding = Dense(512, activation='relu')(response_embedding)
response_embedding = Dense(512, activation='relu')(response_embedding)

score = Dot(axes=1, normalize=True)([text_embedding, response_embedding])

pred = Dense(2, activation='softmax')(score)

text_encoder = Model(inputs=[input_text], outputs=text_embedding)
response_encoder = Model(inputs=[response_text], outputs=response_embedding)
model = Model(inputs=[input_text, response_text], outputs=pred)

上面的代码是我建立模型的方式(它是使用USE作为编码器的双编码器模型)。

我不得不禁用急切执行,因为USE似乎尚未在急切执行环境中工作。如果没有,并且有解决方法,我也非常感谢您的帮助:)

该模型通过以下代码进行训练和保存:

with tf.compat.v1.Session() as session:
    K.set_session(session)
    session.run(tf.compat.v1.global_variables_initializer())  
    session.run(tf.compat.v1.tables_initializer())
    history = model.fit_generator(generator=train_neg_sample_generator, 
                                  validation_data=val_neg_sample_generator, epochs=20,
                                  callbacks=[checkpointer, earlystopper], verbose=0)

,并且将检查点中的权重(保存在hdf5文件中)加载到上面的代码中定义的模型时,该模型将正确加载。因此,下面的代码可以正常工作,只是因为上面已经定义了架构“模型”。

with tf.compat.v1.Session() as session:
    K.set_session(session)
    session.run(tf.compat.v1.global_variables_initializer())
    session.run(tf.compat.v1.tables_initializer())
    model.load_weights('./saved_models/weights.03-0.29.hdf5') 
    tf.keras.models.save_model(model, 'test_model2.hdf5') 
    predicts = model.predict([["how are you?", "how are you?", 'hi', 'my two favorites in one pic!'], ["i'm fine", "what the heck", 'hi', 'same!']])
    print(predicts)
    print(np.argmax(predicts, axis=1))

然后我尝试了2件事。首先,我尝试将架构保存为json格式,加载模型架构,然后加载权重,但是它不起作用。然后,我尝试通过keras.models.save_model保存整个模型,但是它也不起作用。

在两种情况下,他们都返回了 AttributeError:模块'tensorflow'没有属性'placeholder'

如何保存/加载整个模型(如果不是一次加载单独的架构/重量也可以)?

这是整个错误日志

AttributeError                            Traceback (most recent call last)
<ipython-input-31-47468f2533ad> in <module>()
      1 from keras.models import load_model
      2 
----> 3 model2 = load_model('testest.h5')

13 frames
/usr/local/lib/python3.6/dist-packages/keras/engine/saving.py in load_wrapper(*args, **kwargs)
    456                 os.remove(tmp_filepath)
    457             return res
--> 458         return load_function(*args, **kwargs)
    459 
    460     return load_wrapper

/usr/local/lib/python3.6/dist-packages/keras/engine/saving.py in load_model(filepath, custom_objects, compile)
    548     if H5Dict.is_supported_type(filepath):
    549         with H5Dict(filepath, mode='r') as h5dict:
--> 550             model = _deserialize_model(h5dict, custom_objects, compile)
    551     elif hasattr(filepath, 'write') and callable(filepath.write):
    552         def load_function(h5file):

/usr/local/lib/python3.6/dist-packages/keras/engine/saving.py in _deserialize_model(h5dict, custom_objects, compile)
    241         raise ValueError('No model found in config.')
    242     model_config = json.loads(model_config.decode('utf-8'))
--> 243     model = model_from_config(model_config, custom_objects=custom_objects)
    244     model_weights_group = h5dict['model_weights']
    245 

/usr/local/lib/python3.6/dist-packages/keras/engine/saving.py in model_from_config(config, custom_objects)
    591                         '`Sequential.from_config(config)`?')
    592     from ..layers import deserialize
--> 593     return deserialize(config, custom_objects=custom_objects)
    594 
    595 

/usr/local/lib/python3.6/dist-packages/keras/layers/__init__.py in deserialize(config, custom_objects)
    166                                     module_objects=globs,
    167                                     custom_objects=custom_objects,
--> 168                                     printable_module_name='layer')

/usr/local/lib/python3.6/dist-packages/keras/utils/generic_utils.py in deserialize_keras_object(identifier, module_objects, custom_objects, printable_module_name)
    145                     config['config'],
    146                     custom_objects=dict(list(_GLOBAL_CUSTOM_OBJECTS.items()) +
--> 147                                         list(custom_objects.items())))
    148             with CustomObjectScope(custom_objects):
    149                 return cls.from_config(config['config'])

/usr/local/lib/python3.6/dist-packages/keras/engine/network.py in from_config(cls, config, custom_objects)
   1041         # First, we create all layers and enqueue nodes to be processed
   1042         for layer_data in config['layers']:
-> 1043             process_layer(layer_data)
   1044 
   1045         # Then we process nodes in order of layer depth.

/usr/local/lib/python3.6/dist-packages/keras/engine/network.py in process_layer(layer_data)
   1027 
   1028             layer = deserialize_layer(layer_data,
-> 1029                                       custom_objects=custom_objects)
   1030             created_layers[layer_name] = layer
   1031 

/usr/local/lib/python3.6/dist-packages/keras/layers/__init__.py in deserialize(config, custom_objects)
    166                                     module_objects=globs,
    167                                     custom_objects=custom_objects,
--> 168                                     printable_module_name='layer')

/usr/local/lib/python3.6/dist-packages/keras/utils/generic_utils.py in deserialize_keras_object(identifier, module_objects, custom_objects, printable_module_name)
    147                                         list(custom_objects.items())))
    148             with CustomObjectScope(custom_objects):
--> 149                 return cls.from_config(config['config'])
    150         else:
    151             # Then `cls` may be a function returning a class.

/usr/local/lib/python3.6/dist-packages/keras/engine/base_layer.py in from_config(cls, config)
   1101             A layer instance.
   1102         """
-> 1103         return cls(**config)
   1104 
   1105     def count_params(self):

/usr/local/lib/python3.6/dist-packages/keras/legacy/interfaces.py in wrapper(*args, **kwargs)
     89                 warnings.warn('Update your `' + object_name + '` call to the ' +
     90                               'Keras 2 API: ' + signature, stacklevel=2)
---> 91             return func(*args, **kwargs)
     92         wrapper._original_function = func
     93         return wrapper

/usr/local/lib/python3.6/dist-packages/keras/engine/input_layer.py in __init__(self, input_shape, batch_size, batch_input_shape, dtype, input_tensor, sparse, name)
     85                                          dtype=dtype,
     86                                          sparse=self.sparse,
---> 87                                          name=self.name)
     88         else:
     89             self.is_placeholder = False

/usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py in placeholder(shape, ndim, dtype, sparse, name)
    539         x = tf.sparse_placeholder(dtype, shape=shape, name=name)
    540     else:
--> 541         x = tf.placeholder(dtype, shape=shape, name=name)
    542     x._keras_shape = shape
    543     x._uses_learning_phase = False

AttributeError: module 'tensorflow' has no attribute 'placeholder'

2 个答案:

答案 0 :(得分:0)

提供整个错误日志,而不仅仅是其中的一部分。

如果由于保存而确实出错,那么model.save('model.h5')怎么办? 不使用tf.keras.models中的模块,而是使用Model类本身的方法。

但是为什么要使用这些代码?

with tf.compat.v1.Session() as session:
    K.set_session(session)
    session.run(tf.compat.v1.global_variables_initializer())
    session.run(tf.compat.v1.tables_initializer())

我相信您可以立即拨打电话model.fit,而您的tf是version2吗?为什么打compat.v1

Tensorflow 2没有Placeholder,所以我认为可能与此有关。

答案 1 :(得分:0)

与tensorflow版本1.15正常工作 期待tf-hub与tensorflow 2.0和keras完全兼容...