Question

我需要减少其尺寸的keras型号。我理解的是，我可以通过将存储在图层中的权重转换为float16或int来减小尺寸。

我尝试使用以下代码转换float16和int。

# Iterate over all the layers of the network
for layer_idx, layer in enumerate(model.layers):

    # If layer has no weights the move to next layer
    if not layer.get_weights():
        continue

    # Get existing weights
    old_weights = layer.get_weights()

    # List to store new weights
    new_weights = []

    # Iterate over weights
    for idx, weight in enumerate(old_weights):
        # Convert weight and append to new list
        new_weights.append(weight.astype(int))
        # print(weight.dtype)

    model.get_layer(name=layer.name).set_weights(new_weights)

对于float16，模型大小没有减少，对于int我使用上面的代码转换了权重，但在加载模型时面临误差。

 File "network_pruning.py", line 24, in <module>
    custom_objects={'angle_error': angle_error})
  File "/home/aditya/miniconda3/envs/python36/lib/python3.6/site-packages/keras/models.py", line 239, in load_model
    model = model_from_config(model_config, custom_objects=custom_objects)
  File "/home/aditya/miniconda3/envs/python36/lib/python3.6/site-packages/keras/models.py", line 313, in model_from_config
    return layer_module.deserialize(config, custom_objects=custom_objects)
  File "/home/aditya/miniconda3/envs/python36/lib/python3.6/site-packages/keras/layers/__init__.py", line 55, in deserialize
    printable_module_name='layer')
  File "/home/aditya/miniconda3/envs/python36/lib/python3.6/site-packages/keras/utils/generic_utils.py", line 139, in deserialize_keras_object
    list(custom_objects.items())))
  File "/home/aditya/miniconda3/envs/python36/lib/python3.6/site-packages/keras/engine/topology.py", line 2490, in from_config
    process_layer(layer_data)
  File "/home/aditya/miniconda3/envs/python36/lib/python3.6/site-packages/keras/engine/topology.py", line 2476, in process_layer
    custom_objects=custom_objects)
  File "/home/aditya/miniconda3/envs/python36/lib/python3.6/site-packages/keras/layers/__init__.py", line 55, in deserialize
    printable_module_name='layer')
  File "/home/aditya/miniconda3/envs/python36/lib/python3.6/site-packages/keras/utils/generic_utils.py", line 141, in deserialize_keras_object
    return cls.from_config(config['config'])
  File "/home/aditya/miniconda3/envs/python36/lib/python3.6/site-packages/keras/engine/topology.py", line 1253, in from_config
    return cls(**config)
  File "/home/aditya/miniconda3/envs/python36/lib/python3.6/site-packages/keras/legacy/interfaces.py", line 87, in wrapper
    return func(*args, **kwargs)
  File "/home/aditya/miniconda3/envs/python36/lib/python3.6/site-packages/keras/engine/topology.py", line 1348, in __init__
    name=self.name)
  File "/home/aditya/miniconda3/envs/python36/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py", line 488, in placeholder
    x = tf.placeholder(dtype, shape=shape, name=name)
  File "/home/aditya/miniconda3/envs/python36/lib/python3.6/site-packages/tensorflow/python/ops/array_ops.py", line 1777, in placeholder
    return gen_array_ops.placeholder(dtype=dtype, shape=shape, name=name)
  File "/home/aditya/miniconda3/envs/python36/lib/python3.6/site-packages/tensorflow/python/ops/gen_array_ops.py", line 4516, in placeholder
    dtype = _execute.make_type(dtype, "dtype")
  File "/home/aditya/miniconda3/envs/python36/lib/python3.6/site-packages/tensorflow/python/eager/execute.py", line 126, in make_type
    (arg_name, repr(v)))
TypeError: Expected DataType for argument 'dtype' not 'int'.

我甚至不确定这是否是缩小模型尺寸的正确方法。任何人都可以告诉我减少模型大小和复杂性的方法也很好。

非常感谢提前!!

Answer 1

我不会使用整数作为权重。我无法想象它们能够足够粒度，以便您获得像梯度下降等解算器所需的小重量变化，最终找到全局最小值（？）。

换句话说，大多数学习率的值如0.0001或0.001，当最终加入或减去时（对学习率进行一系列数学计算），权重不会将权重值更改为任何右侧的权重值由于dtype是int，因此十进制将被删除。

通过将权重转换为float16或int来减小模型大小

1 个答案: