Question

我目前正在尝试在keras中使用许多预先训练的ImageNet网络来从图像中提取特征。为此，我要删除网络的顶层，根据每个网络要求对输入进行预处理，然后将输出保存在hdf5文件中。我使用了完全相同的方法和代码（仅切换了网络）使用了其他几个经过预训练的网络，它似乎运行得很好。但是，我正在努力工作的网络是“ InceptionResNetV2”。我不相信我的网络有任何问题，只有保存方面的问题-我附加了略微简化的代码版本。在特征提取器中更改模型并在预处理中更改模型意味着它可以完美运行-对于vgg16，vgg19，resnet，inception等-都很好。

db = h5py.File(hdf5_path, mode="w")

featuresDB = db.create_dataset("features", shape=features_shape, dtype="float")

images = [cv2.imread(path, 1) for path in image_paths[start:end]]

images = inception_resnet_v2.preprocess_input(images)

features = feature_extractor.extract(images)

featuresDB[start:end] = features

但是，这会产生以下错误。我试图将进入featuresDB的数据的dtype更改为int，但这没有任何效果。任何建议表示赞赏！

File "h5py/utils.pyx", line 101, in h5py.utils.convert_tuple
TypeError: an integer is required

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "Extract_Features.py", line 111, in <module>
    extract_features(image_paths, hdf5_path=args["features_db"], 
feature_extractor=feature_extractor)

File "Extract_Features.py", line 83, in extract_features
    featuresDB = db.create_dataset("features", shape=features_shape, dtype="float")

File "/home/will/tensorflow/lib/python3.5/site-packages/h5py/_hl/group.py", line 
106, in create_dataset
    dsid = dataset.make_new_dset(self, shape, dtype, data, **kwds)

File "/home/will/tensorflow/lib/python3.5/site-packages/h5py/_hl/dataset.py", line 137, in make_new_dset
    sid = h5s.create_simple(shape, maxshape)

File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper

File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper

File "h5py/h5s.pyx", line 95, in h5py.h5s.create_simple

File "h5py/utils.pyx", line 103, in h5py.utils.convert_tuple
TypeError: Can't convert element 1 (None) to hsize_t

Answer 1

In [201]: f = h5py.File('test.h5','w')

我可以使用以下表达式重新创建您的错误：

In [203]: ds = f.create_dataset('features', shape=(None,3), dtype=float)
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
h5py/utils.pyx in h5py.utils.convert_tuple()

TypeError: an integer is required

During handling of the above exception, another exception occurred:

TypeError                                 Traceback (most recent call last)
<ipython-input-203-a5733d841c5c> in <module>()
----> 1 ds = f.create_dataset('features', shape=(None,3), dtype=float)

~/.local/lib/python3.6/site-packages/h5py/_hl/group.py in create_dataset(self, name, shape, dtype, data, **kwds)
    104         """
    105         with phil:
--> 106             dsid = dataset.make_new_dset(self, shape, dtype, data, **kwds)
    107             dset = dataset.Dataset(dsid)
    108             if name is not None:

~/.local/lib/python3.6/site-packages/h5py/_hl/dataset.py in make_new_dset(parent, shape, dtype, data, chunks, compression, shuffle, fletcher32, maxshape, compression_opts, fillvalue, scaleoffset, track_times)
    135         sid = h5s.create(h5s.NULL)
    136     else:
--> 137         sid = h5s.create_simple(shape, maxshape)
    138 
    139 

h5py/_objects.pyx in h5py._objects.with_phil.wrapper()

h5py/_objects.pyx in h5py._objects.with_phil.wrapper()

h5py/h5s.pyx in h5py.h5s.create_simple()

h5py/utils.pyx in h5py.utils.convert_tuple()

TypeError: Can't convert element 0 (None) to hsize_t

在None参数中可以使用maxshape，但不能在shape中使用-这适用于可调整大小的数据集：

In [204]: ds = f.create_dataset('features', shape=(10,3), maxshape=(None,3), dty
     ...: pe=float)
In [205]: ds
Out[205]: <HDF5 dataset "features": shape (10, 3), type "<f8">
In [206]: ds.resize((20,3))
In [207]: ds
Out[207]: <HDF5 dataset "features": shape (20, 3), type "<f8">

我还没有使用过keras，但是从其他SO问题和文档来看，None允许使用shape。 None不能为numpy数组形状。

在h5py中保存Inception_Resnet_v2功能

1 个答案: