Question

我想将完整的猫和狗数据集下载到我的PC上，并具有实际的jpg文件。我可以直接从例如下载Microsoft，但是我想使用tfds.load数据集函数。

当我尝试时：

(raw_train, raw_validation, raw_test), metadata = tfds.load(
    'cats_vs_dogs',
    split=['train[:80%]', 'train[80%:90%]', 'train[90%:]'],
    with_info=True,
    as_supervised=True,
    data_dir=r'D:\TFProjects\catsdogscompl')

它确实下载了一些内容，但是这些是TFRECORD文件，而不是jpg。如何获得实际的jpg？

Answer 1

TFDS实际上返回tf.data.Dataset。如主页上所述

所有数据集都以tf.data.Datasets形式公开，从而启用了易于使用和高性能的输入管道。

但是您实际上可以加载数据集并将其手动保存在jpeg中。

import tensorflow as tf
import tensorflow_datasets as tfds
from uuid import uuid1
import os
import warnings

(raw_train, raw_validation, raw_test), metadata = tfds.load(
    'cats_vs_dogs',
    split=['train[:80%]', 'train[80%:90%]', 'train[90%:]'],
    with_info=True,
    data_dir=r'D:\TFProjects\catsdogscompl')

def save_dataset_as_jpegs(dataset, path,):
    """

    saves every image to the 'path' using random name + target

    :param dataset: dataset you want to save
    :param path: where you want to store it
    :param metadata: metadata from dataset. required to get class names.
    :return: Nothing. Just saves the dataset as jpegs.
    """

    for obj in dataset:
        im, name = obj['image'], obj['image/filename']
        serialized_im = tf.image.encode_jpeg(im)

        path_and_name = os.path.join(path, name.numpy().decode())
        tf.io.write_file(path_and_name, serialized_im)

save_dataset_as_jpegs(raw_train, 'jpegs_train/')
save_dataset_as_jpegs(raw_validation, 'jpegs_validation/')
save_dataset_as_jpegs(raw_test, 'jpegs_test/')

此代码将raw_test数据集保存到jpegs_test文件夹中。

将tfds.load数据集以jpg格式下载到本地磁盘

1 个答案: