如何解析tfrecords?

时间:2019-08-29 02:22:38

标签: python database parsing tensorflow tfrecord

在解析tfrecords之后遵循此tutorial,我偶然发现在运行model.fit时出错:
AbortedError: Session 581cc3040540fb5e is not found.
this堆栈问题之后,我认为tfrecords的解析存在问题。
我当前查看记录的代码:

count = 0
for example in tf.python_io.tf_record_iterator(train_filenames[0]):
    print(tf.train.Example.FromString(example))
    count +=1
    if count >5:
        break

输出:

features {
  feature {
    key: "filename"
    value {
      bytes_list {
        value: "BM.jpg"
      }
    }
  }
  feature {
    key: "f"
    value {
      bytes_list {
        value: "\211P

}
    }
  }
  feature {
    key: "label"
    value {
      int64_list {
        value: 0
      }
    }
  }
}

我要解析的代码是:

AUTO = tf.data.experimental.AUTOTUNE
image_size = [224,224]
def read_tfrecord(tf_record):
    features = {
        "filename": tf.io.FixedLenFeature([], tf.string), # tf.string means bytestring
        "fundus": tf.io.FixedLenFeature([], tf.string),  
        "label": tf.io.VarLenFeature(tf.float32),
    }
    tf_record = tf.parse_single_example(tf_record, features)

    filename = tf.image.decode_jpeg(tf_record['filename'], channels=3)
    filename = tf.cast(filename, tf.float32) / 255.0  # convert image to floats in [0, 1] range
    filename = tf.reshape(filename, [*image_size, 3]) # explicit size will be needed for TPU

    label = tf.cast(tf_record['label'],tf.float32)

    return filename, label

def load_dataset(filenames):
    option_no_order = tf.data.Options()
    option_no_order.experimental_deterministic = False

    dataset = tf.data.Dataset.from_tensor_slices(filenames)
    dataset = dataset.with_options(option_no_order)
    #dataset = tf.data.TFRecordDataset(filenames, num_parallel_reads=16)
    dataset = dataset.interleave(tf.data.TFRecordDataset, cycle_length=32, num_parallel_calls=AUTO) # faster
    dataset = dataset.map(read_tfrecord, num_parallel_calls=AUTO)
    return dataset

我的解析方式有问题吗?

0 个答案:

没有答案