Question

我尝试从数据集中制作一个one_shot_iterator。

我使用占位符来减少GPU内存的使用，并希望只初始化一次迭代器。

但是我得到了错误：

Traceback (most recent call last):
  File "test_placeholder.py", line 18, in <module>
    it = dset.make_one_shot_iterator()
  File "<...>/site-packages/tensorflow/python/data/ops/dataset_ops.py", line 205, in make_one_shot_iterator
    six.reraise(ValueError, err)
  File "<...>/site-packages/six.py", line 692, in reraise
    raise value.with_traceback(tb)
ValueError: Cannot capture a placeholder (name:Placeholder, 
    type:Placeholder) by value.

测试：

import tensorflow as tf
import numpy as np

buf_size = 50
batch_size = 10
n_rows = 117

a = np.random.choice(7, size=n_rows)
b = np.random.uniform(0, 1, size=(n_rows, 4))

a_ph = tf.placeholder(a.dtype, a.shape)
b_ph = tf.placeholder(b.dtype, b.shape)

with tf.Session() as sess:
    dset = tf.data.Dataset.from_tensor_slices((a_ph, b_ph))
    dset = dset.shuffle(buf_size).batch(batch_size).repeat()
    feed_dict = {a_ph: a, b_ph: b}
    it = dset.make_one_shot_iterator()

    n_batches = len(a) // batch_size
    sess.run(it.initializer, feed_dict=feed_dict)
    for i in range(n_batches):
        a_chunk, b_chunk = it.get_next()
        print(a_chunk, b_chunk)

出了什么问题？

谢谢。

Answer 1

查看importing data的指南

“单次迭代器是迭代器的最简单形式，它仅支持对数据集进行一次迭代，而无需显式初始化。单次迭代器处理几乎所有现有基于队列的输入管道的情况支持，但不支持参数化。”

这就是发生错误的原因，因为此特定迭代器不支持使用占位符进行任何参数化。我们可以改用make_initializable_iterator。

这是经过修改的代码以及您要查找的结果。

buf_size = 50
batch_size = 10
n_rows = 117

a = np.random.choice(7, size=n_rows)
b = np.random.uniform(0, 1, size=(n_rows, 4))

a_ph = tf.placeholder(a.dtype, a.shape)
b_ph = tf.placeholder(b.dtype, b.shape)

with tf.Session() as sess:
    dset = tf.data.Dataset.from_tensor_slices((a_ph, b_ph))
    dset = dset.shuffle(buf_size).batch(batch_size).repeat()
    feed_dict = {a_ph: a, b_ph: b}
    it = dset.make_initializable_iterator()

    n_batches = len(a) // batch_size
    sess.run(it.initializer, feed_dict=feed_dict)
    for i in range(n_batches):
        a_chunk, b_chunk = it.get_next()
        print(a_chunk, b_chunk)

结果：

Tensor("IteratorGetNext:0", shape=(?,), dtype=int32) Tensor("IteratorGetNext:1", shape=(?, 4), dtype=float64)
Tensor("IteratorGetNext_1:0", shape=(?,), dtype=int32) Tensor("IteratorGetNext_1:1", shape=(?, 4), dtype=float64)
Tensor("IteratorGetNext_2:0", shape=(?,), dtype=int32) Tensor("IteratorGetNext_2:1", shape=(?, 4), dtype=float64)
Tensor("IteratorGetNext_3:0", shape=(?,), dtype=int32) Tensor("IteratorGetNext_3:1", shape=(?, 4), dtype=float64)
Tensor("IteratorGetNext_4:0", shape=(?,), dtype=int32) Tensor("IteratorGetNext_4:1", shape=(?, 4), dtype=float64)
Tensor("IteratorGetNext_5:0", shape=(?,), dtype=int32) Tensor("IteratorGetNext_5:1", shape=(?, 4), dtype=float64)
Tensor("IteratorGetNext_6:0", shape=(?,), dtype=int32) Tensor("IteratorGetNext_6:1", shape=(?, 4), dtype=float64)
Tensor("IteratorGetNext_7:0", shape=(?,), dtype=int32) Tensor("IteratorGetNext_7:1", shape=(?, 4), dtype=float64)
Tensor("IteratorGetNext_8:0", shape=(?,), dtype=int32) Tensor("IteratorGetNext_8:1", shape=(?, 4), dtype=float64)
Tensor("IteratorGetNext_9:0", shape=(?,), dtype=int32) Tensor("IteratorGetNext_9:1", shape=(?, 4), dtype=float64)
Tensor("IteratorGetNext_10:0", shape=(?,), dtype=int32) Tensor("IteratorGetNext_10:1", shape=(?, 4), dtype=float64)

one_shot_iterator，占位符，无法捕获占位符

1 个答案: