我有350K行和200列的pandas数据框。尝试使用数据集api构造输入管道时，出现内存错误。例如，当我仅输入10K行时，一切正常，但对于所有行，则并非如此。同样，使用tf.estimator.inputs.pandas_input_fn时一切正常。
这是代码
x_train, x_test, y_train, y_test = train_test_split(train, labels, test_size=0.25)
feature_columns = [tf.feature_column.numeric_column(c) for c in train.columns
if train[c].dtype != 'object']
def train_input_fn():
dataset = tf.data.Dataset.from_tensor_slices((dict(x_train), y_train))
dataset = dataset.shuffle(1000)
dataset = dataset.batch(100)
iterator = dataset.make_one_shot_iterator()
return iterator.get_next()
model = tf.estimator.DNNClassifier(feature_columns=feature_columns, hidden_units=[20, 2]
model.train(input_fn=train_input_fn, steps=1000)
错误消息
INFO:tensorflow:Calling model_fn.
INFO:tensorflow:Done calling model_fn.
INFO:tensorflow:Create CheckpointSaverHook.
INFO:tensorflow:Graph was finalized.
2018-09-28 14:42:03.736495: I T:\src\github\tensorflow\tensorflow\core\platform\cpu_feature_guard.cc:140] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2
2018-09-28 14:42:04.070692: I T:\src\github\tensorflow\tensorflow\core\common_runtime\gpu\gpu_device.cc:1356] Found device 0 with properties:
name: GeForce GTX 1070 major: 6 minor: 1 memoryClockRate(GHz): 1.797
pciBusID: 0000:01:00.0
totalMemory: 8.00GiB freeMemory: 6.63GiB
2018-09-28 14:42:04.072060: I T:\src\github\tensorflow\tensorflow\core\common_runtime\gpu\gpu_device.cc:1435] Adding visible gpu devices: 0
2018-09-28 14:42:05.139979: I T:\src\github\tensorflow\tensorflow\core\common_runtime\gpu\gpu_device.cc:923] Device interconnect StreamExecutor with strength 1 edge matrix:
2018-09-28 14:42:05.140271: I T:\src\github\tensorflow\tensorflow\core\common_runtime\gpu\gpu_device.cc:929] 0
2018-09-28 14:42:05.140461: I T:\src\github\tensorflow\tensorflow\core\common_runtime\gpu\gpu_device.cc:942] 0: N
2018-09-28 14:42:05.141143: I T:\src\github\tensorflow\tensorflow\core\common_runtime\gpu\gpu_device.cc:1053] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 6401 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1070, pci bus id: 0000:01:00.0, compute capability: 6.1)
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Done running local_init_op.
Traceback (most recent call last):
File "C:/Users/.../test.py", line 150, in <module>
nn.train(input_fn=train_input_fn, steps=10)
...
File "C:\Users\...\google\protobuf\text_format.py", line 118, in getvalue
return self._writer.getvalue()
MemoryError
我尝试设置不同的批处理大小，网络体系结构，但错误仍然存在。