Question

虽然在python上使用多个（三个）训练过的tensorflow模型以3个线程（或仅2个）并行运行；我遇到内存不足的情况，但是在每个GPU上分别单独运行（2X3 = 6次）或按照下面的代码配置运行都没问题。

GPU配置-

GeForce GTX 1060 6GB主要版本 totalMemory：5.93GiB空闲内存：5.69GiB memoryClockRate（GHz）：1.7715

GeForce GTX 1050 Ti major：6 minor：1 memoryClockRate（GHz）：1.392 =>已忽略 totalMemory：3.94GiB freeMemory：3.89GiB

各个模型文件的相关（与GPU相关）代码-

1)d = '/gpu:0'
  config=tf.ConfigProto()
  #config.log_device_placement= True
  print("SUNGLASSSSSSSSSSSSSSSSSSSS")
  #config.gpu_options.per_process_gpu_memory_fraction = 0.3
  config=tf.ConfigProto(gpu_options=tf.GPUOptions(visible_device_list='0'))
  with tf.device(d):
      with tf.Session(graph=graph, config=config) as sess:
        results = sess.run(output_operation.outputs[0], {
            input_operation.outputs[0]: t
        })
  results = np.squeeze(results)

2)
d = '/gpu:1'
  config=tf.ConfigProto()
  #config.log_device_placement= True
  print("HATSSSSSSSSSSSSSSSSSS")
  config.gpu_options.per_process_gpu_memory_fraction = 0.35
  config=tf.ConfigProto(log_device_placement=False,gpu_options=tf.GPUOptions(allow_growth=True),allow_soft_placement = True)
  with tf.Session(graph=graph, config=config) as sess:
    results = sess.run(output_operation.outputs[0], {
        input_operation.outputs[0]: t
    })
  results = np.squeeze(results)

3)
d = '/gpu:1'
  config=tf.ConfigProto()
  #config.log_device_placement= True
  print("HANDSNEARFACEEEEEEEEEEEEE")
  config.gpu_options.per_process_gpu_memory_fraction = 0.4
  #config=tf.ConfigProto(log_device_placement=False,gpu_options=tf.GPUOptions(allow_growth=False),allow_soft_placement = True)
  with tf.device(d):
      with tf.Session(graph=graph, config=config) as sess:
        results = sess.run(output_operation.outputs[0], {
            input_operation.outputs[0]: t
        })
  results = np.squeeze(results)

多个受Inception V3训练的模型的GPU问题

0 个答案: