Question

代码tensorflow/examples/tutorials/word2vec/word2vec_basic.py有一条评论# Ops and variables pinned to the CPU because of missing GPU implementation。我还发现GPU无法实现操作tf.nn.nce_loss。那么为什么tf.nn.nce_loss无法在GPU上运行？

我已使用log_device_placement查看MUST BE CPU操作。结果如下：

nce_loss/LogUniformCandidateSampler: (LogUniformCandidateSampler): /job:localhost/replica:0/task:0/cpu:0
I tensorflow/core/common_runtime/simple_placer.cc:841] nce_loss/LogUniformCandidateSampler: (LogUniformCandidateSampler)/job:localhost/replica:0/task:0/cpu:0
nce_loss/ComputeAccidentalHits: (ComputeAccidentalHits): /job:localhost/replica:0/task:0cpu:0
I tensorflow/core/common_runtime/simple_placer.cc:841] nce_loss/ComputeAccidentalHits: (ComputeAccidentalHits)/job:localhost/replica:0/task:0/cpu:0
nce_loss/SparseToDense: (SparseToDense): /job:localhost/replica:0/task:0/cpu:0
I tensorflow/core/common_runtime/simple_placer.cc:841] nce_loss/SparseToDense: (SparseToDense)/job:localhost/replica:0/task:0/cpu:0
nce_loss/concat: (ConcatV2): /job:localhost/replica:0/task:0/cpu:0
I tensorflow/core/common_runtime/simple_placer.cc:841] nce_loss/concat: (ConcatV2)/job:localhost/replica:0/task:0/cpu:0
nce_loss/LogUniformCandidateSampler: (LogUniformCandidateSampler): /job:localhost/replica:0/task:0/cpu:0
I tensorflow/core/common_runtime/simple_placer.cc:841] nce_loss/LogUniformCandidateSampler: (LogUniformCandidateSampler)/job:localhost/replica:0/task:0/cpu:0
nce_loss/ComputeAccidentalHits: (ComputeAccidentalHits): /job:localhost/replica:0/task:0cpu:0
I tensorflow/core/common_runtime/simple_placer.cc:841] nce_loss/ComputeAccidentalHits: (ComputeAccidentalHits)/job:localhost/replica:0/task:0/cpu:0
nce_loss/SparseToDense: (SparseToDense): /job:localhost/replica:0/task:0/cpu:0
I tensorflow/core/common_runtime/simple_placer.cc:841] nce_loss/SparseToDense: (SparseToDense)/job:localhost/replica:0/task:0/cpu:0
nce_loss/concat: (ConcatV2): /job:localhost/replica:0/task:0/cpu:0
I tensorflow/core/common_runtime/simple_placer.cc:841] nce_loss/concat: (ConcatV2)/job:localhost/replica:0/task:0/cpu:0
nce_loss/LogUniformCandidateSampler: (LogUniformCandidateSampler): /job:localhost/replica:0/task:0/cpu:0
I tensorflow/core/common_runtime/simple_placer.cc:841] nce_loss/LogUniformCandidateSampler: (LogUniformCandidateSampler)/job:localhost/replica:0/task:0/cpu:0
nce_loss/ComputeAccidentalHits: (ComputeAccidentalHits): /job:localhost/replica:0/task:0cpu:0
I tensorflow/core/common_runtime/simple_placer.cc:841] nce_loss/ComputeAccidentalHits: (ComputeAccidentalHits)/job:localhost/replica:0/task:0/cpu:0
nce_loss/SparseToDense: (SparseToDense): /job:localhost/replica:0/task:0/cpu:0
I tensorflow/core/common_runtime/simple_placer.cc:841] nce_loss/SparseToDense: (SparseToDense)/job:localhost/replica:0/task:0/cpu:0
nce_loss/concat: (ConcatV2): /job:localhost/replica:0/task:0/cpu:0
I tensorflow/core/common_runtime/simple_placer.cc:841] nce_loss/concat: (ConcatV2)/job:localhost/replica:0/task:0/cpu:0
nce_loss/LogUniformCandidateSampler: (LogUniformCandidateSampler): /job:localhost/replica:0/task:0/cpu:0
I tensorflow/core/common_runtime/simple_placer.cc:841] nce_loss/LogUniformCandidateSampler: (LogUniformCandidateSampler)/job:localhost/replica:0/task:0/cpu:0
nce_loss/ComputeAccidentalHits: (ComputeAccidentalHits): /job:localhost/replica:0/task:0cpu:0
I tensorflow/core/common_runtime/simple_placer.cc:841] nce_loss/ComputeAccidentalHits: (ComputeAccidentalHits)/job:localhost/replica:0/task:0/cpu:0
nce_loss/SparseToDense: (SparseToDense): /job:localhost/replica:0/task:0/cpu:0
I tensorflow/core/common_runtime/simple_placer.cc:841] nce_loss/SparseToDense: (SparseToDense)/job:localhost/replica:0/task:0/cpu:0
nce_loss/concat: (ConcatV2): /job:localhost/replica:0/task:0/cpu:0
I tensorflow/core/common_runtime/simple_placer.cc:841] nce_loss/concat: (ConcatV2)/job:localhost/replica:0/task:0/cpu:0
nce_loss/LogUniformCandidateSampler: (LogUniformCandidateSampler): /job:localhost/replica:0/task:0/cpu:0
I tensorflow/core/common_runtime/simple_placer.cc:841] nce_loss/LogUniformCandidateSampler: (LogUniformCandidateSampler)/job:localhost/replica:0/task:0/cpu:0
nce_loss/ComputeAccidentalHits: (ComputeAccidentalHits): /job:localhost/replica:0/task:0cpu:0
I tensorflow/core/common_runtime/simple_placer.cc:841] nce_loss/ComputeAccidentalHits: (ComputeAccidentalHits)/job:localhost/replica:0/task:0/cpu:0
nce_loss/SparseToDense: (SparseToDense): /job:localhost/replica:0/task:0/cpu:0
I tensorflow/core/common_runtime/simple_placer.cc:841] nce_loss/SparseToDense: (SparseToDense)/job:localhost/replica:0/task:0/cpu:0
nce_loss/concat: (ConcatV2): /job:localhost/replica:0/task:0/cpu:0
I tensorflow/core/common_runtime/simple_placer.cc:841] nce_loss/concat: (ConcatV2)/job:localhost/replica:0/task:0/cpu:0

但是，我仍然不知道哪个特殊的Ops导致了这个问题。是否有任何特殊的Ops无法通过GPU实现？或者这是由于tensorflow本身缺乏实现？

Answer 1

您也可以使用gpu加速器执行此示例。
查看受tensorflow支持的设备https://www.tensorflow.org/guide/using_gpu#supported_devices
您必须添加以下代码行

with tf.device('/gpu:0'):

并删除#with tf.device('/cpu:0'):

此行基本上用gpu替换cpu： https://github.com/tensorflow/tensorflow/blob/bd965956535b1026a0769d04c3f6419d020f5f9c/tensorflow/examples/tutorials/word2vec/word2vec_basic.py#L189

希望这会有所帮助。

为什么张量流中的`tf.nn.nce_loss`无法在GPU上运行？

1 个答案: