如何防止FIFO队列有条件地暂停

时间:2017-08-17 12:21:25

标签: python multithreading machine-learning tensorflow

当队列完整或空时,FIFO Queue将暂停其enqueuedequeue个功能。当你只有一个队列时,这不是问题。

我的情况是我有两个队列,我必须排除以下某些条件:

run_options = tf.RunOptions(timeout_in_ms=10000)
i1, l1, i2, l2 = produce_sample()

if l1 == l2:
    sess.run(enqueue_same_op, feed_dict={x1: i1, y1: l1, x2: i2, y2: l2}, \
             options=run_options)
else:
    sess.run(enqueue_diff_op, feed_dict={x1: i1, y1: l1, x2: i2, y2: l2})

enqueue_same_openqueue_diff_op分别是queue_samequeue_diff的操作。

因为l1<>l2l1==l2更强更强的概率。因此,即使我采用了queue_diff,会话也将暂停加入run_options

是否可以获取队列的大小,或测试(如果已满)?

非常感谢你。

2 个答案:

答案 0 :(得分:0)

  

有没有办法获得队列的大小,或测试它是否已满?

您可以使用size()函数设置队列的大小,并针对队列容量进行测试

capacity=1000 # your queue capacity
dtype=tf.float32
queue_diff = tf.FIFOQueue(capacity, dtype)
enqueue_op = tf.cond(tf.equal(queue_diff.size(), capacity), lambda:is_full(), lambda: enqueue(queue_diff, enqueue_elements..))

# a function to takecare when the queue is full
def is_full():
    return 'Enqueue_Failed'

# a function for enqueue ops
def enqueue(queue, element...):
    queue.enqueue(element)
    return 'Enqueue_Success'

答案 1 :(得分:0)

您可以通过调用size方法获取队列的大小,该方法返回张量:

$ queue = tf.FIFOQueue(capacity=100, dtype=tf.float32)
$ queue.size()
<tf.Tensor 'fifo_queue_Size:0' shape=() dtype=int32>

如果您有一个固定大小的队列,就像上面的示例一样,您可以使用tf.cond函数来调整流量,从而检查它是否已满。

或者,您可以使用capacity=-1使您的队列无限制。严格来说,这不是在开放API中,也没有在官方文档中说明,但可以在源代码中找到它:

def _fifo_queue_v2(component_types, shapes=None, capacity=None,
                   container=None, shared_name=None, name=None):
  r"""A queue that produces elements in first-in first-out order.

  Args:
    component_types: A list of `tf.DTypes` that has length `>= 1`.
      The type of each component in a value.
    shapes: An optional list of shapes (each a `tf.TensorShape` or list of `ints`). Defaults to `[]`.
      The shape of each component in a value. The length of this attr must
      be either 0 or the same as the length of component_types. If the length of
      this attr is 0, the shapes of queue elements are not constrained, and
      only one element may be dequeued at a time.
    capacity: An optional `int`. Defaults to `-1`.
      The upper bound on the number of elements in this queue.
      Negative numbers mean no limit.
    container: An optional `string`. Defaults to `""`.
      If non-empty, this queue is placed in the given container.
      Otherwise, a default container is used.
    shared_name: An optional `string`. Defaults to `""`.
      If non-empty, this queue will be shared under the given name
      across multiple sessions.
    name: A name for the operation (optional).

  Returns:
    A `Tensor` of type `resource`. The handle to the queue.
  """
  result = _op_def_lib.apply_op("FIFOQueueV2",
                                component_types=component_types,
                                shapes=shapes, capacity=capacity,
                                container=container, shared_name=shared_name,
                                name=name)
  return result

如果您对这种依赖感到满意,并且您确信队列不会占用所有可用内存,则可以通过-1,从而大大简化您的代码。