Question

我有一个简单的python脚本，它使用Google pubsub来检测谷歌云存储中的新文件。该脚本只是将新消息添加到另一个线程处理这些消息的队列中：

subscriber = pubsub.SubscriberClient()
subscription_path = subscriber.subscription_path(
    project, subscription_name)

subscriber.subscribe(subscription_path, callback=callback_fun)

while True:
    if not message_queue:
        time.sleep(60)
        continue
    else:
        process_next_message(message_queue.pop())

这里，callback_fun只是将消息添加到队列中：

def callback_fun(message):
    message.ack()
    message_queue.append(message)

我遇到的问题是，在一段时间（可能是几天）之后，订阅者停止接收新的文件通知。如果我停止并重新启动脚本，它会立即收到所有通知。

我想知道是否有其他人遇到类似的问题和/或可以建议解决问题的方法（可能通过打印通常看不到的调试消息）。我现在正试图停止/重新启动订阅者，但我确信这不是在生产环境中使用的最佳选择。

我正在使用google-cloud 0.32.0和google-cloud-pubsub 0.30.1。

感谢。

Answer 1

In general, there can be several reasons why a subscriber may stop receiving messages:

If a subscriber does not ack or nack messages, the flow control limits can be reached, meaning no more messages can be delivered. This does not seem to be the case in your particular instance given that you immediately ack messages. As an aside, I would recommend against acking messages before your queue has processed them unless you are okay with the possibility of messages not being processed. Otherwise, if your app crashes after the ack, but before the message queue processes them, you will have not processed the message and will not get it redelivered since it was acked.
If another subscriber starts up for the same subscription, it could be receiving the messages. In this scenario, one would expect the subscriber to receive a subset of the messages rather than no messages at all.
Publishers just stop publishing messages and therefore there are no messages to receive. If you restart the subscriber and it starts receiving messages again, this probably isn't the case. You can also verify that a backlog is being built up by looking at the Stackdriver metric for subscription/backlog_bytes.

If your problem does not fall into one of those categories, it would be best to reach out to Google Cloud support with your project name, topic name, and subscription name so that they can narrow down the issue to either your user code, the client library, or the service.

Answer 2

我在这个问题上停留了1个小时，所以这就是我解决问题的方法：

将GOOGLE_APPLICATION_CREDENTIALS环境变量设置为另一个服务帐户，该帐户不在正确的项目中

project_id = "my_project_sandbox"

还有

my_project.json（项目使用的服务帐户）

{
  "type": "service_account",
  "project_id": "my_project_prod",
  "private_key_id": "---",
  "private_key": "---",
  ...
}

Answer 3

除了我之前评论中提供的流量控制建议外，您还可以定义{@ 3}}，只要在Pub / Sub主题中发布新消息，就会触发该Cloud Function。这些云功能充当订阅，并且每次发生某个事件（例如正在发布的消息）时都会收到通知。

此tutorial将帮助您开发后备云功能，该功能将在发布/订阅主题中发布消息时触发。

Google Pub / Sub Subscriber暂时不接收消息

3 个答案: