Python Pubsub + Subprocess抱怨记录器处理程序,该如何解决?

时间:2019-01-18 07:16:52

标签: python logging google-cloud-platform subprocess google-cloud-pubsub

编辑:找到解决方案in this new thread

我正在使用经过修改的示例代码来提取从对象更改通知生成的PubSub消息。到目前为止,它可以在 Deb9 VM 中正常运行。 在名为message.data的python字典中,有一对

  

“名称”:“ folder2 / 1.testing”

接下来,我需要传递字符串以发出unix命令,我将其用于子进程。

在一个单独的测试脚本中,但是在 Windows 10本地版本中执行,我发现

testdictionary = {"name":"folder2/1.testing"}

dirpath = "~/subfolder1/"
namepath = testdictionary["name"]
fullpath = dirpath + namepath
subprocess.call(["echo abcd '/%{0}'".format(fullpath)])

是有效的代码,回显

  

abcd〜/ subfolder1 / folder2 / 1.testing

Win10终端中。

但是,当尝试合并代码并在Deb9 VM中运行时,我得到了返回此错误的信息:

  

找不到记录器“ google.cloud.pubsub_v1.subscriber._protocol.streaming_pull_manager”的处理程序

这是我完整的组合代码(经过测试,在将dirpath添加到子流程行之前可以正常工作):

import time
import subprocess

from google.cloud import pubsub_v1

project_id = "redacted"
subscription_name = "redacted"

def receive_messages_with_custom_attributes(project_id, subscription_name):
    """Receives messages from a pull subscription."""
    # [START pubsub_subscriber_sync_pull_custom_attributes]

    subscriber = pubsub_v1.SubscriberClient()
    subscription_path = subscriber.subscription_path(
        project_id, subscription_name)

    def callback(message):
        print('Received message: {}'.format(message.data))
        if message.attributes:
            #print('Attributes:')
            for key in message.attributes:
                value = message.attributes.get(key);
                #commented out to not print to terminal
                #which should not be necessary
                #print('{}: {}'.format(key, value))
        message.ack()

        dirpath = "~/subfolder1/"
        namepath = message.data["name"]
        print(namepath)
        fullpath = dirpath + namepath
        print(fullpath)
        print("this is before subprocess")
        subprocess.call(["echo", "abcd", '/%{0}'.format(fullpath)], shell=True)
            #subprocess.call(["bash job-submit.sh -r hg19.fa.mmi -f '/%{0}' -o file_out.bam".format(fullpath)], shell=True)


    subscriber.subscribe(subscription_path, callback=callback)
    # The subscriber is non-blocking, so we must keep the main thread from
    # exiting to allow it to process messages in the background.
    print('Listening for messages on {}'.format(subscription_path))
    while True:
        time.sleep(60)
    # [END pubsub_subscriber_sync_pull_custom_attributes]

receive_messages_with_custom_attributes(project_id, subscription_name)

我一般不熟悉编码,如果有人可以告诉我为了获得期望的行为而不是记录器错误,可以对代码进行哪些更改,我将不胜感激。

我已经了解了关于import logging的一些信息,但是其他问题的上下文与我的问题相去甚远,解决方案实际上是不可转让的。

完整返回带有测试文件的控制台:

Listening for messages on projects/[redacted]
Received message: {
  "kind": "storage#object",
  "id": "[redacted]/0.testing/1548033442364022",
  "selfLink": "https://www.googleapis.com/storage/v1/b/[redacted]/o/BSD%2F0.testing",
  "name": "BSD/0.testing",
  "bucket": "[redacted]",
  "generation": "1548033442364022",
  "metageneration": "1",
  "contentType": "application/octet-stream",
  "timeCreated": "2019-01-21T01:17:22.363Z",
  "updated": "2019-01-21T01:17:22.363Z",
  "storageClass": "MULTI_REGIONAL",
  "timeStorageClassUpdated": "2019-01-21T01:17:22.363Z",
  "size": "0",
  "md5Hash": "1B2M2Y8AsgTpgAmY7PhCfg==",
  "mediaLink": "https://www.googleapis.com/download/storage/v1/b/[redacted]/o/BSD%2F0.testing?generation=1548033442364022&alt=media",
  "crc32c": "AAAAAA==",
  "etag": "CPb0uvvZ/d8CEAE="
}

No handlers could be found for logger "google.cloud.pubsub_v1.subscriber._protocol.streaming_pull_manager"

按Ctrl + C后的进一步回溯:

^CTraceback (most recent call last):
  File "ps_parse_pull.py", line 47, in <module>
    receive_messages_with_custom_attributes(project_id, subscription_name)
  File "ps_parse_pull.py", line 44, in receive_messages_with_custom_attributes
    time.sleep(60)
KeyboardInterrupt
Exception in thread Thread-Heartbeater (most likely raised during interpreter shutdown):
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
  File "/usr/lib/python2.7/threading.py", line 754, in run
  File "/usr/local/lib/python2.7/dist-packages/google/cloud/pubsub_v1/subscriber/_protocol/heartbeater.py", line 40, in heartbeat
  File "/usr/local/lib/python2.7/dist-packages/google/cloud/pubsub_v1/subscriber/_protocol/streaming_pull_manager.py", line 283, in heartbeat
  File "/usr/local/lib/python2.7/dist-packages/google/api_core/bidi.py", line 453, in is_active
  File "/usr/lib/python2.7/threading.py", line 168, in acquire
<type 'exceptions.TypeError'>: 'NoneType' object is not callable
Exception in thread Thread-2 (most likely raised during interpreter shutdown):
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
  File "/usr/lib/python2.7/threading.py", line 754, in run
  File "/usr/local/lib/python2.7/dist-packages/grpc/_channel.py", line 207, in consume_request_iterator
<type 'exceptions.AttributeError'>: 'NoneType' object has no attribute 'return_from_user_request_generator'
Exception in thread Thread-LeaseMaintainer (most likely raised during interpreter shutdown):
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
  File "/usr/lib/python2.7/threading.py", line 754, in run
  File "/usr/local/lib/python2.7/dist-packages/google/cloud/pubsub_v1/subscriber/_protocol/leaser.py", line 103, in maintain_leases
  File "/usr/local/lib/python2.7/dist-packages/google/cloud/pubsub_v1/subscriber/_protocol/histogram.py", line 142, in percentile
<type 'exceptions.TypeError'>: 'NoneType' object is not callable

0 个答案:

没有答案