previous thread的发展发现,提出问题时的假设是不合时宜的(子流程实际上并没有引起问题),因此我将重点放在帖子上。
我的错误消息:
找不到记录器的处理程序 “ google.cloud.pubsub_v1.subscriber._protocol.streaming_pull_manager”
我的意图:
将Google PubSub消息属性作为Python变量传递,以在以后的代码中重复使用。
我的代码:
import time
import logging
from google.cloud import pubsub_v1
project_id = "redacted"
subscription_name = "redacted"
def receive_messages_with_custom_attributes(project_id, subscription_name):
"""Receives messages from a pull subscription."""
# [START pubsub_subscriber_sync_pull_custom_attributes]
subscriber = pubsub_v1.SubscriberClient()
subscription_path = subscriber.subscription_path(
project_id, subscription_name)
def callback(message):
print('Received message: {}'.format(message.data))
if message.attributes:
#print('Attributes:')
for key in message.attributes:
value = message.attributes.get(key);
#commented out to not print to terminal
#which should not be necessary
#print('{}: {}'.format(key, value))
message.ack()
print("this is before variables")
dirpath = "~/subfolder1/"
print(dirpath)
namepath = message.data["name"]
print(namepath)
fullpath = dirpath + namepath
print(fullpath)
print("this is after variables")
subscriber.subscribe(subscription_path, callback=callback)
# The subscriber is non-blocking, so we must keep the main thread from
# exiting to allow it to process messages in the background.
print('Listening for messages on {}'.format(subscription_path))
while True:
time.sleep(60)
# [END pubsub_subscriber_sync_pull_custom_attributes]
receive_messages_with_custom_attributes(project_id, subscription_name)
运行上述代码的完整控制台输出:
Listening for messages on projects/[redacted]
Received message: {
"kind": "storage#object",
"id": "[redacted]/0.testing/1548033442364022",
"selfLink": "https://www.googleapis.com/storage/v1/b/[redacted]/o/BSD%2F0.testing",
"name": "BSD/0.testing",
"bucket": "[redacted]",
"generation": "1548033442364022",
"metageneration": "1",
"contentType": "application/octet-stream",
"timeCreated": "2019-01-21T01:17:22.363Z",
"updated": "2019-01-21T01:17:22.363Z",
"storageClass": "MULTI_REGIONAL",
"timeStorageClassUpdated": "2019-01-21T01:17:22.363Z",
"size": "0",
"md5Hash": "1B2M2Y8AsgTpgAmY7PhCfg==",
"mediaLink": "https://www.googleapis.com/download/storage/v1/b/[redacted]/o/BSD%2F0.testing?generation=1548033442364022&alt=media",
"crc32c": "AAAAAA==",
"etag": "CPb0uvvZ/d8CEAE="
}
this is before variables
/home/[redacted]
No handlers could be found for logger "google.cloud.pubsub_v1.subscriber._protocol.streaming_pull_manager"
如您所见,第一个字符串和string-defined-as-variable已打印出来,但是在尝试从刚刚生成的字典中定义变量时,代码中断了,并且没有再执行任何print()
了。
Potentially related thread,那个用户正在发布cron作业,并从crontab envpaths找到了一个修复程序,但是我的情况是正在接收并且不使用任何cron作业,但是可能暗示在python后面/之内的另一层? >
有人可以帮助我添加一个处理程序以使此代码按预期运行吗?
答案 0 :(得分:1)
首先,如果我对您的输出显示内容有正确的了解,那么无论何时对Cloud Storage对象进行更改,您都在使用发布/订阅通知来发送消息。这些信息可能会有所帮助。
现在,message.data["name"]
不起作用,因为message.data是BYTES object。因此,不能将其索引为字典。
要将其视为dict,首先必须将其解码为base64 (import base64
)。之后,剩下的就是一个看起来像JSON格式的字符串。然后,您使用json.load()
(不要忘记import json
)将此字符串转换为字典。现在您可以为邮件编制索引了。
此代码为:
print("This is before variables")
dirpath = "/subfolder1/"
print(dirpath)
#Transform the bytes object into a string by decoding it
namepath = base64.b64decode(message.data).decode('utf-8')
#Transform the json formated string into a dict
namepath = json.loads(namepath)
print(namepath["name"])
fullpath = dirpath + namepath["name"]
print(fullpath)
print("this is after variables")
现在,如果您只想读取属性,则可以在顶部正确定义它们,例如:
if message.attributes:
print('Attributes:')
for key in message.attributes:
value = message.attributes.get(key)
print('{}: {}'.format(key, value))
因此,您可以使用:
print("this is before variables")
dirpath = "~/subfolder1/"
print(dirpath)
namepath = message.attributes["objectId"]
print(namepath)
fullpath = dirpath + namepath
print(fullpath)
print("this is after variables")
请记住,在这种特殊情况下,"objectId"
是文件名,因为它是来自发布/订阅的Cloud Storage通知使用的属性。如果您假装发送自定义消息,请将"objectId"
更改为所需的属性名称。
答案 1 :(得分:0)
正如Nahuel和Tripleee所解释的那样,问题在于消息是BYTES而不是字符串。但是,他们的代码不能完全正常工作,仍然抛出错误,我也不知道为什么。通过交叉引用google的pubsub appengine网站的示例代码,以及几个小时的反复试验,我发现以下代码可以正常工作。 可能不雅致和/或有不良做法,在这种情况下,请对其进行编辑并使其更可靠。
#Continues from after message.ack(), above code remains unchanged
#except needing to <import json>
#this makes a message.data a true python dict with strings.
payload = json.loads(message.data.decode('utf-8'))
#this finds the value of the dict with key "name"
namepath = payload["name"]
#this is just a static string to pre-pend to the file path
dirpath = "/home/[redacted]/"
#combine them into a single functioning path
fullpath = dirpath + namepath
#currently type 'unicode', so convert them to type 'str'
fullpath = fullpath.encode("utf-8")
最后,我们将得到一个全路径,其纯类型为'str',供以后的函数/命令使用。