在Google App Engine中使用Dataflow时出现发布程序问题(TypeError:无法将原型文件构建到描述符池中!)

时间:2018-10-23 04:49:12

标签: python google-app-engine google-cloud-platform google-cloud-dataflow google-cloud-pubsub

我正在尝试从Google Analytics(分析)中获取数据并将其推送到pubsub。在数据流中,我从pubsub和google云存储中读取数据,并进行一些比较,最后将其推送到大查询中。发布后,我正在对大型查询执行一些操作。当我在Pycharm IDE中运行代码时,它可以处理一些运行时警告。但是,当我尝试在Google Cloud App Engine中运行时,出现以下错误。

a

我的文件夹包含以下内容:

ERROR: (gcloud.app.deploy) Error Response: [9]
Application startup error:
return util.import_app(self.app_uri)
File "/env/local/lib/python2.7/site-packages/gunicorn/util.py", line 350, in 
import_app
__import__(module)
File "/home/vmagent/app/main.py", line 6, in <module>
from google.cloud import pubsub_v1
File "/env/local/lib/python2.7/site- 
packages/google/cloud/pubsub_v1/__init__.py", line 17, in <module>
from google.cloud.pubsub_v1 import types 
 google.pubsub.v1.Publisher.DeleteTopic: "google.pubsub.v1.DeleteTopicRequest" seems to be defined in "google/cloud/proto/pubsub/v1/pubsub.proto", which is not imported by "google/cloud/pubsub_v1/proto/pubsub.proto".  To use it here, please add the necessary import.

app.yaml:

analyticsreporting.dat  app.yaml  client_key.json  dataflow.py  dfpipe  GoBear.json  main.py  requirements.txt  setup.py  venv

Requirements.txt包含venv的所有python 2.7的pip冻结,我在本地将其冻结以避免任何混乱。

runtime: python
env: flex
service: dataflow-srihari
entrypoint: gunicorn -b :$PORT main:app
manual_scaling:
    instances: 1

代码段:

proto-google-cloud-pubsub-v1==0.15.4
google-api-core==0.1.4
google-api-python-client==1.7.4
google-apitools==0.5.20 
google-auth==1.5.1
google-auth-httplib2==0.0.3
google-cloud==0.34.0
google-cloud-bigquery==0.30.0
google-cloud-core==0.28.1
google-cloud-logging==1.7.0
google-cloud-monitoring==0.30.1
google-cloud-pubsub==0.31.0
google-cloud-storage==1.13.0

要推送:

from google.cloud import pubsub_v1
def pubsub_init():
    publisher=pubsub_v1.PublisherClient()
    topic="topicname"
    return (publisher,topic)

要阅读pubsub:

my_data1 = json.dumps(my_data)
    mydata = my_data1.encode('utf-8')
    publisher.publish(topic, mydata)

请协助我避免此错误

0 个答案:

没有答案