Question

我目前正在尝试学习Google产品，我想实现非常简单的任务。场景是，我正在使用Flask Web框架在应用程序引擎中运行Web应用程序。从表格中我获取输入，并想要发布一条消息到pubsub，然后从他们那里我设置了将通过单词（从html输入）到大查询的数据流。当我使用控制台时，所有这些工作正常。从控制台，我在pubsub上发布了消息，它转到bigquery，但是当我尝试python代码时，它不起作用。这是全景。

app引擎-> pubsub->数据流->大查询。

这是代码。请告诉我我在做什么错。我收到输入字，但是当我这样做时它不会进入大型查询和控制台，

app.yaml

runtime: python
env: flex
entrypoint: gunicorn -b :$PORT main:app

runtime_config:
  python_version: 3
env_variables:
    PUBSUB_TOPIC: projects/data-pipeline-219304/topics/mytest
    PUBSUB_VERIFICATION_TOKEN: 1234test

manual_scaling:
  instances: 1
resources:
  cpu: 1
  memory_gb: 0.5
  disk_size_gb: 10

main.py

import base64
import json
import logging
import os
import argparse

from flask import Flask, redirect, url_for, request, render_template, current_app
from google.cloud import pubsub_v1

app = Flask(__name__)


app.config['PUBSUB_VERIFICATION_TOKEN'] = \
    os.environ['PUBSUB_VERIFICATION_TOKEN']
app.config['PUBSUB_TOPIC'] = os.environ['PUBSUB_TOPIC']
app.config['PROJECT'] = os.environ['GOOGLE_CLOUD_PROJECT']


@app.route('/', methods=['GET', 'POST'])
def index():
    return render_template('index.html')


@app.route('/data', methods = ['POST'])
def handle_data():
    user = request.form.get('nm').encode('utf-8')
    publisher = pubsub_v1.PublisherClient()
    topic_path = publisher.topic_path(
        current_app.config['PROJECT'],
        current_app.config['PUBSUB_TOPIC'])
    publisher.publish(topic_path, data=user)

    receive_messages()
    return 'OK'

def receive_messages():
    """Receives messages from a pull subscription."""
    # [START pubsub_subscriber_async_pull]
    # [START pubsub_quickstart_subscriber]
    import time

    project_id = "data-pipeline-219304"
    subscription_name = "projects/data-pipeline-219304/subscriptions/mytestsubscription"

    subscriber = pubsub_v1.SubscriberClient()
    # The `subscription_path` method creates a fully qualified identifier
    # in the form `projects/{project_id}/subscriptions/{subscription_name}`
    subscription_path = subscriber.subscription_path(
        project_id, subscription_name)

    def callback(message):
        print('Received message: {}'.format(message))
        message.ack()

    subscriber.subscribe(subscription_path, callback=callback)

    # The subscriber is non-blocking. We must keep the main thread from
    # exiting to allow it to process messages asynchronously in the background.
    print('Listening for messages on {}'.format(subscription_path))
    while True:
        time.sleep(60)
    # [END pubsub_subscriber_async_pull]
    # [END pubsub_quickstart_subscriber]

@app.errorhandler(500)
def server_error(e):
    logging.exception('An error occurred during a request.')
    return """
    An internal error occurred: <pre>{}</pre>
    See logs for full stacktrace.
    """.format(e), 500

if __name__ == '__main__':
   app.run(host='127.0.0.1', port=8080, debug=True)

requirements.txt

Flask==1.0.2
google-cloud-pubsub==0.38.0
gunicorn==19.9.0

index.html来自模板

<!doctype html>
<html>
   <body>
      <form action="/data" method = "post">
         <p>Enter Name:</p>
         <p><input type = "text" name = "nm" /></p>
         <p><input type = "submit" value = "submit" /></p>
      </form>
   </body>
</html>

使用python通过pubsub和dataflow将消息从App Engine发布到bigquery

0 个答案: