Question

我在Google App Engine中使用递归方法来遍历分页页面。队列中只能同时有一个任务，因为你只能获得当前页面的下一页，即无法知道要同时添加到队列的所有页面。

递归函数工作正常，但是它不遵守queue.yaml文件中的指令。

我已经创建了一个用于说明问题的测试示例。我希望这个过程需要五分钟。原因是queue.yaml文件应该每分钟只执行一个任务，我将五个任务发送到队列。但是，所有任务都在执行。

我可以在test（）方法中使用sleep函数一起破解某些内容以减慢它的速度，但这会破坏在queue.yaml文件中使用速率限制的目的。

任何帮助将不胜感激！

main.py

@app.route('/test',methods=['POST'])
def test():     
    data = json.loads(request.data)
    current_num,max_num = data['current_num'],data['max_num']
    if current_num <= max_num:      
        current_num += 1
        task = taskqueue.add(
                            queue_name='test-queue',
                            method='POST',
                            url='/test',
                            payload=json.dumps({'current_num':current_num,
                                                'max_num':max_num}))
        message = "current_num: {}, max_num: {}".format(current_num,max_num)                
        print message
        return message
    else:
        print 'finished'
        return 'finished'

@app.route('/task')
def test_task():
    max_num = 5

    task = taskqueue.add(
        queue_name='test-queue',
        method='POST',
        url='/test',
        payload=json.dumps({'current_num':0,'max_num':max_num})
    )

    print "current_num: {}, max_num: {}".format(0,max_num)
    return 'submitted task'

queue.yaml中

queue:

- name: test-queue
  target: app.main
  rate: 1/m
  bucket_size: 1
  max_concurrent_requests: 1
  retry_parameters:
    min_backoff_seconds: 5
    task_retry_limit: 1

日志文件：

WARNING  2018-04-18 12:24:19,824 urlfetch_stub.py:555] Stripped prohibited headers from URLFetch request: ['content-length']
current_num: 0, max_num: 5
INFO     2018-04-18 12:24:20,299 module.py:835] default: "GET /task HTTP/1.1" 200 14
WARNING  2018-04-18 12:24:20,338 dispatcher.py:785] Ignoring instance/version in app.main; multiple versions are not supported in devappserver.
current_num: 1, max_num: 5
INFO     2018-04-18 12:24:20,350 module.py:835] default: "POST /test HTTP/1.1" 200 26
WARNING  2018-04-18 12:24:20,350 dispatcher.py:785] Ignoring instance/version in app.main; multiple versions are not supported in devappserver.
current_num: 2, max_num: 5
INFO     2018-04-18 12:24:20,362 module.py:835] default: "POST /test HTTP/1.1" 200 26
WARNING  2018-04-18 12:24:20,363 dispatcher.py:785] Ignoring instance/version in app.main; multiple versions are not supported in devappserver.
current_num: 3, max_num: 5
INFO     2018-04-18 12:24:20,380 module.py:835] default: "POST /test HTTP/1.1" 200 26
WARNING  2018-04-18 12:24:20,381 dispatcher.py:785] Ignoring instance/version in app.main; multiple versions are not supported in devappserver.
current_num: 4, max_num: 5
INFO     2018-04-18 12:24:20,399 module.py:835] default: "POST /test HTTP/1.1" 200 26
WARNING  2018-04-18 12:24:20,400 dispatcher.py:785] Ignoring instance/version in app.main; multiple versions are not supported in devappserver.
current_num: 5, max_num: 5
INFO     2018-04-18 12:24:20,415 module.py:835] default: "POST /test HTTP/1.1" 200 26
WARNING  2018-04-18 12:24:20,416 dispatcher.py:785] Ignoring instance/version in app.main; multiple versions are not supported in devappserver.
current_num: 6, max_num: 5
INFO     2018-04-18 12:24:20,441 module.py:835] default: "POST /test HTTP/1.1" 200 26
WARNING  2018-04-18 12:24:20,441 dispatcher.py:785] Ignoring instance/version in app.main; multiple versions are not supported in devappserver.
finished
INFO     2018-04-18 12:24:20,490 module.py:835] default: "POST /test HTTP/1.1" 200 8

请注意执行时间：

2018-04-18 12:24:20,299
2018-04-18 12:24:20,350
2018-04-18 12:24:20,362
2018-04-18 12:24:20,380
2018-04-18 12:24:20,399
2018-04-18 12:24:20,415
2018-04-18 12:24:20,441

Answer 1

它与dev服务器的预期行为，如documentation中所述：

开发服务器不尊重队列的速率和桶大小属性。因此，任务尽可能靠近其ETA执行。将速率设置为0并不会阻止任务自动执行。

Google任务队列中的递归不遵守速率限制

1 个答案: