在App Engine中实现Web请求的速率限制或限制算法的有效方法?

时间:2014-03-12 17:40:25

标签: python algorithm google-app-engine throttling rate-limiting

我正在寻找为Web应用程序实现移动时间窗口速率限制算法的有效方法。我正在寻找一种可扩展的算法。

到目前为止,我正在考虑使用分片计数器和内存缓存。

这里的算法是伪语言:

For each request:
1: get the number of requests in the last N minutes from memcache
2: if nothing found in the memcache (memcache flushed or first call?)
3:   get the number of requests in the last N minutes from ndb (expensive!)
4: if the number is to high
5:   block the request
6: increment the sharding counter
7: increment the memcache value (failsafe, if an error occurs here ignore it)
8: process the request

到目前为止,我发现的其他问题并不适用于App Engine的上下文。

1 个答案:

答案 0 :(得分:3)

你可以完全在memcache中做这样的事情,虽然它无法在随机密钥驱逐或冲洗中存活下来:

# Create a key based on time truncated to the minute.
key = 'X' + str(datetime.datetime.utcnow().replace(second=0, microsecond=0))
# Initialize the key and have it expire after a while.
if not memcache.add(key, 1, time=90):
    # If the key already exists, increment the value and save the result.
    count = memcache.incr(key)
    # Do something if it's greater than your per minute rate limit.
    if count > MAX_X_PER_MINUTE:
        raise Error