分片计数器RPC和计算成本增加

时间:2016-01-12 03:17:19

标签: python google-app-engine google-cloud-datastore app-engine-ndb sharding

我正在对Google App Engine的分片计数器实现进行一些诊断,我不确定其他RPC的来源。

当获得20个计数器的非缓存计数时,带有20个分片的开箱即用版本在Appstats中提供以下信息:

RPCs
memcache.get                100  
memcache.set                80  
memcache.del                0  
datastore.get               60  
datastore_commit            0  
datastore_put               0  
datastore_begin_transaction 0

Billed Ops  
datastore_read              420  
datastore_write             0  

Cost                        29,400

供参考,这是计数器代码:

class GeneralCounterShard(ndb.Model):
    """Shards for each named counter."""
    count = ndb.IntegerProperty(default=0)


def get_count(name):
    """Retrieve the value for a given sharded counter.
    Args:
        name: The name of the counter.
    Returns:
        Integer; the cumulative count of all sharded counters for the given
            counter name.
    """
    total = memcache.get(name)
    if total is None:
        total = 0
        all_keys = GeneralCounterShardConfig.all_keys(name)
        for counter in ndb.get_multi(all_keys):
            if counter is not None:
                total += counter.count
        memcache.add(name, total, 60)
    return total

当我更改分片计数器代码以使get_count成为GeneralCounterShard类的“classmethod”时,RPC会发生巨大变化:

RPCs
memcache.get                100  
memcache.set                80  
memcache.del                20  
datastore.get               80  
datastore_commit            20  
datastore_put               20  
datastore_begin_transaction 20

Billed Ops
datastore_read              440  
datastore_write             60   

Cost                        36,800

供参考,以下是更改:

class GeneralShardedCounter(ndb.Model):

    count = ndb.IntegerProperty(default=0)

    @classmethod
    def get_count(name):

        total = memcache.get(name)
        if total is None:
            total = 0
            all_keys = GeneralShardedCounterConfig.all_keys(name)
            for counter in ndb.get_multi(all_keys):
                if counter is not None:
                    total += counter.count
            memcache.add(name, total, 60)
        return total

在回顾中,两者之间的区别是:

RPCs
memcache.get                +0  
memcache.set                +0  
memcache.del                +20  
datastore.get               +20  
datastore_commit            +20  
datastore_put               +20  
datastore_begin_transaction +20 

Billed Ops
datastore_read              +20  
datastore_write             +60  

Cost                        +7,400 

只需将一个方法作为ndb模型的类方法,会导致RPC和读写的跳转是什么?

0 个答案:

没有答案