在GAE中清空数据存储区

时间:2010-01-27 14:52:36

标签: python google-app-engine google-cloud-datastore

我知道你在想什么,'不是那样了!'但是我们这里是谷歌尚未提供更简单的方法。

我一直在使用基于队列的解决方案,该方法运行良好:

导入日期时间 来自模型导入*

DELETABLE_MODELS = [Alpha, Beta, AlphaBeta]

def initiate_purge():
    for e in config.DELETABLE_MODELS:
        deferred.defer(delete_entities, e, 'purging', _queue = 'purging')

class NotEmptyException(Exception): pass

def delete_entities(e, queue):
    try:
        q = e.all(keys_only=True)
        db.delete(q.fetch(200))
        ct = q.count(1)
        if ct > 0:
            raise NotEmptyException('there are still entities to be deleted')
        else:
            logging.info('processing %s completed' % queue)
    except Exception, err:
        deferred.defer(delete_entities, e, then, queue, _queue = queue)
        logging.info('processing %s deferred: %s' % (queue, err))

所有这一切都是为删除某些数据的请求排队(每个类一次),然后如果排队的进程失败或者知道还有一些要删除的东西,它会重新排队。

这比在浏览器上点击刷新10分钟还要糟糕。

但是,我在删除AlphaBeta实体时遇到问题,最后总会留下一些。我认为因为它包含参考属性:

class AlphaBeta(db.Model):
    alpha = db.ReferenceProperty(Alpha, required=True, collection_name='betas')
    beta = db.ReferenceProperty(Beta, required=True, collection_name='alphas')

我尝试删除与这些实体类型相关的索引,但这没有任何区别。

欢迎任何建议。

1 个答案:

答案 0 :(得分:1)

我不相信尝试删除引用仍然存在的实体的实体确实是一个问题,但是您可以随时将此规则重新编写为重新编写任务以串行删除实体而不是并行删除实体:

def initiate_purge():
    deferred.defer(delete_entities, Alpha, _queue = 'purging')

def delete_entities(e):
    try:
        q = e.all(keys_only=True)
        db.delete(q.fetch(200))
        ct = q.count(1)
        if ct > 0:
            raise NotEmptyException('there are still entities to be deleted')
        else:
            logging.info('processing completed')
            if type(e) == Alpha:
                logging.info('spawning delete Beta task.')
                deferred.defer(delete_entities, Beta, _queue = 'purging')
            else if type(e) == Beta:
                logging.info('spawning delete AlphaBeta task.')
                deferred.defer(delete_entities, AlphaBeta, _queue = 'purging')
    except Exception, err:
        deferred.defer(delete_entities, e, _queue = 'purging')
        logging.info('processing deferred: %s' % (err))