Django Celery任务处理csv - 模型不保存

时间:2013-10-09 02:36:50

标签: django csv django-models django-celery

我在尝试在芹菜任务期间保存对模型的更改时遇到了一些行为,但更改没有提交。我有一个用户保存上传记录的模型,一旦上传文件,就运行芹菜任务来处理csv并将过程的结果保存到数据库中,例如,流程状态,处理时间,处理的记录数等。

在models.py中,我有以下方法:

def import_records_data(self):
    total_records = 0
    with self.filepath.file as csvfile:
        reader = csv.reader(csvfile, delimiter=',')
        next(reader, None) # skip header
        for row in reader:
            # process record
            total_records += 1
    return total_records

def process(self):
    self.date_start_processing = datetime.datetime.utcnow().replace(tzinfo=utc)
    try:
        # process upload data,
        records_processed = self.import_records_data()
    except Exception, e:
        self._mark_failed(unicode(e))
    else:
        self._mark_processed(num_records=records_processed)

def _mark_processed(self, num_records, description=None):
    self.status = self.PROCESSED
    self.date_end_processing = datetime.datetime.utcnow().replace(tzinfo=utc)
    self.num_records = num_records
    self.processing_description = description
    self.save()

def _mark_failed(self, description):
    self.status = self.FAILED
    self.processing_description = description
    self.save()

 def was_processing_successful(self):
    return self.status == self.PROCESSED

调用_mark_processed或_mark_failed时,调用self.save()时,更改不会保存到数据库中。从tasks.py:

调用此方法
@task(name='csv-process-upload')
def process_upload(upload_id):
    upload = Upload.objects.get(id=upload_id)
    upload.process()

    if upload.was_processing_successful():
        message_user(
            upload.user,
            "Your upload '%s' was processed successfully, %s records processed" % (
                upload.filename,
                upload.num_records))
    else:
        message_user(
            upload.user,
            "Your upload '%s' could not be processed, error message: %s" % (
                upload.filename,
                upload.processing_description,))

什么可能阻止模型保存?当我在shell中调试_mark_processed并键入self.save()时,更改会反映在数据库中。

1 个答案:

答案 0 :(得分:1)

尝试使用django celery transactions调用您的任务,以确保您的视图在触发任务执行之前完成事务。