使用python DictReader将CSV读入Django模型时遇到问题

时间:2014-01-13 17:16:38

标签: python django csv

我正在使用Cloud9(我读过的目前使用的是Python 2.6,而不是Python 3)来编写Django应用程序。我正在尝试使用DictReader读取CSV文件,并使用CSV中的每一列创建模型的新实例并填充模型字段。

views.py

class GenerateFromCSV(CreateView):
    model = group
    template_name = "my_app/csv_generate.html"
    def form_valid(self, form):
        new_group = form.save()
        the_csv = open(new_group.group_csv, 'rbU')
        fieldnames = ['c_type', 'f_name', 'q_type', 'ans', 'n_questions', 'bucket']
        data_file = csv.DictReader(the_csv, fieldnames = fieldnames, delimiter=',', dialect=csv.excel)
        for row in data_file:
            new_card = Card(
                name = 'card',
                card_type = row['c_type'], 
                file_name = row['f_name'], 
                question_type = row['q_type'], 
                answer = row['ans'], 
                num_questions = row['n_questions'], 
                bucket = row['bucket'],
                exam = new_exam)
            new_card.save()

models.py

class Group(models.Model):
    name = models.CharField(max_length=255, blank = True)
    subject = models.CharField(max_length=255, choices = SUBJECT, blank = True)
    num_questions = models.IntegerField(default=0, blank = True, null = True)
    group_csv = models.FileField(upload_to='csv', blank = True, null = True)
    def __unicode__(self):
        return self.name

class Card(models.Model):
    name = models.CharField(max_length=255, blank = True)
    #ordered same as column order in CSV
    card_type = models.CharField(max_length=255, choices = CARDTYPE, blank = True)
    file_name = models.CharField(max_length=255, blank = True)
    question_type = models.IntegerField(default = 0, blank = True, null = True)
    answer = models.IntegerField(max_length = 1, choices = ANSWERS, blank = True, null = True)
    num_questions = models.IntegerField(default = 0, blank = True, null = True)
    bucket = models.CharField(max_length=255, blank = True)
    exam = models.ForeignKey(Exam)
    def __unicode__(self):
        return self.name or 'card'

使用上面的代码,当我在CSV上调用open()时,我得到一个TypeError(强制转换为Unicode:需要字符串或缓冲区,找到FieldFile)。如果我删除对open()的调用,我会收到错误:'在不带引号的字段中看到的新行字符 - 你需要在通用换行模式下打开文件吗?'

我的CSV格式(并非每列都包含每行中的数据):

3,the_file_name.png,0,"00001",,Equations

这个的正确语法是什么?

修改 这是我的堆栈跟踪:

Traceback:
File "/usr/libexec/openshift/cartridges/c9-0.1/root/python2.6.6/site-packages/Django-1.5-py2.6.egg/django/core/handlers/base.py" in get_response
  115.                         response = callback(request, *callback_args, **callback_kwargs)
File "/usr/libexec/openshift/cartridges/c9-0.1/root/python2.6.6/site-packages/Django-1.5-py2.6.egg/django/views/generic/base.py" in view
  68.             return self.dispatch(request, *args, **kwargs)
File "/usr/libexec/openshift/cartridges/c9-0.1/root/python2.6.6/site-packages/Django-1.5-py2.6.egg/django/views/generic/base.py" in dispatch
  86.         return handler(request, *args, **kwargs)
File "/usr/libexec/openshift/cartridges/c9-0.1/root/python2.6.6/site-packages/Django-1.5-py2.6.egg/django/views/generic/edit.py" in post
  199.         return super(BaseCreateView, self).post(request, *args, **kwargs)
File "/usr/libexec/openshift/cartridges/c9-0.1/root/python2.6.6/site-packages/Django-1.5-py2.6.egg/django/views/generic/edit.py" in post
  165.             return self.form_valid(form)
File "/var/lib/stickshift/52a55ef4e0b8cde0ff000036/app-root/data/705411/zamrdjango/zamr/views.py" in form_valid
  35.         with new_exam.exam_csv.open('rbU') as the_csv:

Exception Type: AttributeError at /import/
Exception Value: 'NoneType' object has no attribute '__exit__'

3 个答案:

答案 0 :(得分:0)

查看FileField的文档,因为问题是您没有将文件名传递给文件open函数:
https://docs.djangoproject.com/en/dev/ref/models/fields/#django.db.models.fields.files.FieldFile

您的代码可能是:

class GenerateFromCSV(CreateView):
    model = group
    template_name = "my_app/csv_generate.html"
    def form_valid(self, form):
        new_group = form.save()
        with new_group.group_csv.open('rbU') as the_csv:
            fieldnames = ['c_type', 'f_name', 'q_type', 'ans', 'n_questions', 'bucket']
            data_file = csv.DictReader(the_csv, fieldnames = fieldnames, delimiter=',', dialect=csv.excel)
            for row in data_file:
                new_card = Card(
                    name = 'item',
                    card_type = row['c_type'], 
                    file_name = row['f_name'], 
                    question_type = row['q_type'], 
                    answer = row['ans'], 
                    num_questions = row['n_questions'], 
                    bucket = row['bucket'],
                    exam = new_exam)
                new_card.save()

答案 1 :(得分:0)

好吧,我最后通过在csv上调用.read()然后调用.splitlines()来获取每行的行和列来修复它。这是我的新views.py:

def form_valid(self, form):
    new_group = form.save()
    print(new_exam.exam_csv.name)
    data = new_group.group_csv.read()
    rows = data.splitlines()
    for row in rows:
        columns = row.split(',')
        print(columns)
        new_card = Card(
            name = columns[0],
            card_type = columns[0], 
            file_name = columns[1], 
            question_type = int(columns[2]),
            answer = columns[3].replace('"', '').strip(), 
            num_questions = columns[4], 
            bucket = columns[5],
            group = new_group)
        print(new_card.name)
        new_card.save()

我确定这不是正确的语法或约定,还有一些事情需要弄清楚(比如列名)但是现在它正确地将所有信息解析为正确的模型字段。

答案 2 :(得分:0)

我做了这样的事情

{# csv-import.html %}
<form action="/csv-import" method="POST" enctype="multipart/form-data">
    {% csrf_token %}
    <input type="file" required name="csvfile" id="id_csvfile" />
    <input type="submit" />
</form>

# views.py
@login_required
def CSVImport(request):
    if request.method == "GET":
        logger.info(f"Get /csv-import")
    elif request.method == "POST":
        logger.info(f"POST /csv-import")
        csv_len = 0
        try:
            csv_file = request.FILES["csvfile"]
            if not csv_file.name.endswith('.csv'):
                messages.error(request,'File is not CSV type')
                return HttpResponseRedirect(reverse("myapp:upload_csv"))
            #if file is too large, return
            if csv_file.multiple_chunks():
                messages.error(request, "Uploaded file is too big (%.2f MB)." % (csv_file.size/(1000*1000),))
                return HttpResponseRedirect(reverse("myapp:upload_csv"))
            file_data = csv.DictReader(csv_file.read().decode('utf-8').splitlines())
            for row in file_data:
                print('row', row)
                csv_len += 1

        except Exception as e:
            logging.getLogger("error_logger").error("Unable to upload file. "+repr(e))
            messages.error(request,"Unable to upload file. "+repr(e))
        messages.add_message(request, messages.INFO, f"Updated {csv_len} meetings.")
    return render(request, "csv-import.html")