Question

我正在尝试创建一个使用CSV数据的django网络应用。我已经创建了一个django模型来存储来自这些csv的数据，我已经将所有csv存储在STATIC_ROOT中。当我加载页面时，被调用的模板（datalandingpage.html）加载没有问题。但是，当我检查django admin时，没有导入任何CSV。如果我猜测，我认为这个缺点与django无法找到文件有关（尽管我只是在猜测，因为我是一个相对较新的开发人员，所以我很可能是错的）。以下是我作为尝试写的所有相关代码。

编辑：所以看起来我的视图功能可能会直接跳到渲染部分，这可能就是我运行时没有看到任何错误的原因。关于为什么会出现这种情况的任何想法？

settings.py：

# Static files (CSS, JavaScript, Images)
# https://docs.djangoproject.com/en/1.11/howto/static-files/

STATIC_URL = '/static/'
STATIC_ROOT = os.path.join(BASE_DIR, 'static/data')

models.py：

from __future__ import unicode_literals
from django.db import models
from django.contrib import admin

class Data(models.Model):
     # A bunch of model objects (CharField/IntegerFields mainly)

    class Meta:
        verbose_name_plural = 'Data Sets'

    def __str__(self):
        return self.someobject

views.py：

from __future__ import unicode_literals
from django.conf import settings
from django.shortcuts import render, render_to_response
from django.template import RequestContext
from .models import Data
import csv
import os

def landing(request):

# Opens all csv files in specified directory
directory = os.path.join(settings.STATIC_ROOT)
for files in os.walk(directory):
    for file in files:
        if file.endswith(".csv"):
            f = open(file, 'r')
            reader = csv.reader(f)

            #Checks the rows/columns and tries to find the specified string that marks the start of data
            for i, row in enumerate(reader):
                target_string = "Some string in the CSV"
                if target_string in row:
                    target_row = i
                    return target_row
                    break

            #Checks the rows/columns and tries to find the specified string that marks the end of data
            for j, row in enumerate(reader):
                end_target_string = "Another string in the CSV"
                if end_target_string in row:
                    end_target_row = j
                    return end_target_row
                    break

            #Begins parsing the csv, but skips until two rows past the target beginning string
            for k, row in enumerate(reader):

                #imports csv data when the row number is between target string and target end string
                if k >= (target_row + 2):
                    row = row.split(',')
                    DataSource = Data.objects.create()
                    DataSource.someobject1 = row[0]
                    DataSource.someobject2 = row[1]
                    DataSource.save()

                # When the parse gets to the row with the end string, it closes the file.
                elif k >= end_target_row:
                    reader.close()
                    break

        # Prints something if the file isn't a csv file
        else:
            print "This isn't a csv file"

return render(request, "Data/datalandingpage.html")

Answer 1

如果我理解正确，您需要从CSV文件创建模型列表。您可以考虑使用现有库：http://django-import-export.readthedocs.io/en/stable/index.html

Answer 2

您在如何循环目录和打开文件时遇到问题。每个目录的os.walk产生了一个(root_dir, directories, files)元组，其中directories是root_dir中的目录名元组，files是文件元组在root_dir

中找到的名称

在您的代码中，您有以下内容：

for files in os.walk(directory):
    for file in files:
        if file.endswith(".csv"):
            f = open(file, 'r')
            reader = csv.reader(f)

这不会做你认为它做的事情，毫无疑问会产生AttributeError，因为files中的for files in os.walk(...)将始终有两个元组，而元组没有{ {1}}方法。

此外，当您正确循环.endswith提供的文件时 - 它们只是文件名。您必须使用根目录加入路径。

要修复，您应该按如下方式对其进行修改....

os.walk

请注意，这会将for root, dirs, files in os.walk(directory): for filename in files: if filename.endswith(".csv"): file_path = os.path.join(root, filename) with open(file_path) as f: reader = csv.reader(f)产生的元组解压缩到os.walk现在root, dirs, files实际上是文件名列表。此外，files用于从名称创建文件的完整路径。

另请注意，我将其修改为使用with open(...)上下文管理器。此块下的代码在打开文件时执行，一旦块结束，文件将为您关闭。

Answer 3

我最终重写了脚本并且它有效。我认为我的奇怪的循环结构导致脚本不能按预期工作（或者根本没有）。我还使用了bulk_create，而不必在每一行之后调用save来提高进程的效率。这是有效的版本：

views.py：

from __future__ import unicode_literals
from django.conf import settings
from django.shortcuts import render, render_to_response
from django.template import RequestContext
from .models import Data
import os
import csv, sys


def import_data(request):

    # Opens all csv files in specified directory

    directory = os.path.join('Some Target Directory')
    count = 0
    for root, dirs, files in os.walk(directory):
        for filename in files:
            if filename.endswith(".csv"):
                file_path = os.path.join(root, filename)
                with open(file_path) as f:
                     reader = csv.reader(f)

                    #Checks the rows/columns and tries to find the specified string that marks the start of data

                    for i, row in enumerate(reader):
                        target_string = "Some Target String"
                        if target_string in row:

                        #Skips to the next line (where the data starts) when the target string is found

                            true_target_row = next(reader)
                            for row in reader:

                                #Counter tells you which file number is importing. 

                                print "importing file number %d" %(count)

                                #Creates a bunch of objects

                                objects = [
                                    Data(
                                        SomeObject = row [n],
                                        ...
                                        ...
                                )
                                for row in reader if row[0] != "Some Target End String"
                            ]

                            # Does a bulk_create after the file is completed

                            Data.objects.bulk_create(objects)
                            count += 1

            # Prints something if the file isn't a csv file

            else:
                print "This isn't a csv file"

    print "Imported %d files! All done!" %(count)

    return render(request, "Data/DataLandingPage.html")

CSV未导入django模型

3 个答案: