读取用户上传的csv文件,并在heroku上以Django模式存储数据

时间:2017-10-08 13:34:38

标签: django csv heroku

这是我的第一个问题。所以我想感谢你的帮助。

我有一个Django School管理应用程序。我希望用户能够读取csv文件并使用特定标头存储在数据库中。

我的代码在本地运行得很好。但是我最近把它推到了heroku上,以便我可以测试它。我可能会注意到所有静态资产都存储在Amazon s3上。它有效。

但是当我尝试读取csv文件时,出现内部服务器错误。

这是我存储学生的代码。

def convert_header(csvHeader):
    cols = [ x.replace(' ', '_').lower() for x in csvHeader ]
    return cols

def import_data(request):
    if request.method == 'GET':
        return render(request, 'school/import.html')

    if request.method == 'POST' and request.FILES['myfile']:

        if request.POST.get("object") == '':
            message = 'You may chose an object'
            return render(request, 'school/import.html', {'message': message })

        if request.POST.get("object") == 'Pupil':
            myfile = request.FILES['myfile'] 

            fs = FileSystemStorage(location='eSchool/media/documents')
            filename = fs.save(myfile.name, myfile)
            uploaded_file_url = fs.path(filename)

            data = csv.reader(open(uploaded_file_url), delimiter=',')

            header = next(data)
            header_cols = convert_header(header)
            i = 0
            k = 0
            for row in data:
                pupil = Pupil()
                for k in range(len(header_cols)):
                    row_item = row[k].split(',')
                    for item in row_item:
                        key = header_cols[k]
                        if key == 'responsible':
                            item = Responsible.objects.get(pk= int(item))
                            print(item.first_name)
                            setattr(pupil, key, item)
                        else:
                            setattr(pupil, key, item)   
                    k +=1

                pupil.save()
                i = i + 1
            detailed = 'Sucessfully created '+ str(i) + ' Pupils '
            return render(request,  'school/import_success.html', {'detailed' : detailed })

我想将数据存储在名为Document的模型中。我创造它。并尝试它我仍然得到错误。请帮忙。

1 个答案:

答案 0 :(得分:0)

我找到了那个问题的解决方案。我首先创建一个Django模型来存储上传文件的url。那样:

class Document(models.Model):
    uploaded_at = models.DateTimeField(auto_now_add=True)
    upload = models.FileField(upload_to='documents')

显然我必须配置MEDIA_ROOT和MEDIA_URL 我配置设置文件以存储Amazon s3上所有媒体资产和所有静态文件的引用。

所以不要使用FileSystemStorage类。我用pip安装一个名为“requests”的包

pip安装请求

我导入了请求,csv和编解码器。

import requests
import codecs

之后我的观点我只是对我的功能进行了修改。

def import_data(request):
	if request.method == 'GET':
		return render(request, 'school/import.html')

	if request.method == 'POST' and request.FILES['myfile']:

		if request.POST.get("object") == '':
			message = 'You may chose an object'
			return render(request, 'school/import.html', {'message': message })

		if request.POST.get("object") == 'Pupil':
			myfile = request.FILES['myfile'] 
			doc = Document()
			doc.upload = myfile
			doc.save()
			print(doc.upload.url)
			if settings.DEBUG == True: # execute that block locally
				data = csv.reader(open(doc.upload.path), delimiter=',') # locally the file is red frm the path so i us doc.upload.path
			else: #this block is executed in production. So the file is red from the url. so I use doc.upload.url
				rep = requests.get(doc.upload.url)
				text = codecs.iterdecode(rep.iter_lines(), 'latin_1') #codecs is use to decode the lines. I don't really master how it works.
				data = csv.reader(text, delimiter = ',')
			header = next(data)
			header_cols = convert_header(header)
			i = 0
			k = 0
			for row in data:
				pupil = Pupil()
				for k in range(len(header_cols)):
					row_item = row[k].split(',')
					for item in row_item:
						key = header_cols[k]
						if key == 'responsible':
							item = Responsible.objects.get(pk= int(item))
							#print(item.first_name)
							setattr(pupil, key, item)
						else:
							setattr(pupil, key, item)	
					k +=1
				
				pupil.save()
				i = i + 1
			detailed = 'Sucessfully created '+ str(i) + ' Pupils '
			return render(request,  'school/import_success.html', {'detailed' : detailed })

所以现在。一切顺利。如果您看到我可以改进此代码的内容,我已经感谢您的帮助了!