Question

我使用python多处理技术和django forms将大量文件上传到服务器端。这是代码

def pooltest(request):
    file_list=request.FILES.getlist('docfile')
    print 'cpu_count() = %d\n' % cpu_count()
    total_no_files = len(request.FILES.getlist('docfile'))
    chunk_size = total_no_files//cpu_count()
    print chunk_size
    t1 = time.time()
    p = Pool()
    p.map(upload_function, file_list,chunksize=chunk_size)
    p.close()
    p.join()

但是当我尝试在发生错误后上传大量文件时

expected string or Unicode object, NoneType found

请帮忙解决这个问题吗？这个池配置是否正确？如何更改此设置以便我可以用最少的时间上传大量文件。

这是追溯

 expected string or Unicode object, NoneType found
Internal Server Error: /pooltest/
Traceback (most recent call last):
  File "/home/akhil/.virtualenvs/resumematch/local/lib/python2.7/site-packages/django/core/handlers/exception.py", line 39, in inner
    response = get_response(request)
  File "/home/akhil/.virtualenvs/resumematch/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 249, in _legacy_get_response
    response = self._get_response(request)
  File "/home/akhil/.virtualenvs/resumematch/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 198, in _get_response
    "returned None instead." % (callback.__module__, view_name)
ValueError: The view matchingapp.views.pooltest didn't return an HttpResponse object. It returned None instead.
2017-02-28 12:52:17,218 [Thread-90   ] [ERROR]  Internal Server Error: /pooltest/
Traceback (most recent call last):
  File "/home/akhil/.virtualenvs/resumematch/local/lib/python2.7/site-packages/django/core/handlers/exception.py", line 39, in inner
    response = get_response(request)
  File "/home/akhil/.virtualenvs/resumematch/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 249, in _legacy_get_response
    response = self._get_response(request)
  File "/home/akhil/.virtualenvs/resumematch/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 198, in _get_response
    "returned None instead." % (callback.__module__, view_name)
ValueError: The view matchingapp.views.pooltest didn't return an HttpResponse object. It returned None instead.

Answer 1

两件事：

您应该使用celery或<queue of your choice>而不是跳转到多处理。
您应该考虑到即使使用多处理，上传所有文件也需要一段时间，客户端将等待您的视图中的某些响应。我感觉到部队的一些超时。

而且，正如@ themanatuf2所说，你的视图函数应该返回一个值，模板或HttpResponse(status=200)

使用django表单和多处理编写文件

1 个答案: