Question

我将Django与DRF结合使用，并且有一个视图集，该视图集将信息写入CSV文件，然后将链接发送给用户。

当检索到的数据不太大时，该功能可以正常工作，因此，在响应时间合理的情况下，一切正常。

问题在于请求需要更长的时间。似乎它只是从请求调用本身全部重新启动

    def get_operations(self, request):
        logger.info("Starting export request...")
   #[...business logic to retrive the data ...]
        file_url = settings.MEDIA_PREFIX_PATH + file_path
        logger.info("Saving response in %s", file_absolute_path)
        f = open(file_absolute_path, "w+")
        all_operations = operations.all()
        i = 0
        for operation in all_operations:
            i = i + 1
            #code to show progress in the log
            if (i / len(all_operations)*1000) % 5 == 0:
                logger.info("Progress:  %d ", (i / len(all_operations)*100))

            f.write("%s,%d,%d,%d,%s,%s\n" % (operation.datetime, operation.amount, operation.field2, operation.field3, operation.field5, operation.field6))
        logger.info("Response saved, sending link %s", file_url)
        return Response(file_url)

在日志中，这就是我得到的

INFO 2018-11-26 11:23:33,525 Starting export request...
INFO 2018-11-26 11:23:34,223 Response retrieved : 17010 records
INFO 2018-11-26 11:23:34,225 Saving response in /tmp/generated_csv/1543231414.2250094.csv
INFO 2018-11-26 11:23:42,825 Progress:  10 
INFO 2018-11-26 11:23:51,161 Progress:  20 
INFO 2018-11-26 11:23:59,072 Progress:  30 
INFO 2018-11-26 11:24:07,694 Starting export request...
INFO 2018-11-26 11:24:08,346 Response retrieved : 17010 records
INFO 2018-11-26 11:24:08,348 Saving response in /tmp/generated_csv/1543231448.3486001.csv
INFO 2018-11-26 11:24:16,653 Progress:  10 
INFO 2018-11-26 11:24:24,271 Progress:  20 
INFO 2018-11-26 11:24:32,248 Progress:  30 
INFO 2018-11-26 11:24:42,573 Starting export request...
INFO 2018-11-26 11:24:44,101 Response retrieved : 17010 records
INFO 2018-11-26 11:24:44,109 Saving response in /tmp/generated_csv/1543231484.1097722.csv
INFO 2018-11-26 11:24:55,092 Progress:  10 
INFO 2018-11-26 11:25:04,131 Progress:  20 
INFO 2018-11-26 11:25:11,242 Progress:  30 
INFO 2018-11-26 11:25:16,801 Starting export request...
INFO 2018-11-26 11:25:17,499 Response retrieved : 17010 records
INFO 2018-11-26 11:25:17,501 Saving response in /tmp/generated_csv/1543231517.5013359.csv

我确定问题不是来自客户端，因为我也尝试过邮递员。

我只是不明白是什么原因导致请求本身全部被调用。

谢谢

Answer 1

我猜测这不是Django或DRF的问题，很可能是gunicorn或您用于服务Django的问题。因此，请尝试使用任何异步过程，而不要在同步模式下执行此大文件操作。例如：

import threading 

...

def process_file(self, file_absolute_path, all_operations):
        f = open(file_absolute_path, "w+")
        i = 0
        for operation in all_operations:
            i = i + 1
            # it will be visible in your logger file, so you can see the progress
            if (i / len(all_operations)*1000) % 5 == 0:
                logger.info("Progress:  %d ", (i / len(all_operations)*100))
                f.write("%s,%d,%d,%d,%s,%s\n" % (operation.datetime, operation.amount, operation.field2, operation.field3, operation.field5, operation.field6))

def get_operations(self, request):
    logger.info("Starting export request...")
    all_operations = operations.all()
    file_url = settings.MEDIA_PREFIX_PATH + file_path
    t = threading.Thread(target=self.process_file,
                         args=(file_absolute_path, all_operations))
    t.start()
    logger.info("Saving response in %s", file_absolute_path)
    logger.info("Response saved, sending link %s", file_url)
    return Response(file_url)

在这里，我给出了使用Python的Threading的示例。但您也可以为此目的使用Celery（及其更好的解决方案）。

花费很长时间，Django drf请求似乎正在重新启动

1 个答案: