Question

我正在创建一个Google App Engine网络应用，以“转换”10K~50M的文件

示例：

用户在网络浏览器中打开http://fixdeck.appspot.com
用户点击“浏览”，选择文件，提交
Servlet将文件加载为InputStream
Servlet转换文件
Servlet将文件另存为OutputStream
用户的浏览器接收转换后的文件并询问保存位置，直接作为对步骤2中请求的响应

（现在我没有实现第4步，servlet将文件发回而不进行转换。）

问题：它适用于15MB文件但不适用于40MB文件，说：“错误：请求实体太大。您的客户发出的请求太大了。”

是否有针对此的解决方法？

源代码：https://github.com/nicolas-raoul/transdeck
理由：http://code.google.com/p/ankidroid/issues/detail?id=697 的

Answer 1

对于HTTP请求和HTTP响应，GAE的hard limits为32MB。这将直接限制上传/下载到GAE应用程序的大小。

修订后的答案（使用Blobstore API。）

Google向Blobstore API提供了处理GAE中较大文件（最高2GB）的权限。概述文档提供了完整的示例代码。您的Web表单会将文件上传到blobstore。然后blobstore API将POST重写回您的servlet，您可以在其中进行转换并将转换后的数据保存回blobstore（作为新blob）。

原始答案（未考虑Blobstore作为选项。）

对于下载，我认为GAE唯一的解决方法是将文件分解为服务器上的多个部分，然后在下载后重新组装。但是，使用直接浏览器实现可能并不可行。

（作为另一种设计，也许您可以将转换后的文件从GAE发送到外部下载位置（例如S3），浏览器可以将其下载而不受GAE限制。我不相信GAE发起的连接有相同的请求/响应大小限制，但我不是肯定的。无论如何，你仍然会受到30秒最长请求时间的限制。要解决这个问题，你必须查看GAE Backend instances并来采用某种异步下载策略。）

为了上传更大的文件，我已经了解了使用HTML5文件API将文件切片到多个块进行上传，然后在服务器上重建的可能性。示例：http://www.html5rocks.com/en/tutorials/file/dndfiles/#toc-slicing-files。但是，由于规格和浏览器功能的变化，我认为解决方案并不实用。

Answer 2

您可以使用the blobstore上传大小为2千兆字节的文件。

Answer 3

上传较大的文件时，您可以考虑将文件分块为Google App Engine支持的少量请求（应小于32MB，这是当前限制）。

通过示例检查此软件包-https://github.com/pionl/laravel-chunk-upload

以下是使用上述软件包的有效代码。

查看

<div id="resumable-drop" style="display: none">
        <p><button id="resumable-browse" class="btn btn-outline-primary" data-url="{{route('AddAttachments', Crypt::encrypt($rpt->DRAFT_ID))}}" style="width: 100%;
height: 91px;">Browse Report File..</button> 
    </div>

JavaScript

 <script>
var $fileUpload = $('#resumable-browse');
var $fileUploadDrop = $('#resumable-drop');
var $uploadList = $("#file-upload-list");

if ($fileUpload.length > 0 && $fileUploadDrop.length > 0) {
    var resumable = new Resumable({
        // Use chunk size that is smaller than your maximum limit due a resumable issue
        // https://github.com/23/resumable.js/issues/51
        chunkSize: 1 * 1024 * 1024, // 1MB
        simultaneousUploads: 3,
        testChunks: false,
        throttleProgressCallbacks: 1,
        // Get the url from data-url tag
        target: $fileUpload.data('url'),
        // Append token to the request - required for web routes
        query:{_token : $('input[name=_token]').val()}
    });

// Resumable.js isn't supported, fall back on a different method
    if (!resumable.support) {
        $('#resumable-error').show();
    } else {
        // Show a place for dropping/selecting files
        $fileUploadDrop.show();
        resumable.assignDrop($fileUpload[0]);
        resumable.assignBrowse($fileUploadDrop[0]);

        // Handle file add event
        resumable.on('fileAdded', function (file) {
            $("#resumable-browse").hide();
            // Show progress pabr
            $uploadList.show();
            // Show pause, hide resume
            $('.resumable-progress .progress-resume-link').hide();
            $('.resumable-progress .progress-pause-link').show();
            // Add the file to the list
            $uploadList.append('<li class="resumable-file-' + file.uniqueIdentifier + '">Uploading <span class="resumable-file-name"></span> <span class="resumable-file-progress"></span>');
            $('.resumable-file-' + file.uniqueIdentifier + ' .resumable-file-name').html(file.fileName);
            // Actually start the upload
            resumable.upload();
        });
        resumable.on('fileSuccess', function (file, message) {
            // Reflect that the file upload has completed
            location.reload();
        });
        resumable.on('fileError', function (file, message) {
             $("#resumable-browse").show();
            // Reflect that the file upload has resulted in error
            $('.resumable-file-' + file.uniqueIdentifier + ' .resumable-file-progress').html('(file could not be uploaded: ' + message + ')');
        });
        resumable.on('fileProgress', function (file) {
            // Handle progress for both the file and the overall upload
            $('.resumable-file-' + file.uniqueIdentifier + ' .resumable-file-progress').html(Math.floor(file.progress() * 100) + '%');
            $('.progress-bar').css({width: Math.floor(resumable.progress() * 100) + '%'});
        });
    }

}
</script>

控制器

 public function uploadAttachmentAsChunck(Request $request, $id) {
    // create the file receiver
    $receiver = new FileReceiver("file", $request, HandlerFactory::classFromRequest($request));

    // check if the upload is success, throw exception or return response you need
    if ($receiver->isUploaded() === false) {
        throw new UploadMissingFileException();
    }

    // receive the file
    $save = $receiver->receive();

    // check if the upload has finished (in chunk mode it will send smaller files)
    if ($save->isFinished()) {
        // save the file and return any response you need, current example uses `move` function. If you are
        // not using move, you need to manually delete the file by unlink($save->getFile()->getPathname())
        $file = $save->getFile();

        $fileName = $this->createFilename($file);
        // Group files by mime type
        $mime = str_replace('/', '-', $file->getMimeType());
        // Group files by the date (week
        $dateFolder = date("Y-m-W");

        $disk = Storage::disk('gcs');
        $gurl = $disk->put($fileName, $file);

        $draft = DB::table('draft')->where('DRAFT_ID','=', Crypt::decrypt($id))->get()->first();

        $prvAttachments = DB::table('attachments')->where('ATTACHMENT_ID','=', $draft->ATT_ID)->get();
        $seqId = sizeof($prvAttachments) + 1;

        //Save Submission Info
        DB::table('attachments')->insert(
            [   'ATTACHMENT_ID' => $draft->ATT_ID,
                'SEQ_ID' => $seqId,
                'ATT_TITLE' => $fileName,
                'ATT_DESCRIPTION' => $fileName,
                'ATT_FILE' => $gurl
            ]
        );

         return response()->json([
            'path' => 'gc',
            'name' => $fileName,
            'mime_type' => $mime,
            'ff' =>  $gurl
        ]);

       
    }

    // we are in chunk mode, lets send the current progress
    /** @var AbstractHandler $handler */
    $handler = $save->handler();

        return response()->json([
            "done" => $handler->getPercentageDone(),
        ]);
    }

    

    /**
     * Create unique filename for uploaded file
     * @param UploadedFile $file
     * @return string
     */
    protected function createFilename(UploadedFile $file)
    {
        $extension = $file->getClientOriginalExtension();
        $filename = str_replace(".".$extension, "", $file->getClientOriginalName()); // Filename without extension

        // Add timestamp hash to name of the file
        $filename .= "_" . md5(time()) . "." . $extension;

        return $filename;
    }

Answer 4

您还可以使用blobstore api直接上传到云存储。 Blow是链接

https://cloud.google.com/appengine/docs/python/blobstore/#Python_Using_the_Blobstore_API_with_Google_Cloud_Storage

upload_url = blobstore.create_upload_url(
  '/upload_handler',
  gs‌_bucket_name = YOUR.BUCKET_NAME)

template_values = { 'upload_url': upload_url } 
_jinjaEnvironment = jinjaEnvironment.JinjaClass.getJinjaEnvironemtVariable()

if _jinjaEnvironment: 
  template = _jinjaEnvironment.get_template('import.html')

然后在index.html：

<form action="{{ upload_url }}" 
      method="POST" 
      enctype="multipart/form-data">
  Upload File:
  <input type="file" name="file">
</form>

将大于40MB的文件上传到Google App Engine？

4 个答案: