将大文件上传到AWS S3时出现TaskCanceledException

时间:2018-09-16 18:48:50

标签: amazon-web-services amazon-s3 asp.net-core task

我正在尝试将一个较大的视频文件(800mb)上传到我的S3存储桶中,但似乎超时。它适用于较小的文件。我的项目是一个ASP.Net Core 2.1应用程序。

这是抛出的异常:

  

处理请求时发生未处理的异常。   SocketException:由于线程退出或应用程序请求,I / O操作已中止。   未知位置

     

IOException:无法从传输连接中读取数据:由于线程退出或应用程序请求,I / O操作已中止。   System.Net.Sockets.Socket + AwaitableSocketAsyncEventArgs.ThrowException(SocketError错误)

     

TaskCanceledException:操作已取消。   AWSS3Helper.cs中的GamerPilot.Video.AWSS3Helper.UploadFileAsync(流流,字符串键,S3CannedACL acl,bool useReducedRedundancy,bool throwOnError,CancellationToken cancelleToken),行770

我的源代码如下:

  public async Task<IVideo> AddVideoAsync(int instructorId, int lectureId, string videoName, string filePath, CancellationToken cancellationToken = default(CancellationToken))
    {
        if (string.IsNullOrEmpty(filePath)) { throw new ArgumentNullException("filePath", "Video filepath is missing"); }
        if (!File.Exists(filePath)) { throw new ArgumentNullException("filePath", "Video filepath does not exists"); }

        //Test video file upload and db row insertion
        using (var stream = File.OpenRead(filePath))
        {
            return await AddVideoAsync(instructorId, lectureId, videoName, stream, cancellationToken);
        }
    }

 public async Task<IVideo> AddVideoAsync(int instructorId, int lectureId, string videoName, Stream videoFile, CancellationToken cancellationToken = default(CancellationToken))
    {
        var video = (Video) await GamerPilot.Video.Helper.Create(_awsS3AccessKey, _awsS3SecretKey, _awsS3BucketName, _awsS3Region)
            .AddVideoAsync(instructorId, lectureId, videoName, videoFile, cancellationToken);

        using (var db = new DbContext(_connectionString))
        {
            db.Videos.Add(video);
            var count = await db.SaveChangesAsync();
        }

        return video;
    }`

 public async Task<IVideo> AddVideoAsync(int instructorId, int lectureId, string videoName, Stream videoFile, CancellationToken cancellationToken = default(CancellationToken))
    {
        if (string.IsNullOrEmpty(videoName)) { throw new ArgumentNullException("videoName", "Video name cannot be empty or null"); }
        if (videoFile == null) { throw new ArgumentNullException("video", "Video stream is missing"); }

        var videoNameCleaned = videoName.Replace(" ", "-").ToLower().Replace(".mp4", "");
        var videoKey = string.Join('/', "videos", instructorId, lectureId, videoNameCleaned + ".mp4");

        using (var aws = new AWSS3Helper(_awsS3AccessKey, _awsS3SecretKey, _awsS3BucketName, _awsS3Region))
        {
            try
            {
                //THIS FAILS ------
                await aws.UploadFileAsync(videoFile, videoKey, Amazon.S3.S3CannedACL.PublicRead, true, true, cancellationToken);

            }
            catch (Exception ex)
            {

                throw;
            }
        }

        return new Video
        {
            InstructorId = instructorId,
            LectureId = lectureId,
            Name = videoName,
            S3Key = videoKey,
            S3Region = _awsS3Region.SystemName,
            S3Bucket = _awsS3BucketName,
            Created = DateTime.Now
        };
    }

我该如何解决?

1 个答案:

答案 0 :(得分:0)

S3本身没有一般约束,可以阻止您上载800MB文件。但是,使用AWS时需要处理重试和超时。从您的问题尚不清楚您是否在使用Amazon的SDK,(我找不到GamerPilot.Video.AWSS3Helper.UploadFileAsync的来源)。但是,如果您按照以下条件使用Amazon .NET的SDK,则应该为您处理此问题:

Programming with the AWS SDK for .NET - Retries and Timeouts

Using the AWS SDK for .NET for Multipart Upload (High-Level API)