使用虚拟路径提供程序和Amazon S3 SDK上载文件

时间:2012-11-19 22:33:14

标签: c# amazon-s3

此问题的背景基于我正在开发的虚拟文件系统。我正在使用的概念是针对不同类型的存储类型的虚拟路径提供程序,即本地文件系统,dropbox和amazon s3。我的虚拟文件基类如下所示:

public abstract class CommonVirtualFile : VirtualFile {
    public virtual string Url {
        get { throw new NotImplementedException(); }
    }
    public virtual string LocalPath {
        get { throw new NotImplementedException(); }
    }
    public override Stream Open() {
        throw new NotImplementedException();
    }
    public virtual Stream Open(FileMode fileMode) {
        throw new NotImplementedException();
    }
    protected CommonVirtualFile(string virtualPath) : base(virtualPath) { }
}

第二个Open方法的实现就是我的问题。如果我们查看本地文件系统的实现,即在磁盘上保存文件,它看起来像这样:

public override Stream Open(FileMode fileMode) {
    return new FileStream("The_Path_To_The_File_On_Disk"), fileMode);
}

如果我想在本地文件系统上保存文件,则看起来像这样:

    const string virtualPath = "/assets/newFile.txt";
    var file = HostingEnvironment.VirtualPathProvider.GetFile(virtualPath) as CommonVirtualFile;
    if (file == null) {
        var virtualDir = VirtualPathUtility.GetDirectory(virtualPath);
        var directory = HostingEnvironment.VirtualPathProvider.GetDirectory(virtualDir) as CommonVirtualDirectory;
        file = directory.CreateFile(VirtualPathUtility.GetFileName(virtualPath));
    }
    byte[] fileContent;
    using (var fileStream = new FileStream(@"c:\temp\fileToCopy.txt", FileMode.Open, FileAccess.Read)) {
        fileContent = new byte[fileStream.Length];
        fileStream.Read(fileContent, 0, fileContent.Length);
    }
    // write the content to the local file system
    using (Stream stream = file.Open(FileMode.Create)) {
        stream.Write(fileContent, 0, fileContent.Length);
    }

我想要的是,如果我切换到我的亚马逊s3虚拟路径提供程序,我希望此代码直接工作而不做任何更改,所以总结一下,如何使用amazon s3 sdk解决这个问题,我应该如何实现我的在我的amazon s3虚拟路径提供程序中打开(FileMode fileMode)方法?

1 个答案:

答案 0 :(得分:1)

嘿,我也支持这个问题,我解决了它实现一个流。

这是我做的方式,也许有帮助:

public static Stream OpenStream(S3TransferUtility transferUtility, string key)
    {                     
        byte[] buffer  = new byte[Buffersize + Buffersize/2];

        S3CopyMemoryStream s3CopyStream =
            new S3CopyMemoryStream(key, buffer, transferUtility)
            .WithS3CopyFileStreamEvent(CreateMultiPartS3Blob);

        return s3CopyStream;
    }

我的Stream with constructor覆盖了close和write(array,offset,count)方法,并将流部分上传到了amazon s3。

public class S3CopyMemoryStream : MemoryStream
    {

        public S3CopyMemoryStream WithS3CopyFileStreamEvent(StartUploadS3CopyFileStreamEvent doing)
        {
            S3CopyMemoryStream s3CopyStream = new S3CopyMemoryStream(this._key, this._buffer, this._transferUtility);

            s3CopyStream.StartUploadS3FileStreamEvent = new S3CopyMemoryStream.StartUploadS3CopyFileStreamEvent(CreateMultiPartS3Blob);

            return s3CopyStream;
        }

        public S3CopyMemoryStream(string key, byte[] buffer, S3TransferUtility transferUtility)
            : base(buffer)
        {
            if (buffer.LongLength > int.MaxValue)
                throw new ArgumentException("The length of the buffer may not be longer than int.MaxValue", "buffer");

            InitiatingPart = true;
            EndOfPart = false;
            WriteCount = 1;
            PartETagCollection = new List<PartETag>();

            _buffer = buffer;
            _key = key;
            _transferUtility = transferUtility;
        }

事件StartUploadS3FileStreamEvent调用启动,uploadpart和完成上传的调用。

或者你可以实现一个更容易的FileStream,因为你可以使用

TransferUtilityUploadRequest request =
            new TransferUtilityUploadRequest()
            .WithAutoCloseStream(false).WithBucketName(
                transferUtility.BucketName)
                .WithKey(key)
                .WithPartSize(stream.PartSize)
                .WithInputStream(stream) as TransferUtilityUploadRequest;

        transferUtility.Upload(request);

在重写FileStream的close方法。缺点是你必须先将整个数据写入磁盘,然后才能上传它。