仅将更新的文件上传到Blob存储

时间:2018-10-02 09:52:13

标签: c# azure azure-functions azure-blob-storage

我有一种方法可以将XML文件从文件夹上载到Blob存储中。连接到该Blob存储,我有一个Blob触发器,它可以监听Blob存储中的更改,获取文件,然后向服务器发出PUT请求。我弄清楚了,然后开始工作。

我的问题是,当我想更新文件夹中的特定文件并运行我的代码时,该文件夹中的所有文件似乎都被重新上传,我的Blob触发器消失了,对所有文件进行PUT。我只想对文件夹中更改过的文件进行PUT(当然,除了我最初上传到Blob的内容外)。

到目前为止,我的代码与我的经验水平一样基本。对于导入,我遵循一个简单的指南。

将文件上传到Blob存储的代码:

class Program
{
    static void Main(string[] args)
    {
        Console.WriteLine("Starting...");
        string accountName = ConfigurationManager.AppSettings["accountName"];
        string accountKey = ConfigurationManager.AppSettings["accountKey"];
        string localFolder = ConfigurationManager.AppSettings["mySourceFolder"];
        string destContainer = ConfigurationManager.AppSettings["destContainer"];

        var stringReturned = BlobSetup(accountName, accountKey, localFolder, destContainer);

        Console.WriteLine(stringReturned);
        Console.Read();

    }
    static async Task UploadBlob(CloudBlobContainer container, string key, string filePath, bool deleteAfter)
    {
        //Get a blob reference to write this file to
        var blob = container.GetBlockBlobReference(key);

        await blob.UploadFromFileAsync(filePath);           

        Console.WriteLine("Uploaded {0}", filePath);
        //if delete of file is requested, do that
        if (deleteAfter)
        {
            File.Delete(filePath);
        }
    }
    static async Task<string> BlobSetup(string accountName, string accountKey, string localFolder, string destContainer)
     {
        var storageAccount = new CloudStorageAccount(new StorageCredentials(accountName, accountKey), true);
        var blobClient = storageAccount.CreateCloudBlobClient();

        var container = blobClient.GetContainerReference(destContainer);
        //create container if not exists
        await container.CreateIfNotExistsAsync();
        await container.SetPermissionsAsync(new BlobContainerPermissions()
        {
            PublicAccess = BlobContainerPublicAccessType.Blob
        });


        string[] fileEntries = Directory.GetFiles(localFolder);        
        foreach (string filePath in fileEntries)
        {
            //Handle only json and xml? 
            if(filePath.EndsWith(".json") || filePath.EndsWith(".xml"))
            {                  
                string keys = Path.GetFileName(filePath);

                await UploadBlob(container, keys, filePath, false);
            }

        }
        return "some response";
    }

执行PUT的我的Blob触发器:

 public static class BlobTriggerExample
{
    const string serverUrl= "theurl";
    static HttpClient client = new HttpClient();

    [FunctionName("BlobTriggerExample")]
    public static async Task Run([BlobTrigger("myblob/{name}", Connection = "AzureWebJobsStorage")]CloudBlockBlob myBlob, string name, TraceWriter log)
    {          
        string putUrlString = "";
        string idValue = "";
        XDocument xdoc = new XDocument();
        myBlob.StreamMinimumReadSizeInBytes = 20 * 1024 * 1024;
        await myBlob.FetchAttributesAsync();
        //Read stream
        var blobStream = await myBlob.OpenReadAsync();

            xdoc = new XDocument(XDocument.Load(blobStream));

        //Read root node(resourceType)
        string resourceType = xdoc.Root.Name.LocalName;

        //Get id value
        idValue = xdoc.Descendants().Where(x => x.Name.LocalName == "id").First().LastAttribute.Value;

        //Build redirect string
        putUrlString = serverUrl + resourceType + "/" + idValue;

        //PUT
        var httpContent = new StringContent(xdoc.ToString(), Encoding.UTF8, "application/xml");
        var response = await client.PutAsync(putUrlString, httpContent);    

        Console.WriteLine($"Response: {response}");
        Console.Read();

        log.Info($"C# Blob trigger function Processed blob\n Name:{name} \n Size: {myBlob.StreamWriteSizeInBytes} Bytes : Response message: {response}");
    }
}

我的猜测是,我希望能够通过执行某种检查是否已经存在完全相同的文件来控制要上传到Blob存储中的文件。还是我想在执行PUT之前在Blob触发器中进行某种检查?

我要从其上载的文件夹中的文件名始终相同(必须),即使某些内容可能已更改。

有没有人愿意为我提供一些有关如何实现这一目标的准则?我已经闲逛了几个小时,却一无所获。

1 个答案:

答案 0 :(得分:1)

是的,您的代码循环遍历并上载了本地文件夹中的所有文件。 Blob触发器仅看到Blob已被写入,并且不知道其内容是否已更改(或是否重要),因此它也处理所有Blob。

您需要做的是在上传之前将本地文件与blob存储中的文件进行比较,以查看它们是否为新版本,因此在UploadBlob方法中,您需要以下内容

// Get a blob reference to write this file to
var blob = container.GetBlockBlobReference(key);
// If the blob already exists
if (await blob.ExistsAsync()) {
    // Fetch the blob's properties
    await blob.FetchAttributesAsync();
    // Only proceed if modification time of local file is newer
    if (blob.Properties.LastModified > File.GetLastWriteTimeUtc(filePath))
        return;
}

如果仅检查修改时间还不够,那么您还可以将自己的元数据(例如校验和)附加到Blob并将其用于比较-请参见https://docs.microsoft.com/en-us/azure/storage/blobs/storage-properties-metadata

相关问题