Azure Data Lake商店通过C#脚本创建文件夹

时间:2017-06-22 06:49:16

标签: c# azure azure-data-lake

我正在尝试在datalake store内创建新文件夹,但代码中没有错误,但datalake store中没有反映任何内容。

代码:

using System;
using System.Collections.Generic;

using System.Text;
using System.Threading.Tasks;
using System.IO;
using System.Threading;

using Microsoft.Azure.Management.DataLake.Store;
using Microsoft.Azure.Management.DataLake.Store.Models;
using Microsoft.Rest.Azure.Authentication;
namespace test_dlstore
{
    class Program
    {
        private static DataLakeStoreAccountManagementClient _adlsClient;
        private static DataLakeStoreFileSystemManagementClient _adlsFileSystemClient;

        private static string _adlsAccountName;
        private static string _subId;
        private static void Main(string[] args)
        {
            _adlsAccountName = "mycreds@mysite.com"; 

            _subId = "2342342-97ce-a54b2-ba6e-234234234234234";

            string localFolderPath = @"C:\myfolder\"; // TODO: Make sure this exists and can be overwritten.
            string localFilePath = Path.Combine(localFolderPath, "try.txt");
            string remoteFolderPath = "adl://mystore.azuredatalakestore.net/myfolder";
            string remoteFilePath = Path.Combine(remoteFolderPath, "try.txt");

            SynchronizationContext.SetSynchronizationContext(new SynchronizationContext());
            var tenant_id = "my-tenant-id"; 
            var nativeClientApp_clientId = "1950a258-227b-4e31-a9cf-717495945fc2";


            var activeDirectoryClientSettings = ActiveDirectoryClientSettings.UsePromptOnly(nativeClientApp_clientId, new Uri("urn:ietf:wg:oauth:2.0:oob"));
            var creds = UserTokenProvider.LoginWithPromptAsync(tenant_id, activeDirectoryClientSettings).Result;

            _adlsClient = new DataLakeStoreAccountManagementClient(creds) { SubscriptionId = _subId };
            _adlsFileSystemClient = new DataLakeStoreFileSystemManagementClient(creds);
            Console.WriteLine(ListAdlStoreAccounts());
            Console.WriteLine(AppendToFile("adl://mystore.azuredatalakestore.net/myfolder/stage/testfile.txt", "abcdefghijklmnopqrstuvwxyz"));
            CreateDirectory("adl://mystore.azuredatalakestore.net/myfolder/newdir");

            Console.ReadLine();

        }
        // Append to file
        public static string AppendToFile(string path, string content)
        {
            using (var stream = new MemoryStream(Encoding.UTF8.GetBytes(content)))
            {
                Console.WriteLine(_adlsAccountName, path, content);
                Console.WriteLine(path);
                Console.WriteLine(content);
                _adlsFileSystemClient.FileSystem.AppendAsync(_adlsAccountName, path, stream);
                return "Tried";
            }
        }
        public static string CreateDirectory(string path)
        {
            _adlsFileSystemClient.FileSystem.MkdirsAsync(_adlsAccountName, path);
            return "Tried Creating directory.";
        }
}
}

执行上述代码后,程序退出且没有错误。正在建立联系。

此外,它还显示了datalake个商店,但这些商店在数据湖商店中根本无法执行任何操作。

我是azure的新手请帮助我。

1 个答案:

答案 0 :(得分:1)

我在我身边做了一个演示测试,它在侧面正常工作。以下是我的详细步骤:

<强>制备

注册AD应用程序并将角色分配给applcation ,请详细说明 参考Azure official tutorials。之后,我们可以从Azure门户获得 tenantId appId secretKey

<强>步骤:

1.创建一个C#控制台项目

2.参考Microsoft.Azure.Management.DataLake.Store SDK,更多详情请参阅packages.config文件部分。

3.添加以下代码

 var applicationId = "appid";
 var secretKey = "secretkey";
 var tenantId = "tenantid";
 var adlsAccountName = "adlsAccount Name";
 var creds = ApplicationTokenProvider.LoginSilentAsync(tenantId, applicationId, secretKey).Result;
 var adlsFileSystemClient = new DataLakeStoreFileSystemManagementClient(creds,clientTimeoutInMinutes:60); 
 adlsFileSystemClient.FileSystem.Mkdirs(adlsAccountName, "/tomtest/newfolder");

4.运行测试代码

enter image description here

5.从天蓝色的门户网站查看。

enter image description here

Packages.config

<?xml version="1.0" encoding="utf-8"?>
<packages>
  <package id="Microsoft.Azure.Management.DataLake.Store" version="2.2.0" targetFramework="net452" />
  <package id="Microsoft.IdentityModel.Clients.ActiveDirectory" version="3.13.8" targetFramework="net452" />
  <package id="Microsoft.Rest.ClientRuntime" version="2.3.8" targetFramework="net452" />
  <package id="Microsoft.Rest.ClientRuntime.Azure" version="3.3.7" targetFramework="net452" />
  <package id="Microsoft.Rest.ClientRuntime.Azure.Authentication" version="2.2.0-preview" targetFramework="net452" />
  <package id="Newtonsoft.Json" version="9.0.2-beta1" targetFramework="net452" />
</packages>