我引用了网站https://github.com/toddkitta/azure-content/blob/master/articles/data-lake-analytics/data-lake-analytics-get-started-net-sdk.md来提交U-SQL作业,但上面的GitHub示例在Authentication AcquireToken方法中显示控制台应用程序中的错误。
请建议我使用.Net SDK
提交U-SQL作业的示例答案 0 :(得分:3)
正如Rick van den Bosch提到我们需要使用AcquireTokenAsync方法,我也按照文档进行了演示。并改变我身边的一些代码,然后它在我身边正常工作。以下是我的详细步骤。
<强>制备强>:
1.Registry原生AD应用程序和添加 Windows Azure服务管理API 权限,更多详细信息,请参阅Azure official tutorials。之后,我们可以从Azure门户获取 tenantId , appId ,重定向URI 。
2.创建Data Lake Analytics和Data Lake Store帐户。我允许访问数据湖商店中的文件夹或文件,更多详细信息请参阅另一个SO thread。
3.在本地文件夹中准备一个脚本文件,我从你提到的document获取脚本。
C:\Tom\SampleUSQLScript.txt
4.将SearchLog.tsv上载到Azure存储帐户并将其设置为公共
<强>步骤:强>
1.创建一个控制台项目并引用相应的SDK,详情请参阅packages.config部分。
2.添加如何获取TokenCredentials功能
public static TokenCredentials AuthenticateUser(string tenantId, string resource, string appClientId, Uri appRedirectUri, string userId = "")
{
var authContext = new AuthenticationContext("https://login.microsoftonline.com/" + tenantId);
var tokenAuthResult = authContext.AcquireTokenAsync(resource, appClientId, appRedirectUri, new PlatformParameters(PromptBehavior.Auto),
UserIdentifier.AnyUser).Result;
return new TokenCredentials(tokenAuthResult.AccessToken);
}
测试TokenCredentials功能
3.添加SetupClients功能
public static void SetupClients(TokenCredentials tokenCreds, string subscriptionId)
{
_adlaClient = new DataLakeAnalyticsAccountManagementClient(tokenCreds) {SubscriptionId = subscriptionId};
_adlaJobClient = new DataLakeAnalyticsJobManagementClient(tokenCreds);
_adlaCatalogClient = new DataLakeAnalyticsCatalogManagementClient(tokenCreds);
_adlsClient = new DataLakeStoreAccountManagementClient(tokenCreds) {SubscriptionId = subscriptionId};
_adlsFileSystemClient = new DataLakeStoreFileSystemManagementClient(tokenCreds);
}
4.添加SubmitJobByPath功能
public static string SubmitJobByPath(string scriptPath, string jobName)
{
var script = File.ReadAllText(scriptPath);
var jobId = Guid.NewGuid();
var properties = new USqlJobProperties(script);
var parameters = new JobInformation(jobName, JobType.USql, properties, priority: 1000, degreeOfParallelism: 1);
var jobInfo = _adlaJobClient.Job.Create(_adlaAccountName,jobId, parameters);
return jobId.ToString();
}
5.添加其他相关功能
public static void UploadFile(string srcFilePath, string destFilePath, bool force = true)
{
var parameters = new UploadParameters(srcFilePath, destFilePath, _adlsAccountName, isOverwrite: force);
var frontend = new DataLakeStoreFrontEndAdapter(_adlsAccountName, _adlsFileSystemClient);
var uploader = new DataLakeStoreUploader(parameters, frontend);
uploader.Execute();
}
// Download file
public static void DownloadFile(string srcPath, string destPath)
{
var stream = _adlsFileSystemClient.FileSystem.Open(srcPath, _adlsAccountName);
var fileStream = new FileStream(destPath, FileMode.Create);
stream.CopyTo(fileStream);
fileStream.Close();
stream.Close();
}
public static JobResult WaitForJob(string jobId)
{
var jobInfo = _adlaJobClient.Job.Get(_adlaAccountName,Guid.Parse(jobId));
while (jobInfo.State != JobState.Ended)
{
jobInfo = _adlaJobClient.Job.Get(_adlaAccountName, Guid.Parse(jobId));
}
return jobInfo.Result.Value;
}
public static void WaitForNewline(string reason, string nextAction = "")
{
if (!String.IsNullOrWhiteSpace(nextAction))
{
Console.WriteLine(reason + "\r\nPress ENTER to continue...");
Console.ReadLine();
Console.WriteLine(nextAction);
}
else
{
Console.WriteLine(reason + "\r\nPress ENTER to continue...");
Console.ReadLine();
}
6.添加测试提交工作代码。
private static void Main(string[] args)
{
_adlsAccountName = "data lake store account"; // TODO: Replace this value with the name for a created Store account.
_adlaAccountName = "data lake analytics"; // TODO: Replace this value with the name for a created Analytics account.
string localFolderPath = @"C:\tom\"; // TODO: Make sure this exists and contains the U-SQL script.
// Authenticate the user
// For more information about applications and instructions on how to get a client ID, see:
// https://azure.microsoft.com/en-us/documentation/articles/resource-group-create-service-principal-portal/
var tokenCreds = AuthenticateUser("common", "https://management.core.windows.net/",
"application id", new Uri("http://localhost")); // TODO: Replace applicaion id and redirect url values.
SetupClients(tokenCreds, "subscription id"); // TODO: Replace subscription value.
// Run sample scenarios
// Transfer the source file from a public Azure Blob container to Data Lake Store.
CloudBlockBlob blob = new CloudBlockBlob(new Uri("https://tomnew.blob.core.windows.net/adls-sample-data/SearchLog.tsv"));
blob.DownloadToFile(localFolderPath + "SearchLog.tsv", FileMode.Create); // from WASB
UploadFile(localFolderPath + "SearchLog.tsv", "/mytempdir/SearchLog.tsv"); // to ADLS
WaitForNewline("Source data file prepared.", "Submitting a job.");
// Submit the job
string jobId = SubmitJobByPath(localFolderPath + "SampleUSQLScript.txt", "My First ADLA Job");
WaitForNewline("Job submitted.", "Waiting for job completion.");
// Wait for job completion
WaitForJob(jobId);
WaitForNewline("Job completed.", "Downloading job output.");
}
7.从当地调试并检查天蓝门户的结果。
packages.config
<?xml version="1.0" encoding="utf-8"?>
<packages>
<package id="Microsoft.Azure.KeyVault.Core" version="1.0.0" targetFramework="net46" />
<package id="Microsoft.Azure.Management.DataLake.Analytics" version="3.0.0" targetFramework="net46" />
<package id="Microsoft.Azure.Management.DataLake.Store" version="1.0.4" targetFramework="net46" />
<package id="Microsoft.Azure.Management.DataLake.StoreUploader" version="1.0.1-preview" targetFramework="net46" />
<package id="Microsoft.Data.Edm" version="5.8.2" targetFramework="net46" />
<package id="Microsoft.Data.OData" version="5.8.2" targetFramework="net46" />
<package id="Microsoft.Data.Services.Client" version="5.8.2" targetFramework="net46" />
<package id="Microsoft.IdentityModel.Clients.ActiveDirectory" version="3.14.1" targetFramework="net46" />
<package id="Microsoft.Rest.ClientRuntime" version="2.3.8" targetFramework="net46" />
<package id="Microsoft.Rest.ClientRuntime.Azure" version="3.3.7" targetFramework="net46" />
<package id="Newtonsoft.Json" version="6.0.8" targetFramework="net46" />
<package id="System.ComponentModel.EventBasedAsync" version="4.0.11" targetFramework="net46" />
<package id="System.Dynamic.Runtime" version="4.0.0" targetFramework="net46" />
<package id="System.Linq.Queryable" version="4.0.0" targetFramework="net46" />
<package id="System.Net.Requests" version="4.0.11" targetFramework="net46" />
<package id="System.Spatial" version="5.8.2" targetFramework="net46" />
<package id="WindowsAzure.Storage" version="8.1.4" targetFramework="net46" />
</packages>
<强>更新强>
如果我们想使用静默登录,我们可以使用以下代码来获取tokenCreds var tokenCreds = AuthenticateSlientUser("https://management.core.windows.net/", tenantId, applicationId, secretKey)
public static TokenCredentials AuthenticateSlientUser(string resource,string tenantId, string appClientId, string secretKey)
{
var authContext = new AuthenticationContext("https://login.microsoftonline.com/" + tenantId);
var tokenAuthResult = authContext.AcquireTokenAsync(resource, new ClientCredential(appClientId, secretKey)).Result;
return new TokenCredentials(tokenAuthResult.AccessToken);
}
答案 1 :(得分:0)
来自msdn的样本没有AcquireTokenAsync方法 - Arron
有时文档滞后。他们可能更新了这个软件包,但还没有更新文档。使用Async方法,你会没事的。
您可以轻松使用AcquireTokenAsync
方法。如下所述称之为。这将等待异步方法并返回结果,即使在同步方法中也是如此。
var token = authContext.AcquireTokenAsync(parameters).Result;
有关要使用的参数,请查看文档:{{3}}
您可以在线找到多个工作代码示例。 AuthenticationContext.AcquireTokenAsync Method