Question

我使用返回Stream的FileSystemOperationsExtensions.Open方法，我可以从中读取。有时，当服务从流中读取大文件（~150-300 Mb）时，服务会出现以下例外情况：

System.IO.IOException: The read operation failed, see inner exception. ---> System.Net.WebException: The request was aborted: The request was canceled.
at System.Net.ConnectStream.Read(Byte[] buffer, Int32 offset, Int32 size)
at System.Net.Http.HttpClientHandler.WebExceptionWrapperStream.Read(Byte[] buffer, Int32 offset, Int32 count)

"ClassName": "System.IO.IOException",
"Message": "Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host."
 at System.Net.ConnectStream.Read(Byte[] buffer, Int32 offset, Int32 size)\r\n   
 at System.Net.Http.HttpClientHandler.WebExceptionWrapperStream.Read(Byte[] buffer, Int32 offset, Int32 count)

它随机发生。此外，我创建了一个DataLakeStoreFileSystemManagementClient类的对象，超时60分钟，但这些错误发生在它之前。它可能需要3,10,20或任何分钟。当然，我可以使用偏移重新读取流，但它需要额外的时间进行开发。也许还有另一种方法可以避免这些例外。有人可以帮我吗？

Answer 1

我用270M +大小的文件做了3次演示测试，它总能正常运行。请尝试使用以下代码进行测试。我们还可以从data lake store get started net sdk获得更多数据存储演示代码。

演示代码：

new ProcessedType { Name = g2.Name, g2.MyCount }

包配置文件：

var applicationId = "Application Id";
                var secretKey = "secretkey";
                var tenantId = "tenant id";
                var adlsAccountName = "Account name";
                var creds = ApplicationTokenProvider.LoginSilentAsync(tenantId, applicationId, secretKey).Result;
                var adlsFileSystemClient = new DataLakeStoreFileSystemManagementClient(creds,clientTimeoutInMinutes:60);
                var srcPath = "/mytempdir/ForDemoCode.zip";
                var destPath = @"c:\tom\ForDemoCode1.zip";

                Stopwatch stopWatch = new Stopwatch();
                stopWatch.Start();
                using (var stream = adlsFileSystemClient.FileSystem.Open(adlsAccountName, srcPath))
                using (var fileStream = new FileStream(destPath, FileMode.Create))
                {
                    stream.CopyTo(fileStream);
                }
                var file = new FileInfo(destPath);
                Console.WriteLine($"File size :{file.Length}");
                stopWatch.Stop();
                // Get the elapsed time as a TimeSpan value.
                TimeSpan ts = stopWatch.Elapsed;
                // Format and display the TimeSpan value.
                string elapsedTime = $"{ts.Hours:00}:{ts.Minutes:00}:{ts.Seconds:00}.{ts.Milliseconds/10:00}";
                Console.WriteLine("RunTime " + elapsedTime);
                Console.ReadKey();

Azure Data Lake Store - 从文件读取时出错

1 个答案: