.NET异步流读/写
我一直在尝试解决这个“并发编程”考试练习（在C＃中）：
知道
Stream类包含int Read(byte[] buffer, int offset, int size)和void Write(byte[] buffer, int offset, int size)方法，在C＃中实现NetToFile方法，该方法将从NetworkStream net个实例收到的所有数据复制到{ {1}}实例。要进行传输，请使用异步读取和同步写入，以避免在读取操作期间阻塞一个线程。当FileStream file读取操作返回值0时，传输结束。为简化起见，不必支持控制取消操作。
net我一直试图解决这个问题，但我正在努力解决与问题本身相关的问题。但首先，这是我的代码：
void NetToFile(NetworkStream net, FileStream file);
我的问题是在问题陈述中说：
要进行传输，请使用异步 读取和同步写入，避免 读取期间要阻止的一个线程 操作
我不确定我的解决方案是否能完成本练习中所需的内容，因为我使用public static void NetToFile(NetworkStream net, FileStream file) {
byte[] buffer = new byte[4096]; // buffer with 4 kB dimension
int offset = 0; // read/write offset
int nBytesRead = 0; // number of bytes read on each cycle
IAsyncResult ar;
do {
// read partial content of net (asynchronously)
ar = net.BeginRead(buffer,offset,buffer.Length,null,null);
// wait until read is completed
ar.AsyncWaitHandle.WaitOne();
// get number of bytes read on each cycle
nBytesRead = net.EndRead(ar);
// write partial content to file (synchronously)
fs.Write(buffer,offset,nBytesRead);
// update offset
offset += nBytesRead;
}
while( nBytesRead > 0);
}
等待异步读取完成。
另一方面，我并没有真正弄清楚在这种情况下什么是“非阻塞”解决方案，因为AsyncWaitHandle.WaitOne()写入是同步的......并且要做到这一点，我必须等到FileStream读完后继续NetworkStream写作，不是吗？
请你帮帮我吗？
[编辑1] 使用回调解决方案
好的，如果我理解Mitchel Sellers和willvv回复的内容，我会被建议使用回调方法将其转换为“非阻塞”解决方案。这是我的代码，然后：
FileStream您可能已经注意到，我坚持使用回调方法，因为我没有引用byte[] buffer; // buffer
public static void NetToFile(NetworkStream net, FileStream file) {
// buffer with same dimension as file stream data
buffer = new byte[file.Length];
//start asynchronous read
net.BeginRead(buffer,0,buffer.Length,OnEndRead,net);
}
//asynchronous callback
static void OnEndRead(IAsyncResult ar) {
//NetworkStream retrieve
NetworkStream net = (NetworkStream) ar.IAsyncState;
//get number of bytes read
int nBytesRead = net.EndRead(ar);
//write content to file
//... and now, how do I write to FileStream instance without
//having its reference??
//fs.Write(buffer,0,nBytesRead);
}
实例，我想调用“Write（...）”方法。
此外，这不是一个线程安全的解决方案，因为FileStream字段已公开，并且可能在并发的byte[]调用之间共享。我不知道如何在外部范围内暴露这个NetToFile字段来解决这个问题......我几乎可以肯定它不会以这种方式暴露出来。
我不想使用lambda或匿名方法解决方案，因为这不在“并发编程”课程的课程中。
6 个答案:
答案 0 :(得分：52)
尽管这可能会影响人们的功课，但考虑到这已经超过一年了，这是实现这一目标的正确方法。您只需要重叠您的读/写操作 - 不需要产生额外的线程，或者需要任何其他线程。
public static class StreamExtensions
{
private const int DEFAULT_BUFFER_SIZE = short.MaxValue ; // +32767
public static void CopyTo( this Stream input , Stream output )
{
input.CopyTo( output , DEFAULT_BUFFER_SIZE ) ;
return ;
}
public static void CopyTo( this Stream input , Stream output , int bufferSize )
{
if ( !input.CanRead ) throw new InvalidOperationException( "input must be open for reading" );
if ( !output.CanWrite ) throw new InvalidOperationException( "output must be open for writing" );
byte[][] buf = { new byte[bufferSize] , new byte[bufferSize] } ;
int[] bufl = { 0 , 0 } ;
int bufno = 0 ;
IAsyncResult read = input.BeginRead( buf[bufno] , 0 , buf[bufno].Length , null , null ) ;
IAsyncResult write = null ;
while ( true )
{
// wait for the read operation to complete
read.AsyncWaitHandle.WaitOne() ;
bufl[bufno] = input.EndRead(read) ;
// if zero bytes read, the copy is complete
if ( bufl[bufno] == 0 )
{
break ;
}
// wait for the in-flight write operation, if one exists, to complete
// the only time one won't exist is after the very first read operation completes
if ( write != null )
{
write.AsyncWaitHandle.WaitOne() ;
output.EndWrite(write) ;
}
// start the new write operation
write = output.BeginWrite( buf[bufno] , 0 , bufl[bufno] , null , null ) ;
// toggle the current, in-use buffer
// and start the read operation on the new buffer.
//
// Changed to use XOR to toggle between 0 and 1.
// A little speedier than using a ternary expression.
bufno ^= 1 ; // bufno = ( bufno == 0 ? 1 : 0 ) ;
read = input.BeginRead( buf[bufno] , 0 , buf[bufno].Length , null , null ) ;
}
// wait for the final in-flight write operation, if one exists, to complete
// the only time one won't exist is if the input stream is empty.
if ( write != null )
{
write.AsyncWaitHandle.WaitOne() ;
output.EndWrite(write) ;
}
output.Flush() ;
// return to the caller ;
return ;
}
public static async Task CopyToAsync( this Stream input , Stream output )
{
await input.CopyToAsync( output , DEFAULT_BUFFER_SIZE ) ;
return;
}
public static async Task CopyToAsync( this Stream input , Stream output , int bufferSize )
{
if ( !input.CanRead ) throw new InvalidOperationException( "input must be open for reading" );
if ( !output.CanWrite ) throw new InvalidOperationException( "output must be open for writing" );
byte[][] buf = { new byte[bufferSize] , new byte[bufferSize] } ;
int[] bufl = { 0 , 0 } ;
int bufno = 0 ;
Task<int> read = input.ReadAsync( buf[bufno] , 0 , buf[bufno].Length ) ;
Task write = null ;
while ( true )
{
await read ;
bufl[bufno] = read.Result ;
// if zero bytes read, the copy is complete
if ( bufl[bufno] == 0 )
{
break;
}
// wait for the in-flight write operation, if one exists, to complete
// the only time one won't exist is after the very first read operation completes
if ( write != null )
{
await write ;
}
// start the new write operation
write = output.WriteAsync( buf[bufno] , 0 , bufl[bufno] ) ;
// toggle the current, in-use buffer
// and start the read operation on the new buffer.
//
// Changed to use XOR to toggle between 0 and 1.
// A little speedier than using a ternary expression.
bufno ^= 1; // bufno = ( bufno == 0 ? 1 : 0 ) ;
read = input.ReadAsync( buf[bufno] , 0 , buf[bufno].Length );
}
// wait for the final in-flight write operation, if one exists, to complete
// the only time one won't exist is if the input stream is empty.
if ( write != null )
{
await write;
}
output.Flush();
// return to the caller ;
return;
}
}
干杯。
答案 1 :(得分：16)
我怀疑这是最快的代码（.NET任务抽象有一些开销），但我认为这是整个异步复制事件的更清晰方法。
我需要一个CopyTransformAsync我可以传递一个委托做一些事情，因为块通过了复制操作。例如复制时计算消息摘要。这就是为什么我有兴趣推出自己的选择。
<强>调查结果：
- CopyToAsync bufferSize是敏感的（需要大缓冲区）
- FileOptions.Asynchronous - ＆gt;使它非常缓慢（不确定为什么会这样）
- FileStream对象的bufferSize可以更小（并不重要）
-
Serial测试显然是最快且资源最耗费的
这是我发现的内容和the complete source code我用来测试它的程序。在我的机器上，这些测试是在SSD磁盘上运行的，相当于文件副本。通常情况下，你不想只使用它来复制文件，而是当你有一个网络流（这就是我的用例），那就是你想要使用这样的东西。
4K buffer
Serial... in 0.474s
CopyToAsync... timed out
CopyToAsync (Asynchronous)... timed out
CopyTransformAsync... timed out
CopyTransformAsync (Asynchronous)... timed out
8K buffer
Serial... in 0.344s
CopyToAsync... timed out
CopyToAsync (Asynchronous)... timed out
CopyTransformAsync... in 1.116s
CopyTransformAsync (Asynchronous)... timed out
40K buffer
Serial... in 0.195s
CopyToAsync... in 0.624s
CopyToAsync (Asynchronous)... timed out
CopyTransformAsync... in 0.378s
CopyTransformAsync (Asynchronous)... timed out
80K buffer
Serial... in 0.190s
CopyToAsync... in 0.355s
CopyToAsync (Asynchronous)... in 1.196s
CopyTransformAsync... in 0.300s
CopyTransformAsync (Asynchronous)... in 0.886s
160K buffer
Serial... in 0.432s
CopyToAsync... in 0.252s
CopyToAsync (Asynchronous)... in 0.454s
CopyTransformAsync... in 0.447s
CopyTransformAsync (Asynchronous)... in 0.555s
在这里，您可以看到Process Explorer，运行测试时的性能图。基本上每个 top （在三个图中较低的一个）是串行测试的开始。随着缓冲区大小的增加，您可以清楚地看到吞吐量如何显着增加。它似乎计划在80K左右，这是.NET框架CopyToAsync方法在内部使用的。
这里的好处是最终的实现并不复杂：
static Task CompletedTask = ((Task)Task.FromResult(0));
static async Task CopyTransformAsync(Stream inputStream
, Stream outputStream
, Func<ArraySegment<byte>, ArraySegment<byte>> transform = null
)
{
var temp = new byte[bufferSize];
var temp2 = new byte[bufferSize];
int i = 0;
var readTask = inputStream
.ReadAsync(temp, 0, bufferSize)
.ConfigureAwait(false);
var writeTask = CompletedTask.ConfigureAwait(false);
for (; ; )
{
// synchronize read
int read = await readTask;
if (read == 0)
{
break;
}
if (i++ > 0)
{
// synchronize write
await writeTask;
}
var chunk = new ArraySegment<byte>(temp, 0, read);
// do transform (if any)
if (!(transform == null))
{
chunk = transform(chunk);
}
// queue write
writeTask = outputStream
.WriteAsync(chunk.Array, chunk.Offset, chunk.Count)
.ConfigureAwait(false);
// queue read
readTask = inputStream
.ReadAsync(temp2, 0, bufferSize)
.ConfigureAwait(false);
// swap buffer
var temp3 = temp;
temp = temp2;
temp2 = temp3;
}
await writeTask; // complete any lingering write task
}
尽管存在巨大的缓冲区，但这种交错读/写方法的速度比BCL CopyToAsync快18％。
出于好奇，我确实将异步调用更改为典型的开始/结束异步模式调用，并且没有改善一点情况，这使情况变得更糟。对于我喜欢抨击任务抽象开销的所有内容，当你用async / await关键字编写代码时，它们会做一些漂亮的事情，并且读取该代码要好得多！
答案 2 :(得分：12)
您将需要使用NetStream读取中的回调来处理此问题。坦率地说，将复制逻辑包装到自己的类中可能更容易，这样您就可以维护活动Streams的实例。
这是我接近它的方式（未经测试）：
public class Assignment1
{
public static void NetToFile(NetworkStream net, FileStream file)
{
var copier = new AsyncStreamCopier(net, file);
copier.Start();
}
public static void NetToFile_Option2(NetworkStream net, FileStream file)
{
var completedEvent = new ManualResetEvent(false);
// copy as usual but listen for completion
var copier = new AsyncStreamCopier(net, file);
copier.Completed += (s, e) => completedEvent.Set();
copier.Start();
completedEvent.WaitOne();
}
/// <summary>
/// The Async Copier class reads the input Stream Async and writes Synchronously
/// </summary>
public class AsyncStreamCopier
{
public event EventHandler Completed;
private readonly Stream input;
private readonly Stream output;
private byte[] buffer = new byte[4096];
public AsyncStreamCopier(Stream input, Stream output)
{
this.input = input;
this.output = output;
}
public void Start()
{
GetNextChunk();
}
private void GetNextChunk()
{
input.BeginRead(buffer, 0, buffer.Length, InputReadComplete, null);
}
private void InputReadComplete(IAsyncResult ar)
{
// input read asynchronously completed
int bytesRead = input.EndRead(ar);
if (bytesRead == 0)
{
RaiseCompleted();
return;
}
// write synchronously
output.Write(buffer, 0, bytesRead);
// get next
GetNextChunk();
}
private void RaiseCompleted()
{
if (Completed != null)
{
Completed(this, EventArgs.Empty);
}
}
}
}
答案 3 :(得分：11)
/// <summary>
/// Copies a stream.
/// </summary>
/// <param name="source">The stream containing the source data.</param>
/// <param name="target">The stream that will receive the source data.</param>
/// <remarks>
/// This function copies until no more can be read from the stream
/// and does not close the stream when done.<br/>
/// Read and write are performed simultaneously to improve throughput.<br/>
/// If no data can be read for 60 seconds, the copy will time-out.
/// </remarks>
public static void CopyStream(Stream source, Stream target)
{
// This stream copy supports a source-read happening at the same time
// as target-write. A simpler implementation would be to use just
// Write() instead of BeginWrite(), at the cost of speed.
byte[] readbuffer = new byte[4096];
byte[] writebuffer = new byte[4096];
IAsyncResult asyncResult = null;
for (; ; )
{
// Read data into the readbuffer. The previous call to BeginWrite, if any,
// is executing in the background..
int read = source.Read(readbuffer, 0, readbuffer.Length);
// Ok, we have read some data and we're ready to write it, so wait here
// to make sure that the previous write is done before we write again.
if (asyncResult != null)
{
// This should work down to ~0.01kb/sec
asyncResult.AsyncWaitHandle.WaitOne(60000);
target.EndWrite(asyncResult); // Last step to the 'write'.
if (!asyncResult.IsCompleted) // Make sure the write really completed.
throw new IOException("Stream write failed.");
}
if (read <= 0)
return; // source stream says we're done - nothing else to read.
// Swap the read and write buffers so we can write what we read, and we can
// use the then use the other buffer for our next read.
byte[] tbuf = writebuffer;
writebuffer = readbuffer;
readbuffer = tbuf;
// Asynchronously write the data, asyncResult.AsyncWaitHandle will
// be set when done.
asyncResult = target.BeginWrite(writebuffer, 0, read, null, null);
}
}
答案 4 :(得分：9)
没有人提到TPL，这很奇怪
Here PFX团队（Stephen Toub）关于如何实现并发异步流副本的非常好的帖子。该帖子包含对样本的过时重新发布，所以这里是正确的一个：
获取Parallel Extensions Extras from code.msdn然后
var task = sourceStream.CopyStreamToStreamAsync(destinationStream);
// do what you want with the task, for example wait when it finishes:
task.Wait();
另请考虑使用J.Richer的AsyncEnumerator。
答案 5 :(得分：0)
你是对的，你正在做的基本上是同步读取，因为你使用WaitOne（）方法，它只是停止执行，直到数据准备好，这基本上与使用Read（）代替它of BeginRead（）和EndRead（）。
你需要做的是，在BeginRead（）方法中使用callback参数，用它来定义一个回调方法（或一个lambda表达式），当读取信息时将调用此方法（在回调方法你必须检查流的结尾，并写入输出流），这样你就不会阻塞主线程（你不需要WaitOne（）和EndRead（）。
希望这有帮助。
- 我写了这段代码，但我无法理解我的错误
- 我无法从一个代码实例的列表中删除 None 值，但我可以在另一个实例中。为什么它适用于一个细分市场而不适用于另一个细分市场？
- 是否有可能使 loadstring 不可能等于打印？卢阿
- java中的random.expovariate()
- Appscript 通过会议在 Google 日历中发送电子邮件和创建活动
- 为什么我的 Onclick 箭头功能在 React 中不起作用？
- 在此代码中是否有使用“this”的替代方法？
- 在 SQL Server 和 PostgreSQL 上查询，我如何从第一个表获得第二个表的可视化
- 每千个数字得到
- 更新了城市边界 KML 文件的来源？