行为类似于线程的C#可重用或持久性任务

时间:2019-05-23 19:48:48

标签: c# multithreading asynchronous task tpl-dataflow

使用线程,您可以创建持久的,可重用的局部变量,这些变量对于诸如客户端连接之类的事情很有用。但是,对于诸如System.Threading.Tasks.Dataflow中的ActionBlock之类的任务,似乎没有任何形式的持久性或可重用性。因此,对于涉及与客户端交互的ActionBlock,我的理解是,您需要从头开始初始化客户端连接,或者需要在更高范围内重用客户端连接(带有锁定?)。

用例:我正在使用一个反转控件的.NET库。大部分逻辑(除了启动和关闭之外)必须在一个名为ProcessEventsAsync的Task方法中,由库调用,该方法接收IEnumerable数据。 ProcessEventsAsync必须处理所有数据,然后将其发送给某些下游使用者。为了提高性能,我正在尝试使用Tasks并行化ProcessEventsAsync中的逻辑。我还想从此任务中收集一些性能指标。

让我详细说明我在做什么:

internal class MyClass
{

  private String firstDownStreamConnectionString;
  private String secondDownStreamConnectionString;
  private SomeClient firstClient;
  private SomeClient secondClient;
  private ReportingClient reportingClient;
  private int totalUnhandledDataCount;

  public MyClass(String firstDownStreamConnectionString, String secondDownStreamConnectionString, String reportingClientKey)
  {
      this.firstDownStreamConnectionString = firstDownStreamConnectionString;
      this.secondDownStreamConnectionString = secondDownStreamConnectionString;
      this.DegreeOfParallelism = Math.Max(Environment.ProcessorCount - 1, 1);
      this.reportingClient = new ReportingClient (reportingClientKey, DegreeOfParallelism);
      this.totalUnhandledDataCount = 0;
  }
  // called once when the framework signals that processing is about to be ready
  public override async Task OpenAsync(CancellationToken cancellationToken, PartitionContext context)
  {
    this.firstClient = SomeClient.CreateFromConnectionString(this.firstDownStreamConnectionString);
    this.secondClient = SomeClient.CreateFromConnectionString(this.secondDownStreamConnectionString );
    await Task.Yield();
  }

  // this is called repeatedly by the framework
  // outside of startup and shutdown, it is the only entrypoint to my logic
  public override async Task ProcessEventsAsync(CancellationToken cancellationToken, PartitionContext context, IEnumerable<Data> inputData)
  {
    ActionBlock<List<Data>> processorActionBlock = new ActionBlock<List<Data>>(
      inputData =>
      {
        SomeData firstDataset = new SomeData();
        SomeData secondDataset = new SomeData();
        int unhandledDataCount = 0;
        foreach (Data data in inputData)
        {
          // if data fits one set of criteria, put it in firstDataSet
          // if data fits other set of criteria, put it in secondDataSet
          // otherwise increment unhandledDataCount
        }
        Interlocked.Add(ref this.totalUnhandledDataCount, unhandledDataCount);
        lock (this.firstClient)
        {
          try
          {
            firstDataset.SendData(this.firstClient);
          } catch (Exception e)
          {
            lock(this.reportingClient)
            {
              this.reportingClient.LogTrace(e);
            }
          }
        }
        lock (this.secondClient)
        {
          try
          {
            secondDataset.SendData(this.secondClient);
          } catch (Exception e)
          {
            lock(this.reportingClient)
            {
              this.reportingClient.LogTrace(e);
            }
          }
        }
      },
      new ExecutionDataflowBlockOptions
      {
        MaxDegreeOfParallelism = this.DegreeOfParallelism
      });
    // construct as many List<Data> from inputData as there is DegreeOfParallelism
    // put that in a variable called batches
    for(int i = 0; i < DegreeOfParallelism; i++)
    {
      processorActionBlock.Post(batches[i]);
    }
    processorActionBlock.Complete();
    processorActionBlock.Completion.Wait();
    await context.CheckpointAsync();
  }
}

我试图只保留相关代码,省略了处理逻辑,大多数度量收集,数据发送方式,关闭逻辑等。

我想利用某种Task的风格,以实现可重用性。我既不想为所有正在运行的此类任务重用单个客户端连接,也不想让每个任务每次调用时都创建一个新的客户端连接。我确实希望每个类似线程的任务都具有一组持久的客户端连接。理想情况下,我也不想创建一个包装Task或扩展System.Threading.Tasks.Dataflow中抽象类/接口的新类。

2 个答案:

答案 0 :(得分:0)

您所描述的内容听起来像是异步委托或Func。

例如:

Func<Task> TestFunc = async () =>
{
    Console.WriteLine("Begin");
    await Task.Delay(100);
    Console.WriteLine("Delay");
    await Task.Delay(100);
    Console.WriteLine("End");
};

如果该函数在范围内,则只需:

await TestFunc();

您可以根据需要多次重复使用它。您还可以更改功能以接受参数。


编辑

您也可以尝试AsyncLocal 。根据文档:

  

由于基于任务的异步编程模型倾向于抽象化线程的使用,因此AsyncLocal实例可用于跨线程持久化数据。

     

当与当前线程关联的值更改时,AsyncLocal类还提供可选的通知,这是因为它是通过设置Value属性来显式更改的,或者是在线程遇到等待或其他上下文转换时隐式更改的。

答案 1 :(得分:0)

听起来您只需要一个用于存储依赖项的类?

void Main()
{
    var doer1 = new ThingDoer();
    var doer2 = new ThingDoer();

    // A & B use one pair of clients, and C & D use another pair
    var taskA = doer1.DoTheThing();
    var taskB = doer1.DoTheThing();

    var taskC = doer2.DoTheThing();
    var taskD = doer2.DoTheThing();
}

public class ThingDoer
{
    private SomeClient _someClient;
    private SomeErrorReportingClient _someErrorReportingClient;

    public ThingDoer(SomeClient someClient, SomeErrorReportingClient someErrorReportingClient)
    {
        _someClient = someClient;
        _someErrorReportingClient = someErrorReportingClient;
    }

    public ThingDoer()
        : this(new SomeClient, new SomeErrorReportingClient)
    {
    }

    public async Task DoTheThing()
    {
        // Implementation here
    }
}

“可重用性”的概念与任务并不完全兼容。