多个生产者消费者同时在C#中工作

时间:2015-02-10 11:51:48

标签: c# .net multithreading

我想了解C#中生产者/消费者模式的最通用用法。 以下是我想到的条件:

  • 制作人不断推进数据。
  • 消费者不断地获取(处理和删除)数据。
  • 两种操作的速度可以比另一种操作的速度高得多。
  • 没有两个消费者应该获得相同的数据。

我知道TPL,.Net 4.x等提供了很多类来处理这种情况。但是在深入研究所有这些课程之前,必须知道他们解决了什么问题。如果有人能指出我使用传统的Monitor / lock和Thread类解决这个问题的文章/资源/示例,我将不胜感激。

很有可能有人跑来跑去喊这个问题不是建设性的,或者到目前为止我做了什么。请相信我,我做了一些谷歌搜索,找不到符合上述条件的所有条件。所有示例都是单个生产者或单个消费者或使用较新的类。如果我想解决一个疯狂的问题,请原谅。但请提及为什么你认为这是错误的。

我不是在寻找勺子喂食的答案或实施,因为我知道它需要大量的代码。我只是在寻找能找到帮助我实现的东西的指针。如果你的善意(即时间)允许,我会感激一点解释!

感谢阅读。

2 个答案:

答案 0 :(得分:2)

这是我躺在的一些古老的测试代码。它使用.Net 3.x时代类来实现线程安全的生产者/消费者队列。

因为它太旧了,它有一些粗糙的位 - 比如公开的SyncLock属性。我绝不会在实际代码中公开披露这个内容!

但是,这段代码至少说明了使用基本结构实现的工作生产者/消费者队列。

using System;
using System.Collections;
using System.Collections.Generic;
using System.Threading;

namespace Demo
{
    public sealed class ProducerConsumerQueue<T>: IEnumerable<T>
    {
        /// <summary>Has the queue been closed?</summary>

        public bool IsClosed
        {
            get
            {
                return _isClosed;
            }
        }

        /// <summary>The object used to synchronize access to the queue.</summary>

        public object SyncLock
        {
            get
            {
                return _syncLock;
            }
        }

        /// <summary>Tell the queue that no more items will be added.</summary>
        /// <param name="waitUntilEmpty">Wait until the queue is empty before returning?</param>

        public void Close(bool waitUntilEmpty)
        {
            lock (_syncLock)
            {
                _isClosed = true;
                Monitor.PulseAll(_syncLock);     // Wake up all consumers.

                if (waitUntilEmpty)
                {
                    while (_queue.Count > 0)
                    {
                        Monitor.Wait(_syncLock);
                    }
                }
            }
        }

        /// <summary>Adds an item the the queue.</summary>
        /// <param name="item">The item to be added.</param>

        public void Add(T item)
        {
            lock (_syncLock)
            {
                if (_isClosed)
                {
                    throw new InvalidOperationException("The queue has been closed.");
                }

                _queue.Enqueue(item);

                if (_queue.Count == 1)
                {
                    Monitor.Pulse(_syncLock);    // Added first item to the queue; wake up a consumer.
                }
            }
        }

        /// <summary>Typesafe Enumerator access to the queue items.</summary>

        public IEnumerator<T> GetEnumerator()
        {
            while (true)
            {
                T item;

                lock (_syncLock)
                {
                    if (_queue.Count > 0)
                    {
                        item = _queue.Dequeue();

                        if (_isClosed && (_queue.Count == 0))
                        {
                            Monitor.PulseAll(_syncLock);    // Tell producer we're done.
                        }
                    }
                    else if (_isClosed)
                    {
                        yield break;                // All done.
                    }
                    else
                    {
                        Monitor.Wait(_syncLock);    // Waits for another item to enter the queue or the queue to be closed.
                        continue;                   // Back to "while".
                    }
                }

                yield return item;  // Yield outside of the lock.
            }
        }

        /// <summary>Non-typesafe Enumerator access to the queue items.</summary>

        IEnumerator IEnumerable.GetEnumerator()
        {
            return GetEnumerator();
        }


        private readonly Queue<T> _queue = new Queue<T>();
        private readonly object _syncLock = new object();

        private bool _isClosed;
    }

    public static class Program
    {
        private static void Main()
        {
            ProducerConsumerQueue<string> queue = new ProducerConsumerQueue<string>();

            // Spawn first consumer thread.

            ThreadPool.QueueUserWorkItem
            (
                delegate
                {
                    foreach (string item in queue)
                    {
                        Console.WriteLine("Consumer 1 is consuming: " + item);
                        randomSleep(200); // simulate business
                    }

                    Console.WriteLine("Consumer 1 exited cleanly.");
                }
            );

            // Spawn second consumer thread.

            ThreadPool.QueueUserWorkItem
            (
                delegate
                {
                    foreach (string item in queue)
                    {
                        Console.WriteLine("Consumer 2 is consuming: " + item);
                        randomSleep(250); // simulate business
                    }

                    Console.WriteLine("Consumer 2 exited cleanly.");
                }
            );

            // Spawn third consumer thread.

            ThreadPool.QueueUserWorkItem
            (
                delegate
                {
                    foreach (string item in queue)
                    {
                        Console.WriteLine("Consumer 3 is consuming: " + item);
                        randomSleep(300); // simulate business
                    }

                    Console.WriteLine("Consumer 3 exited cleanly.");
                }
            );

            // Spawn first producer thread.

            ThreadPool.QueueUserWorkItem
            (
                delegate
                {
                    for (int i = 0;; i++)
                    {
                        lock (queue.SyncLock)
                        {
                            if (!queue.IsClosed)
                            {
                                string item = "Producer 1 Item " + i.ToString();
                                Console.WriteLine("Producer 1 is adding: " + item);
                                queue.Add(item);
                            }
                        }

                        if (i < 50)     // Slowly add the first 50.
                        {
                            randomSleep(500); // simulate business
                        }
                        else            // Quickly add the remainder to get a backlog.
                        {
                            randomSleep(150); // simulate business
                        }
                    }
                }
            );

            // Spawn second producer thread.

            ThreadPool.QueueUserWorkItem
            (
                delegate
                {
                    for (int i = 0;; i++)
                    {
                        lock (queue.SyncLock)
                        {
                            if (!queue.IsClosed)
                            {
                                string item = "Producer 2 Item " + i.ToString();
                                Console.WriteLine("Producer 2 is adding: " + item);
                                queue.Add(item);
                            }
                        }

                        if (i < 50)     // Slowly add the first 50.
                        {
                            randomSleep(600); // simulate business
                        }
                        else            // Quickly add the remainder to get a backlog.
                        {
                            randomSleep(120); // simulate business
                        }
                    }
                }
            );

            Thread.Sleep(20000);                        // Allow a few seconds for things to happen.
            Console.WriteLine("Closing queue...");
            queue.Close(true);
            Thread.Sleep(1000);
            Console.WriteLine("Press [return] to exit");
            Console.ReadLine();
        }

        private static void randomSleep(int max)
        {
            int delay;

            lock (_random)
            {
                delay = _random.Next(max + 100);
            }

            if (delay > 100)
            {
                Thread.Sleep(delay-100);
            }
        }

        static readonly Random _random = new Random();
    }
}

答案 1 :(得分:0)

您始终可以使用实际的队列系统。 ZeroMQ快速闪电并具有不错的.NET绑定。