生产者/消费者通过队列获取不同的数据

时间:2015-07-18 11:11:55

标签: c# multithreading queue pass-by-reference thread-priority

解决!!!谢谢

请注意你在Queue中放入了什么样的对象。如果你输入一个值,比如int,那么enqueue会复制一下,每个人都很高兴。如果你把一个引用,比如byte [],string,enqueue把这个引用放到队列中,那么问题就来了。如果在消费者阅读之前更改了此引用,则消费者将阅读已更改版本的数据。

要避免此问题,请在rxThread中排队后立即获取新版本的帧引用。代码:

public void rxThreadFunc()
        {
         byte[] data = new byte[datalen];//declare for the first iteration.
            int j = 0;
            while (true)
            {

                for (int i = 0; i < rxlen; i++)
                {
                    data[j] = (byte)i;
                    j++;
                    if (j >= datalen)
                    {
                        j = 0;
                        mQ.Add(data);
                        using (StreamWriter fwriter = new StreamWriter("C:\\testsave\\rxdata", true))
                        {
                            for (int k = 0; k < datalen; k++)
                            {
                                fwriter.Write(data[k]);
                                fwriter.Write(",");
                            }
                            fwriter.Write("\n");
                        }
                     data = new byte[datalen];//create new reference after enqueue/Add
                    }
                }

            }
        }//rxThreadFunc()

UPDATE1 我刚刚编写了另一个更简单的代码,所以每个人都可以在没有serialport硬件的情单击按钮以运行程序。 我认为线程优先级导致了这个问题,在不改变rxthread的线程优先级的情况下,dataProc线程将获得正确的数据。但我还是不知道为什么。

rxThread.Priority = ThreadPriority.Hightest

using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Windows.Forms;
using System.Collections.Concurrent;
using System.Threading;
using System.IO;
namespace WindowsFormsApplication1
{
    public partial class Form1 : Form
    {
        public Form1()
        {
            InitializeComponent();

        }

        private void button1_Click(object sender, EventArgs e)
        {
            Thread rxThread = new Thread(rxThreadFunc);
            rxThread.Priority = ThreadPriority.Highest;//this causes problem
            rxThread.Start();
            Thread procThread = new Thread(dataProc);
            procThread.Start();
        }
        BlockingCollection<byte[]> mQ = new BlockingCollection<byte[]>();
        int datalen = 30;         
        int rxlen = 200;
        public void rxThreadFunc()
        {
            int j = 0;
            while (true)
            {
               byte[] data = new byte[datalen];//is this in the right place?
                for (int i = 0; i < rxlen; i++)
                {
                    data[j] = (byte)i;
                    j++;
                    if (j >= datalen)
                    {
                        j = 0;
                        mQ.Add(data);
                        using (StreamWriter fwriter = new StreamWriter("C:\\testsave\\rxdata", true))
                        {
                            for (int k = 0; k < datalen; k++)
                            {
                                fwriter.Write(data[k]);
                                fwriter.Write(",");
                            }
                            fwriter.Write("\n");
                        }
                    }
                }

            }
        }//rxThreadFunc()
        public void dataProc()
        {
            byte[] outData = new byte[datalen];
            while (true)
            {
                if (mQ.Count > 1)
                {
                    outData=mQ.Take();
                    using(StreamWriter fwriter=new StreamWriter("C:\\testsave\\dataProc",true))
                    {
                        for (int i = 0; i < datalen; i++)
                        {
                            fwriter.Write(outData[i]);
                            fwriter.Write(",");
                        }
                        fwriter.Write("\n");
                    }
                }
            }
        }

    }
}

说明 我写这个包含两个线程的应用程序。 RxThread从串口接收数据,按标题0x55 0xaa排序,将以下30个字节放入FrameStruct类,然后将此FrameStruct放入Queue。 dataProcess线程从Queue获取帧,然后将其存储到磁盘。

的SerialPort =(RXBUFF)=&GT; RxThread =(rxFrame,队列)=&GT; dataProcess ==&GT;磁盘

问题: 通过dataProcess线程接收并保存到磁盘的数据以某种方式被破坏。

尝试: 这是我试过的参考资料。

  1. 我尝试过BlockedCollection,它自然是线程安全的,而不是Queue,它仍然无法正常工作。所以我猜这不是Queue的问题。
  2. 我试图在FrameStruct中添加另一个成员int cnt,它在RxThread中的messageQ.Enqueue()之前自行增加。然后dataProcess线程可以正确地获取它。所以我认为数据[]可能有问题,但是......
  3. 但是我尝试将字节数据[30]而不是FrameStruct放入队列,不起作用。
  4. 另外我认为dataProcess收到的时间也是正确的。

    5。如果我在RxThread中的Monitor.Pulse()之后放了一个Thread.sleep(20),问题就解决了,但我不明白为什么???如果我换到另一台电脑怎么办?

  5. 这是代码快照。

    //declared:
    //Queue<FrameStruct>messageQ=new Queue<FrameStruct>;
    //object _LockerMQ=new object();
    private void RxThread()
        {
            int bytestoread, i;
            bool f55 = false;//55 flag
            bool fs = false;//frame start flag
            int j=0;//data index in FrameStruct
            int m_lMaxFram=32;
            bytestoread = 0;
            FrameStruct rxFrame = new FrameStruct((int)m_lMaxFrame);
            while (true)
            {
                if (Serial_Port.IsOpen == true)
                {
                    if ((bytestoread = Serial_Port.BytesToRead) > m_lMaxFrame*2)//get at least two frames
                    {
                            rxbuff = new byte[bytestoread];  
                            Serial_Port.Read(rxbuff, 0, bytestoread);
                            for (i = 0; i < bytestoread; i++)
                            {
                                if (rxbuff[i] == 0x55)
                                {
                                    f55 = true;
                                    continue;
                                }
                                if (rxbuff[i] == 0xaa && f55)
                                {//frame header 0x55, 0xaa
                                    //new frame start
                                    fs = true;
                                    f55 = false;
                                    j = 0;//rxframe index;
                                    rxFrame.time = DateTime.Now;//store the datetime when this thread gets this frame
                                    continue;
                                }
                                if (fs && j < m_lMaxFrame - 2)
                                {//frame started but not ended
                                    rxFrame.data[j] = rxbuff[i];
                                    j++;
                                }
                                if (j >= (m_lMaxFrame - 2) && fs)
                                {//frame ended if j=30, reaches the end of rxFrame.data
                                    fs = false;
                                    lock(_LockerMQ)
                                    {
                                        messageQ.Enqueue(rxFrame);
                                    Monitor.Pulse(_LockerMQ);
                                    }
                                     //Thread.Sleep(20);//if uncomment this sleep, problem solved
                                  using (StreamWriter fWriter = new StreamWriter("c:\\testsave\\RXdata", true))//save rxThread result into a file rawdata
                        {
                            fWriter.Write(rxFrame.time.ToString("yyyy/MM/dd HH:mm:ss.fff"));
                            fWriter.Write(",");
                            for (int k = 0; k < m_lMaxFrame - 2; k++)
                            {
                                fWriter.Write(rxFrame.data[k]);
                                fWriter.Write(",");
                            }
                            fWriter.Write("\n");
                        }
                                }
                               }
                    }//if ((bytestoread=Serial_Port.BytesToRead) > 0)
                    rxbuff = null;
                    Thread.Sleep(20);
                }//(Serial_Port.IsOpen==true)
                Thread.Sleep(100);
            }//while(true),RxThread sleep
        }//private void RxThread()
    

    DataProcess线程:

     public void dataProcess()
        {
          while (true)
            {
                lock (_LockerMQ)
                {
                    while (messageQ.Count < 1) Monitor.Wait(_LockerMQ);//get at least one frame data
                    f_NewFrame = messageQ.Count;
                    if (f_NewFrame > 0)
                    {
                        procFrame = messageQ.Dequeue();
                        using (StreamWriter fWriter = new StreamWriter("c:\\testsave\\dPdata", true))
                        {
                            fWriter.Write(procFrame.time.ToString("yyyy/MM/dd HH:mm:ss.fff"));
                            fWriter.Write(",");
                            for (int i = 0; i < m_lMaxFrame - 2; i++)
                            {
                                fWriter.Write(procFrame.data[i]);
                                fWriter.Write(",");
                            }
                            fWriter.Write("\n");
                        }
                    }//if(f_NewFrame>0)
    
                }//lock(messageQ)
      }
    }
    

    FrameStruct包含时间成员和数据[30]

    class FrameStruct
    {
            public FrameStruct(int m_lMaxFrame)
            {
                time = DateTime.Now;
                data = new byte[m_lMaxFrame - 2];
            }
            public DateTime time;
            public volatile byte[] data;//volatile doesn't help
    }
    

    RxThread保存的rxData是正确的,显示:

    2015/07/18 18:40:26.125,127,255,255,255,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,111,51,204,
    2015/07/18 18:40:26.177,128,0,255,255,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,112,51,204,
    2015/07/18 18:40:26.177,128,0,255,255,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,113,51,204,
    2015/07/18 18:40:26.297,127,255,255,255,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,114,51,204,
    2015/07/18 18:40:26.298,127,255,255,255,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,115,51,204,
    2015/07/18 18:40:26.298,127,255,255,255,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,116,51,204,
    2015/07/18 18:40:26.299,127,255,255,255,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,117,51,204,
    2015/07/18 18:40:26.420,127,255,255,255,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,118,51,204,
                                                                                         //^this columns is accumulated number
    

    由dataProcessThread保存的dPdata为WRONG,显示:

    2015/07/18 18:40:31.904,127,255,255,255,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,227,51,204,
    2015/07/18 18:40:31.905,127,255,255,255,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,228,51,204,
    2015/07/18 18:40:31.905,127,255,255,255,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,229,51,204,
    2015/07/18 18:40:32.026,128,0,255,255,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,231,51,204,
    2015/07/18 18:40:32.026,128,0,255,255,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,231,51,204,
    2015/07/18 18:40:32.147,128,0,255,255,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,232,51,204,
    2015/07/18 18:40:32.148,128,0,255,255,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,233,51,204,
    2015/07/18 18:40:32.148,128,0,255,255,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,234,51,204,
    2015/07/18 18:40:32.269,128,0,255,255,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,236,51,204,
    2015/07/18 18:40:32.269,128,0,255,255,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,236,51,204,
    2015/07/18 18:40:32.510,128,0,255,255,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,237,51,204,
    2015/07/18 18:40:32.512,128,0,255,255,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,240,51,204,
    2015/07/18 18:40:32.512,128,0,255,255,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,240,51,204,
    2015/07/18 18:40:32.514,127,255,255,255,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,240,51,204,
    2015/07/18 18:40:32.514,127,255,255,255,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,241,51,204,
    2015/07/18 18:40:32.635,128,0,255,255,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,243,51,204,
    2015/07/18 18:40:32.635,128,0,255,255,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,243,51,204,
                                                                                       //^this accumulated number is not correct
    

    请帮忙!

    谢谢!

2 个答案:

答案 0 :(得分:0)

FrameStruct是一个类(而不是结构),当你将它排队时,你反复使用相同的引用。

答案 1 :(得分:0)

更新2

这是基本的生产者消费者,只需添加你的逻辑:

class Program
{

    static BlockingCollection<int> mQ = new BlockingCollection<int>();

    static void Main(string[] args)
    {

        Thread rxThread = new Thread(rxThreadFunc);
        rxThread.Priority = ThreadPriority.Highest;//this causes problem
        rxThread.Start();
        Thread procThread = new Thread(dataProc);
        procThread.Start();
        Console.ReadLine();
    }


    static public void rxThreadFunc()
    {
        for (int i = 0; i < 10; i++)
        {
            mQ.Add(i);
        }
    }


    static public void dataProc()
    {
        foreach (int outData in mQ.GetConsumingEnumerable())
        {
            Console.WriteLine(outData);
        }
    }


}

回答更新1:

  • 是的,这是正确的地方

现在,由于您正在使用的BlockingCollection,使用者(dataProc)可能非常简单(只需删除循环和计数检查,它将为您执行所有这些同步):

 foreach (byte[] outData in _taskQ.GetConsumingEnumerable())
 {
     using(StreamWriter fwriter=new StreamWriter("C:\\testsave\\dataProc",true))
                {
                    for (int i = 0; i < datalen; i++)
                    {
                        fwriter.Write(outData[i]);
                        fwriter.Write(",");
                    }
                    fwriter.Write("\n");
                }
 }

现在,问题可能与原帖不同。也许是因为这里生产者也将数据写入文件?

<强>原始

它有很多解释,但解决方案很简单,只需将FrameStruct的初始化添加到第二个&#34; if&#34;语句:

rxFrame = new FrameStruct((int)m_lMaxFrame);

如您保存的数据文件所示:在损坏的文件中,每次缺少值时(例如230) - 您都有其他值重复(例如231)。缺失值的计数等于重复值的计数。

原因是您将同一对象实例的引用添加到队列中。 让我们看看下面的场景:RxThread循环N次,在上下文切换到dataProcess线程之前,它将N个参考添加到队列的同一个FrameStruct实例。此实例中的数据将是上下文切换之前的最后一次读取循环迭代的数据。 现在发生上下文切换:dataProcess循环M&lt;在上下文切换回RxThread之前N次,所以它从队列中读取M个元素,但它们都指向同一个实例,因此它将M次相同的行写入文件(最后一个如前所述)

现在,为什么Thread.Sleep有帮助。 简短的回答:每次通过RxThread将1个元素添加到队列时,上下文切换到dataProcess线程的概率非常高。实际上它是:读一个 - &gt;上下文切换 - &gt;写一个......再说一遍。

长答案是:在dataProcess线程执行Monitor.Wait后,它进入等待队列,上下文切换调度RxThread。现在,线程将第一个元素添加到队列中并执行Monitor.Pulse。这会将dataProcess线程移动到就绪队列。但不一定要安排它立即运行,因此RxThread可以进行另一次迭代。但是,如果你执行Thread.Sleep - 现在将安排上下文切换和ataProcess线程的概率非常高。