Question

实际上我必须创建大量线程来使用UDP协议发送pcap文件，当线程完全发送pcap文件然后睡眠一段时间。当我创建超过3100个线程并且程序抛出OutOfMemoryException时，我将线程休眠到420秒虚拟内存已满。

我搜索了互联网有关此问题，但发现一个线程只需要1MB创建和pcap文件只有60KB，而我的3100线程消耗超过12GB（1.06 * 3100 <12GB）。另一方面物理内存的使用不超过200MB。我必须同时创建超过5000个线程

我在做错了什么？有谁可以帮助我？

感谢

我的代码：

public static void send_pcap_file_with_single_port()
    {
        string callID = Call_ID;
        try
        {
            //CREATING CONNECTION HERE
            using (FileStream stream = new FileStream("g711a.pcap", FileMode.Open, FileAccess.ReadWrite, FileShare.ReadWrite))
            {


                for (Pos = 0; Pos < (StreamBytes - ChunkSize); Pos += ChunkSize)
                {
                    //creating RTP_header here

                    stream.Read(RTP_payload, 0, ChunkSize);
                    //combining both the byte arrays
                    System.Buffer.BlockCopy(RTP_header, 0, Bytes_to_send, 0, RTP_header.Length);
                    System.Buffer.BlockCopy(RTP_payload, 0, Bytes_to_send, 16, RTP_payload.Length);

                    RTPpacket_queue.Enqueue(Bytes_to_send);
                    //RTP_handler.Send(Bytes_to_send, Bytes_to_send.Length, remote_EP);
                }

                //done processing here
                stream.Close();
                stream.Dispose();

                RTP_count++;
                GC.Collect();
            }
            System.Threading.Thread.Sleep(420000);


        }
        catch (Exception e)
        {
            //using (StreamWriter sw = new StreamWriter(stream_logFile))
            //{
            //    sw.WriteLine(e.ToString());
            //}
            //send_BYE_message_toSIPp(client, "BYE", 5060, 2, callID);
            Console.WriteLine(e.ToString());
        }
    }

在这里创建线程：

Thread RTP_sender = new Thread(new ThreadStart(send_pcap_file_with_single_port));
                            RTP_sender.Start();

Answer 1

简单来说，你可以通过在长期堆中创建物体来耗尽垃圾收集器（物体可以存活超过几秒钟）。修复将是在需要时释放并重新创建线程。

在任何情况下，默认情况下i5有2个内核，如果你有3个或更多线程，那么它们在同一个cpu上运行它们。运行3000+它们意味着每个1500，除非他们试图在同一个地方写（如果他们开始像地狱一样阻止），这不是问题。

Answer 2

为了证明你不需要5000个永久线程来完成这样的事情，我已经创建了一个示例程序。

该程序并没有做太多的事情，但它所做的是它创建了5000个对象，每个对象在需要完成其工作时创建一个线程。除了简单地按照随机间隔睡觉之外，还没有做任何实际工作。

只需运行程序，让它运行一段时间并密切关注其内存使用情况。你会看到它非常容易管理，同时仍然可以在5000个对象上工作。

您可能需要在将这种方法应用于您的情况时真正具有创造性，但您可以按照我正在做的事情做一些事情。

namespace Test
{
    using System;
    using System.Collections.Generic;
    using System.ComponentModel;
    using System.Diagnostics;
    using System.Threading;

    public static class MainClass
    {
        public static Random sleeper = new Random ();

        public static void Main (string[] args)
        {
            Stopwatch timer = new Stopwatch ();
            List<WorkerClass> workload = new List<WorkerClass> ();

            // Create a workload of 5000 objects
            for (int i = 0; i < 5000; i++) {
                workload.Add (new WorkerClass ());
            }

            int fires = 0;
            // Start processing the workload
            while (true) {
                // We'll measure the time it took to go through the entire workload
                // to illustrate that it does not take all that long.
                timer.Restart ();
                foreach (WorkerClass w in workload) {
                    // for each of the worker objects in the entire workload
                    // we decrease its internal counter by 1.
                    // Because after the loop is done, we sleep for 1 secondd
                    // that amounts to reducing the counter by 1 every second.
                    w.counter--;
                    if (w.counter == 0) {
                        fires++;
                        // Once the counter hits 0, do the work.
                        w.DoWork ();
                    }
                }
                timer.Stop ();
                Console.WriteLine ("Processing the entire workload of {0} objects took {1} milliseconds, {2} workers actually fired.", workload.Count, timer.ElapsedMilliseconds, fires);
                fires = 0;
                Thread.Sleep (1000);
            }
        }
    }

    public class WorkerClass
    {
        public int counter = 0;

        public WorkerClass ()
        {
            // When the worker is created, set its internal counter
            // to a random value between 5 and 10.
            // This is to mimic sleeping it for a random interval.
            // Also see the primary loop in MainClass.Main
            this.counter = MainClass.sleeper.Next (5, 10);
        }

        public void DoWork ()
        {
            // Whenever we do the work, we'll create a background worker thread
            // that actually does the work.
            BackgroundWorker work = new BackgroundWorker ();
            work.RunWorkerCompleted += (object sender, RunWorkerCompletedEventArgs e) => {
                // This simulates going back to sleep for a random interval, see
                // the main loop in MainClass.Main
                this.counter = MainClass.sleeper.Next (5, 10);
            };
            work.DoWork += (object sender, DoWorkEventArgs e) => {
                // Simulate working by sleeping a random interval
                Thread.Sleep (MainClass.sleeper.Next (2000, 5000));
            };
            // And now we actually do the work.
            work.RunWorkerAsync ();
        }
    }
}

outofmemoryexception：线程使用过多的虚拟内存，大约12GB

2 个答案: