如何使用并行foreach添加字符串而没有任何问题?

时间:2014-12-10 15:45:19

标签: c# string parallel-processing parallel.foreach

这是我目前的代码:

Parallel.ForEach(Arguments, Argument =>
{
    if (Argument != Command_Name)
    {
        WebRequest web_request = WebRequest.Create("https://www.aol.com/?command=1&domain=" + Argument);
        web_request.Timeout = 5000;
        ((HttpWebRequest)web_request).UserAgent = "Mozilla Firefox 5.0";
        HttpWebResponse web_response = (HttpWebResponse)web_request.GetResponse();
        StreamReader response = new StreamReader(web_response.GetResponseStream(), Encoding.UTF8);
        Message += Argument + " => " + response.ReadToEnd() + Environment.NewLine;
    }
});

此代码无法正常工作,我正在寻找一个小的替代方案。此代码返回Message字符串中的一些参数...多线程字符串添加的好方法是什么?这就是我需要的。

更多信息:消息字符串有时会返回a,b和c,而其他消息字符串只返回a或b ...

感谢您的任何帮助,谢谢!

5 个答案:

答案 0 :(得分:5)

您可以使用线程安全的集合来存储ConcurrentBag<T>之类的消息。然后在循环之后进行隐藏:

var messages = new ConcurrentBag<string>();
Parallel.ForEach(Arguments, Argument =>
{
   ... 
   messages.Add(Argument + " => " + response.ReadToEnd());
}
var result = string.Join(Environment.NewLine, messages);

答案 1 :(得分:1)

你可以锁定Message +=部分:

//scoped to the same scope as the Message object
private Object thisLock = new Object();

var textResult = response.ReadToEnd() + Environment.NewLine;
lock(thisLock)
{
    Message += Argument + " => " + textResult;
}

答案 2 :(得分:1)

对于收集元素AsParallel()更合适:

var lines = Arguments.AsParallel().Select(Argument =>
{
    if (Argument == Command_Name) return null;

    WebRequest web_request = WebRequest.Create("https://www.aol.com/?command=1&domain=" + Argument);
    web_request.Timeout = 5000;
    ((HttpWebRequest)web_request).UserAgent = "Mozilla Firefox 5.0";
    HttpWebResponse web_response = (HttpWebResponse)web_request.GetResponse();
    StreamReader response = new StreamReader(web_response.GetResponseStream(), Encoding.UTF8);

    return Argument + " => " + response.ReadToEnd();
}).ToArray();

var concate = string.Join(Environment.NewLine, lines);

答案 3 :(得分:0)

您可以使用concurrent collection并将其转换为一个字符串

var ThreadSafeList = new System.Collections.Concurrent.ConcurrentBag<string>();
Parallel.ForEach(Arguments, Argument =>
{
    if (Argument != Command_Name)
    {
        WebRequest web_request = WebRequest.Create("https://www.aol.com/?command=1&domain=" + Argument);
        web_request.Timeout = 5000;
        ((HttpWebRequest)web_request).UserAgent = "Mozilla Firefox 5.0";
        HttpWebResponse web_response = (HttpWebResponse)web_request.GetResponse();
        StreamReader response = new StreamReader(web_response.GetResponseStream(), Encoding.UTF8);
        ThreadSafeList.Add(Argument + " => " + response.ReadToEnd());                    
    }
});
string Result = string.Join("", ThreadSafeList);

或尝试锁定您要写的项目。

StringBuilder Message = new StringBuilder();
Parallel.ForEach(Arguments, Argument =>
{
    if (Argument != Command_Name)
    {
        WebRequest web_request = WebRequest.Create("https://www.aol.com/?command=1&domain=" + Argument);
        web_request.Timeout = 5000;
        ((HttpWebRequest)web_request).UserAgent = "Mozilla Firefox 5.0";
        HttpWebResponse web_response = (HttpWebResponse)web_request.GetResponse();
        StreamReader response = new StreamReader(web_response.GetResponseStream(), Encoding.UTF8);
        lock(Message) //locking here
        {
           Message.AppendLine(Argument + " => " + response.ReadToEnd());
        }
    }
});

答案 4 :(得分:-1)

在这种情况下实现线程安全wuold可能最好涉及lock statement。切记永远不要锁定您不具有独占访问权限的对象,在本例中为cabe方法范围。

object key = new object(); // Empty object serves lightest for locks
Parallel.ForEach(Arguments, Argument =>
{
    if (Argument != Command_Name)
    {
        WebRequest web_request = WebRequest.Create("https://www.aol.com/?command=1&domain=" + Argument);
        web_request.Timeout = 5000;
        ((HttpWebRequest)web_request).UserAgent = "Mozilla Firefox 5.0";
        HttpWebResponse web_response = (HttpWebResponse)web_request.GetResponse();
        StreamReader response = new StreamReader(web_response.GetResponseStream(), Encoding.UTF8);

        lock(key)
            Message += Argument + " => " + response.ReadToEnd() + Environment.NewLine;
    }
});