发送多个httpclient或httpresponse时,UI挂起

时间:2019-04-18 18:54:33

标签: c# web-scraping httpclient httpresponse

试图从URL列表中获取页面源。但它会挂起UI。我更喜欢httpclient,所以我可以使用单个实例。请让我知道我该怎么办,这样在操作过程中不会挂起ui。

    async Task AccessTheWebAsync(CancellationToken ct)
    {
        HttpClient client = new HttpClient();

        List<string> urlToSearch = UrlToSearchList();
        IEnumerable<Task<string>> downloadTasksQuery =from url in urlToSearch select ProcessURL2(url, client, ct);
        List<Task<string>> downloadTasks = downloadTasksQuery.ToList();

        while (downloadTasks.Count > 0)
        {
            Task<string> firstFinishedTask = await Task.WhenAny(downloadTasks);
            downloadTasks.Remove(firstFinishedTask);
            string length = await firstFinishedTask;
            resultsTextBox.Text += $"\r\nLength of the download:  {length}";
        }
    }

    private List<string> UrlToSearchList()
    {
        var urlToSearchList = new List<string>();
        foreach (var item in lbURLToSearch.Items)
        {
            urlToSearchList.Add(item.ToString());
        }
        return urlToSearchList;
    }

    async Task<string> ProcessURL2(string url, HttpClient client, CancellationToken ct)
    {
        HttpResponseMessage response = await client.GetAsync(url, ct);

        var contents = await response.Content.ReadAsStringAsync();
        return contents;
    }
    private void CancelButton_Click(object sender, EventArgs e)
    {
        if (cts != null)
        {
            cts.Cancel();
        }
    }

    private async void StartButton_Click(object sender, EventArgs e)
    {
        resultsTextBox.Clear();
        cts = new CancellationTokenSource();

        try
        {
            await AccessTheWebAsync(cts.Token);
            resultsTextBox.Text += "\r\nDownloads complete.";
        }
        catch (OperationCanceledException)
        {
            resultsTextBox.Text += "\r\nDownloads canceled.\r\n";
        }
        catch (Exception)
        {
            resultsTextBox.Text += "\r\nDownloads failed.\r\n";
        }

        cts = null;
    }

我还尝试了各种异步方法。仍然相同的结果,它挂起。下面是我尝试过的一种异步方法。

    private async void TsbtnStartURLScraper_Click(object sender, EventArgs e)
    {
        for (int i = 0; i < lbURLToSearch.Items.Count; i++)
        {
            txtScrapeURL.Text = lbURLToSearch.Items[i].ToString();
            string urlAddress = lbURLToSearch.Items[i].ToString();
            var returnedresult = await DownloadStringV1(urlAddress);
            resultsTextBox.Text += returnedresult;
        }

    }
    public async Task<String> DownloadStringV1(String url)
    {
        // good code
        var request = await reusableHTTPClient.GetAsync(url);
        var download = await request.Content.ReadAsStringAsync();
        return download;
    }

注意:我想做的是,遍历URL列表并从中获取页面源。但是当我这样做时,它冻结了表单。

0 个答案:

没有答案