Question

我正在尝试下载

＆＃34; https://www.google.com/search?sclient=psy-ab&biw=1472&bih=740&espv=2&tbm=vid&btnG=Search&q=%25%25%25#q=iran&tbm=nws＆＃34 ;;

通过以下代码：

        string url = "https://www.google.com/search?sclient=psy-ab&biw=1472&bih=740&espv=2&tbm=vid&btnG=Search&q=%25%25%25#q=iran&tbm=nws";
        try
        {
            string htmlPage = "";

            //http request preparing
            CookieContainer CC = new CookieContainer();
            HttpWebRequest objRequest = (HttpWebRequest)WebRequest.Create(url);
            objRequest.Timeout = 60000;
            objRequest.Proxy = null;
            objRequest.UseDefaultCredentials = true;
            objRequest.KeepAlive = false; //THIS DOES THE TRICK
            objRequest.ProtocolVersion = HttpVersion.Version10; // THIS DOES THE TRICK
            objRequest.CookieContainer = CC;

            //http request sending
            using (HttpWebResponse objResponse = (HttpWebResponse)objRequest.GetResponse())
            {
                using (Stream strm = objResponse.GetResponseStream())
                {
                    using (StreamReader objStreamReader = new StreamReader(strm))
                    {
                        htmlPage = objStreamReader.ReadToEnd();
                    }
                }
            };
            if (htmlPage.Contains("No results found for") || htmlPage.Contains("(without quotes):") || htmlPage.Contains("Make sure all words are spelled correctly."))
            {
                return dtResult;
            }
            else
            {
                Regex objEgEx = new Regex(@"[\r\n][ ]+\.[\r\n][ ]+");
                htmlPage = objEgEx.Replace(htmlPage, string.Empty);

                int startIndex = htmlPage.IndexOf("<div class =\"g\">");
                if (startIndex == -1)
                { Console.Write("problem in parsing"); }

但是HttpWebRequest下载谷歌的第一页而不是我为它保存的网址，这是Google搜索结果的视频搜索服务的地址＆＃39;页。我该怎么做才能下载我想要的网址？

Answer 1

您正在下载页面，而不是查询。由于搜索谷歌没有加载新页面但更新页面。也许看看google search api

HttpWebRequest不检索我设置的url

1 个答案: