WPF c# WebCrawler

时间:2016-10-20 19:50:30

标签: c# wpf web-crawler

I´m trying to develop a web crawler to extract some information from my company's web site, but I'm getting the error as below:

An exception of type System.Net.WebException occurred in System.dll but was not handled in user code

Additional information: The remote server returned an error: (500) Internal Server Error.

Here is the requestHeaders of the website:

Accept:text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Charset:ISO-8859-1,utf-8;q=0.7,*;q=0.3
Accept-Encoding:gzip,deflate,sdch
Accept-Language:en-US,en;q=0.8
Connection:keep-alive
Cookie:WASReqURL=https://:9446/ProcessPortal/jsp/index.jsp; com_ibm_bpm_process_portal_hash=null
Host:ca8webp.itau:9446
User-Agent:Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.17 (KHTML, like Gecko) Chrome/24.0.1312.60 Safari/537.17

Here is my code:

public bool Acessar(CredencialWAO credencial)
{
            bool logado = true;

            #region REQUISIÇÃO PAGINA INICIAL

            ParametrosCrawler parametrosRequest = new ParametrosCrawler("https://ca8webp.itau:9446/ProcessPortal/login.jsp");
            parametrosRequest.Accept = "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8";
            parametrosRequest.AcceptEncoding = "gzip, deflate, sdch";
            parametrosRequest.AcceptLanguage = "en-US,en;q=0.8";
            parametrosRequest.UserAgent = "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.17 (KHTML, like Gecko) Chrome/24.0.1312.60 Safari/537.17";
            parametrosRequest.CacheControl = "max-age=0";
}

protected override WebResponse GetWebResponse(WebRequest request)
{
            var response = base.GetWebResponse(request);

            AtualizarCookies(((HttpWebResponse)response).Cookies);

            return response;
}

The error occurs when calling "base.GetWebResponse(request)"

0 个答案:

没有答案