如何通过HttpWebRequest c#获取网站上的隐藏内容?

时间:2019-04-07 04:12:29

标签: c# web-scraping httpwebrequest webclient

我想从网站(电话号码)中获取隐藏的内容,当我点击“Wyświetlnumer”按钮(显示号码)时,电话号码就会显​​示出来。发送HttpWebRequest时我无法访问号码。我有403错误。

我注意到我必须要求:

  • CSRF令牌-在源站点中,
  • phoneCode-在相关站点中,
  • cookie:_csrf,_abck-来自服务器的响应

例如: https://www.otodom.pl/oferta/dom-posesja-z-wlasnym-stawem-blisko-gorzowa-ID3V94s.html

数字是: 515 185 943

正确回答:

{"meta":{"status_code":200,"message":"Action applied successfully","created_at":"2019-04-07 02:48:54.480582966 +0000 UTC"},"data":[{"phone":"515185943","phonecode":"249-203-107-114-16-68-207-238-198-45-92-145-239-111-199-163-61-250-147-7-28-250-146-14-117-171-73-180-30-72-65-214-53-218-59-19"}]}

我的代码
我的代码可以在手机上工作1到3次,或者根本不工作。

请求_csrf,_abck

HttpWebRequest req = (HttpWebRequest)WebRequest.Create("https://www.otodom.pl/oferta/dom-posesja-z-wlasnym-stawem-blisko-gorzowa-ID3V94s.html");
req.Timeout = 20000;
req.Method = "GET";
req.UserAgent = "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.2 (KHTML, like Gecko) Chrome/15.0.874.121 Safari/535.2";
req.Host = "www.otodom.pl";
req.KeepAlive = true;
req.Headers.Add("Upgrade-Insecure-Requests", "1");

WebResponse resp = req.GetResponse();
cookieHeader = resp.Headers["Set-cookie"]; //_csrf, _abck

索取电话号码

HttpWebRequest getRequest = (HttpWebRequest)WebRequest.Create("https://www.otodom.pl/frontera/api/item/owner/phone/249-203-107-114-16-68-207-238-198-45-92-145-239-111-199-163-61-250-147-7-28-250-146-14-117-171-73-180-30-72-65-214-53-218-59-19");
getRequest.Method = "GET";
getRequest.Timeout = 20000;
getRequest.UserAgent = "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.2 (KHTML, like Gecko) Chrome/15.0.874.121 Safari/535.2";
getRequest.ContentType = "application/json; charset=UTF-8";
getRequest.Host = "www.otodom.pl";
getRequest.KeepAlive = true;
getRequest.Accept = "application/json, text/plain, */*";
getRequest.Headers.Add("Cookie", cookieHeader);
getRequest.Headers.Add("CSRF-Token", token);
getRequest.Headers.Add("SITECODE", "otodompl");
getRequest.Headers.Add("MICRO_LOCALE", "pl");
getRequest.Headers.Add("SITE_LANG", "pl");
getRequest.Headers.Add("SITESTAGE", "PROD");
getRequest.Headers.Add("SSR-Proxying-Atlas", "true");

WebResponse getResponse = getRequest.GetResponse();
using (StreamReader sr = new StreamReader(getResponse.GetResponseStream()))
{
  strPhone = sr.ReadToEnd();
}

我不知道哪里出了问题。

0 个答案:

没有答案