如何使用c#获取inspect元素代码

时间:2014-02-25 04:55:08

标签: c# html-agility-pack

我想从URL获取文本,但文本未在源代码中显示。我只能在inspect元素中看到它。无论如何,在C#中获取页面的Inspect元素的内容。我用c#尝试htmlagilitypack,但是给出了null异常。

Class1 class1obj = new Class1(); 
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);  
request.Method = "GET"; 
HttpWebResponse response = (HttpWebResponse)request.GetResponse(); 
if (response.StatusCode == HttpStatusCode.OK) 
{ 
    Stream receiveStream = response.GetResponseStream(); 
    StreamReader readStream = null; 
    if (response.CharacterSet == null)
        readStream = new StreamReader(receiveStream); 
    else
        readStream = new StreamReader(receiveStream, Encoding.GetEncoding(response.CharacterSet)); 
    class1obj.Data1 = readStream.ReadToEnd(); 
    Console.WriteLine(class1obj.Data1);
    readStream.Close();
 }

 HtmlAgilityPack.HtmlDocument doc = new HtmlAgilityPack.HtmlDocument();
 doc.LoadHtml(class1obj.Data1);
 // HtmlNodeCollection tables2 =     doc.DocumentNode.SelectNodes(".//div[@class='tv_ch_detila h_scroll_detail']");
 HtmlNode rows2 = doc.DocumentNode.SelectSingleNode(".//div[@class='sub_text']");
 Console.WriteLine(rows2.InnerText);

1 个答案:

答案 0 :(得分:0)

这是在页面加载后动态加载的加密数据。得到上述数据的网址是:

http://www.whatsonindia.com/appi/user?channelgenre=all&context=applicationname%3Dsourcebits%3Bheadendid%3D0&dateselected=0&mode=getTVGuideInfo&pageno=1&responseformat=json&responselanguage=English&starthour=0&totalhrdata=24&userid=-1

它包含JSON格式的“数据”和“pki”节点。你不能只是在等待页面被完全加载时回复。

相反,您只需向给定的URL发出请求并解密文本即可。这是我在这里可以假设的唯一方式。不可靠的方法是在Web浏览器控件中加载URL并等待几秒钟(比如5-10),假设所有内容都已加载,然后解析DocumentText或Web浏览器控件。

我设法为电视节目获取此JSON数据:

{
"gettvguidewithgenre": {
    "tvguideprogrammewithgenrelist": [
      {
        "programmeid": "10000000934920000",
        "starttime": "2014-02-25T00:00:00+05:30",
        "endtime": "2014-02-25T01:40:00+05:30",
        "channelid": "10000000000110000",
        "programmename": "Ghatak",
        "m2ecategoryname": "Film",
        "m2esubcategoryname": "Action",
        "Isrecommended": "1"
      },
      {
        "programmeid": "30000000549824121",
        "starttime": "2014-02-25T01:40:00+05:30",
        "endtime": "2014-02-25T03:50:00+05:30",
        "channelid": "10000000000110000",
        "programmename": "Time Limit-36 Ghante",
        "m2ecategoryname": "Film",
        "m2esubcategoryname": "Drama",
        "Isrecommended": "1"
      },
      {
        "programmeid": "30000000549816018",
        "starttime": "2014-02-25T03:50:00+05:30",
        "endtime": "2014-02-25T05:40:00+05:30",
        "channelid": "10000000000110000",
        "programmename": "Volcano High",
        "m2ecategoryname": "Film",
        "m2esubcategoryname": "Action",
        "Isrecommended": "1"
      },
      {
        "programmeid": "30000000549673440",
        "starttime": "2014-02-25T05:40:00+05:30",
        "endtime": "2014-02-25T08:10:00+05:30",
        "channelid": "10000000000110000",
        "programmename": "Maa Beti",
        "m2ecategoryname": "Film",
        "m2esubcategoryname": "Drama",
        "Isrecommended": "1"
      },
      {
        "programmeid": "10000001008740000",
        "starttime": "2014-02-25T08:10:00+05:30",
        "endtime": "2014-02-25T11:05:00+05:30",
        "channelid": "10000000000110000",
        "programmename": "Dus",
        "m2ecategoryname": "Film",
        "m2esubcategoryname": "Action",
        "Isrecommended": "1"
      },
      {
        "programmeid": "30000000549644318",
        "starttime": "2014-02-25T11:05:00+05:30",
        "endtime": "2014-02-25T13:55:00+05:30",
        "channelid": "10000000000110000",
        "programmename": "Money Hai Toh Honey Hai",
        "m2ecategoryname": "Film",
        "m2esubcategoryname": "Comedy",
        "Isrecommended": "1"
      },
      {
        "programmeid": "30000000549646489",
        "starttime": "2014-02-25T13:55:00+05:30",
        "endtime": "2014-02-25T17:15:00+05:30",
        "channelid": "10000000000110000",
        "programmename": "Farz",
        "m2ecategoryname": "Film",
        "m2esubcategoryname": "Drama",
        "Isrecommended": "1"
      },
      {
        "programmeid": "30000000549709799",
        "starttime": "2014-02-25T17:15:00+05:30",
        "endtime": "2014-02-25T20:00:00+05:30",
        "channelid": "10000000000110000",
        "programmename": "Action Replayy",
        "m2ecategoryname": "Film",
        "m2esubcategoryname": "Comedy",
        "Isrecommended": "1"
      },
      {
        "programmeid": "30000000549664553",
        "starttime": "2014-02-25T20:00:00+05:30",
        "endtime": "2014-02-25T23:25:00+05:30",
        "channelid": "10000000000110000",
        "programmename": "Indian",
        "m2ecategoryname": "Film",
        "m2esubcategoryname": "Action",
        "Isrecommended": "1"
      },
      {
        "programmeid": "30000000549781987",
        "starttime": "2014-02-25T23:25:00+05:30",
        "endtime": "2014-02-26T00:00:00+05:30",
        "channelid": "10000000000110000",
        "programmename": "Narsimha - Man Of Power",
        "m2ecategoryname": "Film",
        "m2esubcategoryname": "Drama",
        "Isrecommended": "1"
      },
      {
        "programmeid": "30000000549741259",
        "starttime": "2014-02-25T00:00:00+05:30",
        "endtime": "2014-02-25T02:00:00+05:30",
        "channelid": "10000000000540000",
        "programmename": "The Dirty Picture",
        "m2ecategoryname": "Film",
        "m2esubcategoryname": "Drama",
        "Isrecommended": "1"
      },
      {
        "programmeid": "30000000549867681",
        "starttime": "2014-02-25T02:00:00+05:30",
        "endtime": "2014-02-25T02:40:00+05:30",
        "channelid": "10000000000540000",
        "programmename": "Extraa Shots Special",
        "m2ecategoryname": "TV Show",
        "m2esubcategoryname": "Entertainment",
        "Isrecommended": "1"
      },
      {
        "programmeid": "30000000549867681",
        "starttime": "2014-02-25T02:40:00+05:30",
        "endtime": "2014-02-25T03:00:00+05:30",
        "channelid": "10000000000540000",
        "programmename": "Extraa Shots Special",
        "m2ecategoryname": "TV Show",
        "m2esubcategoryname": "Entertainment",
        "Isrecommended": "1"
      },
      {
        "programmeid": "30000000549867681",
        "starttime": "2014-02-25T03:00:00+05:30",
        "endtime": "2014-02-25T03:25:00+05:30",
        "channelid": "10000000000540000",
        "programmename": "Extraa Shots Special",
        "m2ecategoryname": "TV Show",
        "m2esubcategoryname": "Entertainment",
        "Isrecommended": "1"
      },
      {
        "programmeid": "30000000549867681",
        "starttime": "2014-02-25T03:25:00+05:30",
        "endtime": "2014-02-25T03:40:00+05:30",
        "channelid": "10000000000540000",
        "programmename": "Extraa Shots Special",
        "m2ecategoryname": "TV Show",
        "m2esubcategoryname": "Entertainment",
        "Isrecommended": "1"
      },
      {
        "programmeid": "30000000549867681",
        "starttime": "2014-02-25T03:40:00+05:30",
        "endtime": "2014-02-25T04:00:00+05:30",
        "channelid": "10000000000540000",
        "programmename": "Extraa Shots Special",
        "m2ecategoryname": "TV Show",
        "m2esubcategoryname": "Entertainment",
        "Isrecommended": "1"
      },
      {
        "programmeid": "30000000549867681",
        "starttime": "2014-02-25T04:00:00+05:30",
        "endtime": "2014-02-25T04:25:00+05:30",
        "channelid": "10000000000540000",
        "programmename": "Extraa Shots Special",
        "m2ecategoryname": "TV Show",
        "m2esubcategoryname": "Entertainment",
        "Isrecommended": "1"
      },
      {
        "programmeid": "10000001078790000",
        "starttime": "2014-02-25T18:08:00+05:30",
        "endtime": "2014-02-25T20:30:00+05:30",
        "channelid": "10000000000210000",
        "programmename": "Aadmi",
        "m2ecategoryname": "Film",
        "m2esubcategoryname": "Action",
        "Isrecommended": "1"
      },
      {
        "programmeid": "30000000549651842",
        "starttime": "2014-02-25T20:30:00+05:30",
        "endtime": "2014-02-25T23:16:00+05:30",
        "channelid": "10000000000210000",
        "programmename": "Dum",
        "m2ecategoryname": "Film",
        "m2esubcategoryname": "Action",
        "Isrecommended": "1"
      },
      {
        "programmeid": "30000000549828153",
        "starttime": "2014-02-25T23:16:00+05:30",
        "endtime": "2014-02-26T00:00:00+05:30",
        "channelid": "10000000000210000",
        "programmename": "Aakhri Baazi",
        "m2ecategoryname": "Film",
        "m2esubcategoryname": "Romance",
        "Isrecommended": "1"
      }
    ],
    "channelinfo": [
      {
        "ChannelID": "10000000000110000",
        "ChannelName": "STAR GOLD",
        "LCN": "0",
        "Operator_Id": "2645",
        "logofileurl": "http://images.whatsonindia.com/WhatsOnTv/Images/applicationchannellogo/10000000000110000.png",
        "ischannelfavorite": "false"
      },
      {
        "ChannelID": "10000000000540000",
        "ChannelName": "MAX",
        "LCN": "0",
        "Operator_Id": "2645",
        "logofileurl": "http://images.whatsonindia.com/WhatsOnTv/Images/applicationchannellogo/10000000000540000.png",
        "ischannelfavorite": "false"
      },
      {
        "ChannelID": "10000000000620000",
        "ChannelName": "TEN SPORTS",
        "LCN": "0",
        "Operator_Id": "2645",
        "logofileurl": "http://images.whatsonindia.com/WhatsOnTv/Images/applicationchannellogo/10000000000620000.png",
        "ischannelfavorite": "false"
      },
      {
        "ChannelID": "10000000000700000",
        "ChannelName": "HBO",
        "LCN": "0",
        "Operator_Id": "2645",
        "logofileurl": "http://images.whatsonindia.com/WhatsOnTv/Images/applicationchannellogo/10000000000700000.png",
        "ischannelfavorite": "false"
      },
      {
        "ChannelID": "10000000000100000",
        "ChannelName": "STAR MOVIES",
        "LCN": "0",
        "Operator_Id": "2645",
        "logofileurl": "http://images.whatsonindia.com/WhatsOnTv/Images/applicationchannellogo/10000000000100000.png",
        "ischannelfavorite": "false"
      },
      {
        "ChannelID": "10000000000080000",
        "ChannelName": "STAR PLUS",
        "LCN": "0",
        "Operator_Id": "2645",
        "logofileurl": "http://images.whatsonindia.com/WhatsOnTv/Images/applicationchannellogo/10000000000080000.png",
        "ischannelfavorite": "false"
      },
      {
        "ChannelID": "10000000000210000",
        "ChannelName": "ZEE CINEMA",
        "LCN": "0",
        "Operator_Id": "2645",
        "logofileurl": "http://images.whatsonindia.com/WhatsOnTv/Images/applicationchannellogo/10000000000210000.png",
        "ischannelfavorite": "false"
      }
    ]
  }
}

它使用CryptoJS进行加密和解密。