使用html敏捷包从html li获取数据列表

时间:2019-08-26 11:01:46

标签: c# html-agility-pack

检查下面的代码。在这里,我试图从html li标记中获取值列表。但是问题出在var allItems上,这没有抓住所有li的列表。之后,我想从每个li中获取三个值标题,价格,itemId。我已经像波纹管一样尝试并停留在var allItems上,因为这无法获取li数据。我该如何解决?

C#

string html = string.Empty;
                HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
                request.Method = "GET";
                request.AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate;
                var rnd = new Random();
                var proxy = proxies[rnd.Next(proxies.Count())];
                request.Proxy = new WebProxy(proxy.IP, proxy.Port);
                request.Proxy.Credentials = new NetworkCredential(proxy.Username, proxy.Password);
                request.Timeout = 10000;
                request.KeepAlive = true;
                using (var response = (HttpWebResponse)request.GetResponse())
                {
                    using (var stream = response.GetResponseStream())
                    using (var reader = new StreamReader(stream))
                    {
                        html = reader.ReadToEnd();

                    }
                }
                HtmlDocument htmlDoc = new HtmlDocument();
                htmlDoc.LoadHtml(html);

                var allItems = htmlDoc.DocumentNode.SelectNodes("//ol[@id='searchResults']");  //this not returning all `li` from `searchResults`

                var items = new List<ThirdPartyData>();
                foreach (var item in allItems)
                {
                    items.Add(new ThirdPartyData
                    {
                        title = "", // need to fill title from `li` <span class="main-title">
                        price = "", // need to fill price from `li` <span class="price__fraction">
                        ItemId = "" // need to fill ItemId from `li` class="rowItem item highlighted item--stack item--has-row-logo new "
                    });
                }

HTML:

<ol id="searchResults" class="section search-results list-view stack search-results-core search-results-std list--has-row-logos list--has-fulfillment" imgtype="listingLightweight">
   <li class="results-item highlighted article stack ">
      <div class="rowItem item highlighted item--stack item--has-row-logo new " id="MLA810704332">
         <div class="item__image item__image--stack">
            <div class="images-viewer" item-url="https://articulo.mercadolibre.com.ar/MLA-810704332-asus-rog-phone-2-gaming-128gb8gb-dual-ram-snapdragon-855-_JM" item-id="MLA810704332">
               <div class="image-content"> <a href="https://articulo.mercadolibre.com.ar/MLA-810704332-asus-rog-phone-2-gaming-128gb8gb-dual-ram-snapdragon-855-_JM" class="figure item-image item__js-link"> <img class="lazy-load" width="160" height="160" alt="Asus Rog Phone 2 Gaming 128gb+8gb Dual Ram - Snapdragon 855+" src="https://http2.mlstatic.com/asus-rog-phone-2-gaming-128gb-8gb-dual-ram-snapdragon-855-D_NQ_NP_832021-MLA31925153141_082019-V.webp"> </a> </div>
            </div>
         </div>
         <div class="item__info-container highlighted ">
            <div class="item__info item--hide-right-col ">
               <h2 class="item__title list-view-item-title"> <a href="https://articulo.mercadolibre.com.ar/MLA-810704332-asus-rog-phone-2-gaming-128gb8gb-dual-ram-snapdragon-855-_JM" class="item__info-title"> <span class="main-title"> Asus Rog Phone 2 Gaming 128gb+8gb Dual Ram - Snapdragon 855+ </span> </a></h2>
               <div class="price__container">
                  <div class="item__price "> <span class="price__symbol">$</span> <span class="price__fraction">51.999</span></div>
               </div>
               <div class="item__stack_column highlighted">
                  <div class="item__stack_column__info">
                     <div class="stack_column_item shipping highlighted">
                        <div class="item__shipping free-shipping highlighted">
                           <p class="stack-item-info "> Envío gratis</p>
                        </div>
                     </div>
                     <div class="stack_column_item status">
                        <div class="item__status">
                           <div class="item__condition"> </div>
                        </div>
                     </div>
                  </div>
               </div>
               <div class="stack_colum_right without-attributes without-reviews">
                  <div class="stack_column_right__bottom "> </div>
               </div>
            </div>
         </div>
         <form class="item__bookmark-form" action="/search/bookmarks/MLA810704332/make" method="post" id="bookmarkForm">
            <button type="submit" class="bookmarks favorite " data-id="MLA810704332">
               <div class="item__bookmark">
                  <div class="icon"></div>
               </div>
            </button>
            <input type="hidden" name="method" value="add"> <input type="hidden" name="itemId" value="MLA810704332"> 
         </form>
      </div>
   </li>
   <li class="results-item highlighted article stack ">
      <div class="rowItem item highlighted item--stack item--has-row-logo new " id="MLA799147144">
         <div class="item__image item__image--stack">
            <div class="images-viewer" item-url="https://articulo.mercadolibre.com.ar/MLA-799147144-2-pack-orzero-for-asus-rog-phone-zs600kl-tempered-glass-sc-_JM" item-id="MLA799147144">
               <div class="image-content"> <a href="https://articulo.mercadolibre.com.ar/MLA-799147144-2-pack-orzero-for-asus-rog-phone-zs600kl-tempered-glass-sc-_JM" class="figure item-image item__js-link"> <img class="lazy-load" width="160" height="160" alt="[2 Pack] Orzero For Asus Rog Phone Zs600kl Tempered Glass Sc" src="https://http2.mlstatic.com/2-pack-orzero-for-asus-rog-phone-zs600kl-tempered-glass-sc-D_NQ_NP_934744-MLA31661896785_082019-V.webp"> </a> </div>
            </div>
         </div>
         <div class="item__info-container highlighted ">
            <div class="item__info item--hide-right-col ">
               <h2 class="item__title list-view-item-title"> <a href="https://articulo.mercadolibre.com.ar/MLA-799147144-2-pack-orzero-for-asus-rog-phone-zs600kl-tempered-glass-sc-_JM" class="item__info-title"> <span class="main-title"> [2 Pack] Orzero For Asus Rog Phone Zs600kl Tempered Glass Sc </span> </a></h2>
               <div class="price__container">
                  <div class="item__price "> <span class="price__symbol">$</span> <span class="price__fraction">923</span> <span class="price__decimals">21</span></div>
               </div>
               <div class="item__stack_column highlighted">
                  <div class="item__stack_column__info">
                     <div class="stack_column_item installments highlighted"><span class="item-installments item__installments--show-card-icon highlighted free-interest "> <span class="item-installments-text"> Hasta 6 cuotas sin interés </span></span> </div>
                     <div class="stack_column_item status">
                        <div class="item__status">
                           <div class="item__condition"> </div>
                        </div>
                     </div>
                  </div>
               </div>
               <div class="stack_colum_right without-attributes without-reviews">
                  <div class="stack_column_right__bottom "> </div>
               </div>
            </div>
         </div>
         <form class="item__bookmark-form" action="/search/bookmarks/MLA799147144/make" method="post" id="bookmarkForm">
            <button type="submit" class="bookmarks favorite " data-id="MLA799147144">
               <div class="item__bookmark">
                  <div class="icon"></div>
               </div>
            </button>
            <input type="hidden" name="method" value="add"> <input type="hidden" name="itemId" value="MLA799147144"> 
         </form>
      </div>
   </li>
</ol>

0 个答案:

没有答案