使用selenium java迭代多个页面的产品列表

时间:2016-07-11 16:06:14

标签: java selenium web-scraping

我需要查看产品清单:

  • 点击它
  • 在产品页面上执行某些操作(这些操作很简单且已经完成)
  • 返回我已经在的列表
  • 重复该页面的所有产品

完成当前页面的产品列表后,我希望能够单击下一步并对下一页上的产品执行相同的步骤。

这是问题:当它到达当前页面的10个项目时,我不知道如何更改到另一个页面并重新开始。

示例代码html:

<!DOCTYPE html> 
<html lang="pt-br">
  <head>
    <title>Produtos</title>
    <meta charset="utf-8">
  </head>
  <body>
<div class="srp-actions blue-button"><a class="primary-action-button label" href="">Produto 1</a></div>
<div class="srp-actions blue-button"><a class="primary-action-button label" href="">Produto 2</a></div>
<div class="srp-actions blue-button"><a class="primary-action-button label" href="">Produto 3</a></div>
<div class="srp-actions blue-button"><a class="primary-action-button label" href="">Produto 4</a></div>

    <br/>

<div id="pagination">
        <ul class="pagination">
        <li class="active">1</li>
        <li class="link"><a class="page-link" href="" title="Página 2">2</a></li>
        <li class="link"><a class="page-link" href="" title="Página 2">3</a></li>
        <li class="link"><a class="page-link" href="" title="Página 2">4</a></li>
        <li class="next"><a class="page-link" href="" rel="next" title="Avançar">next</a></li>
        </ul>
</div>
</body>
</html>

Code Java:

int size = driver.findElements(By.className("page-link")).size();
    System.out.println("Numero de paginas : " + size);
    for(int j = 1 ; j < size ; j++) {                
        if (j < 2) {// we don't need to navigate to the first page
            driver.findElement(By.linkText("Avançar >")).click(); // navigate to page number j
        }           

        String pagesearch = driver.getCurrentUrl();


    for(int i=1;i< links.size();i++){

    links= driver.findElements(By.linkText("Produto"));

    WebDriverWait wait3 = new WebDriverWait(driver, 30);
    wait3.until(ExpectedConditions.visibilityOfAllElementsLocatedBy(By.linkText("Produto")));

    links.get(i).click();

    Thread.sleep(2000); 

1 个答案:

答案 0 :(得分:1)

The basic logic would be:

  1. Go to the first product list page.
  2. Grab all the product links. Navigate to each product page and do stuff.
  3. Go back to the product list page and click Next.

You will repeat 2 & 3 until there is no Next link.

String url = "http://www.example.com"; // the first page of the product list page
driver.get(url);
List<WebElement> next;
while (true)
{
    // wait for and get all the product links
    WebDriverWait wait = new WebDriverWait(driver, 10);
    List<WebElement> productLinks = wait.until(ExpectedConditions.visibilityOfAllElementsLocatedBy(By.linkText("Produto")));
    // loop through the product links
    for (WebElement productLink : productLinks)
    {
        driver.get(productLink.getAttribute("href")); // navigate to product page
        // do stuff on product page
    }

    // now we're done with all products on this page, go back to the product list page that we were last on
    driver.get(url);

    // look for a Next link
    next = driver.findElements(By.cssSelector("li.next"));
    if (next.isEmpty())
    {
        // Next link DOES NOT exist, exit loop
        break;
    }

    // Next link DOES exists, click it to go to the next page
    next.get(0).click();
    // may need to wait for page transition here
    url = driver.getCurrentUrl(); // store the current product list page