如何使用Java和Jsoup从页面源获取数据

时间:2018-07-17 13:58:33

标签: java html css jsoup

如何从以下位置获取值 $ 23,000,000 $ 47,351,251 以下页面的来源? 我只想从源代码中获取这些值,但是我不确定做到这一点的最佳方法。

  <div class="txt-block">
            <h4 class="inline">Budget:</h4>$23,000,000
            <span class="attribute">(estimated)</span>
        </div>

        <div class="txt-block">
            <h4 class="inline">Opening Weekend USA:</h4> $260,382,
<span class="attribute">20 December 2013</span>, <span class="attribute">Limited Release</span>
        </div>

        <div class="txt-block">
<h4 class="inline">Gross USA:</h4> $25,568,251
        </div>
        <div class="txt-block">
<h4 class="inline">Cumulative Worldwide Gross:</h4> $47,351,251
        </div>

我尝试过这样:

    String url = "https://www.imdb.com/title/tt1798709";
    Connection connection = Jsoup.connect(url);
    Document document = connection.get();
    Elements element = document.getElementsByClass("txt-block");


    String gross = "";
    String budget = "";

    String budgetRegex = "Budget:.*";
    String grossRegex = "Cumulative Worldwide Gross:.*";

    for (Element e : element) {
        if (e.text().matches(budgetRegex)) {
            String text = e.text();
            budget = StringUtils.substringBetween(text, "$", " ");
            break;
        } else {
            budget = null;
        }
    }
    for (Element e : element) {
        if (e.text().matches(grossRegex)) {
            String text = e.text();
            gross = StringUtils.substringAfter(text, "$");
                break;
        } else {
            gross = null;
        }

    }
    System.out.println(gross + ", " + budget);

它正在工作,但是有更好的解决方案吗?

2 个答案:

答案 0 :(得分:0)

使用ownText()而不是子字符串,仅循环一次,而不是两次。试试这个:

    String url = "https://www.imdb.com/title/tt1798709";
    Connection connection = Jsoup.connect(url);
    Document document = connection.get();
    Elements elements = document.select("div.txt-block");

    String gross = "";
    String budget = "";

    final String budgetRegex = "Budget:";
    final String grossRegex = "Cumulative Worldwide Gross:";

    for (Element e : elements) {
        final String h4Text = e.getElementsByTag("h4").first().text();
        switch (h4Text) {
            case budgetRegex:
                budget = e.ownText();
                break;
            case grossRegex:
                gross = e.ownText();
                break;
        }
        if (!gross.isEmpty() && !budget.isEmpty()) { //this IF is optional, just added for performance
            break;
        }
    }
    System.out.println(gross + ", " + budget);

答案 1 :(得分:0)

您可以使用jsoup伪选择器来完成这项工作:

    Document document = Jsoup.parse(html);
    String budget = document.select("div:contains(Budget:)").first().ownText();
    String gross = document.select("div:contains(Cumulative Worldwide Gross:)").first().ownText();
    System.out.println(gross + ", " + budget);

有关伪选择器的更多信息,您可以在这里找到:https://jsoup.org/cookbook/extracting-data/selector-syntax