使网络抓取工具通过网站进行解析

时间:2015-11-10 23:56:08

标签: java html

今天早些时候我发布了 Finding number of occurrences on website

但我收到的答案并没有像我希望的那样多。我试图告诉爬虫从其找到的网站中读取文本并搜索一个给定的单词。我发现了这个:

 org.jsoup.nodes.Document dom = Jsoup.parse(html);

但是我不知道如何实现它。请帮忙

履带

public void crawlFrom(String link){ // TODO

    try
    {
        Connection connection = Jsoup.connect(link).userAgent(USER_AGENT);
        Document htmlDocument = connection.get();
        this.htmlDocument = htmlDocument;
        System.out.println("Received web page at " + link);

        Elements linksOnPage = htmlDocument.select("a[href]");
        System.out.println("------------------\nFound (" + linksOnPage.size() + ") links\n------------------");

        for(Element newLink : linksOnPage)
        {
            this.linkListe.add(newLink.absUrl("href"));

        }
    }
    catch(IOException ioe)
    {
        // We were not successful in our HTTP request
        System.out.println("Error in out HTTP request " + ioe);
    }
       System.out.println(linkListe);
    return;
}

搜索器

public int searchHits(String target, String aften){ // TODO
    String[] out = new String[0];
    int occurrences = 0;

    if (aften.contains(target)) {
       occurrences++;
    }
    return occurrences;
    }

1 个答案:

答案 0 :(得分:0)

我不确定aftentarget是什么,但我会给你一段代码,用于搜索文本中单词的出现次数。

public int searchHits(String target, String aften){ // TODO

    int index = 0;
    int occurrences = 0;

    while(index != -1){

        index = aften.indexOf(target,index); // start search from index 0

        if(index != -1){
            occurrences ++; //if found, increment the counter
            index += target.length(); // set the next starting index to be after this current index
        }
    }
    return occurrences;
}

--------------------- UPDATE ---------------------

将您的抽象方法从int searchHits(String word);更改为int searchHits(String target, String aften);