Html在Groovy中啜饮

时间:2016-11-05 00:14:21

标签: java groovy web-scraping

我正在尝试解析作为巨型字符串来到我身边的HTML。当我到达第13行时,NodeChild page = it.parent()

我能够找到我正在寻找的密钥,但数据来自我This Is Value One In My KeyThis is Value Two in my KeyThis is Value Three In My Key等等。我看到一个反复出现的趋势,两者之间的分隔符始终是UppercaseUppercase(无空间)。

我想以某种方式将其ArrayList放入其中。是否有一种我遗漏的方法from the docs能够自动执行此操作?有没有更好的方法来解析这个?

class htmlParsingStuff{

    private def slurper = new XmlSlurper(new Parser())

    private void slurpItUp(String rawHTMLString){
        ArrayList urlList = []
        def htmlParser = slurper.parseText(rawHTMLString)

        htmlParser.depthFirst().findAll() {
            //Loop through all of the HTML Tags to get to the key that I am looking for 
            //EDIT: I see that I am able to iterate through the parent object, I just need a way to figure out how to get into that object
            boolean trigger = it.text() == 'someKey'
            if (trigger){
                //I found the key that I am looking for
                NodeChild page = it.parent()
                page = page.replace('someKey', '')
                LazyMap row = ["page": page, "type": "Some Type"]
                urlList.add(row)
            }
        }
    }
}

1 个答案:

答案 0 :(得分:1)

我无法为您提供工作代码,因为我不知道您的具体HTML。

但是:不要使用XmlSlurper来解析HTML,HTML格式不正确,因此XmlSlurper不适合这项工作。

对于HTML,请使用JSoup之类的库。如果你有一些JQuery知识,你会发现它更容易使用。由于您没有发布HTML代码段,因此我编写了自己的示例:

@Grab(group='org.jsoup', module='jsoup', version='1.10.1')
import org.jsoup.Jsoup

def html = """
<html>
<body>
  <table>
    <tr><td>Key 1</td></tr>
    <tr><td>Key 2</td></tr>
    <tr><td>Key 3</td></tr>
    <tr><td>Key 4</td></tr>
    <tr><td>Key 5</td></tr>
  </table>
</body>
</html>"""

def doc = Jsoup.parse(html)
def elements = doc.select('td')
def result = elements.collect {it.text()}
// contains ['Key 1', 'Key 2', 'Key 3', 'Key 4', 'Key 5']

操纵您将使用的文件

def doc = Jsoup.parse(html)
def elements = doc.select('td')
elements.each { oldElement ->
    def newElement = new Element(Tag.valueOf('td'), '')
    newElement.text('Another key')
    oldElement.replaceWith(newElement)
}
println doc.outerHtml()