Groovy XML解析(HTML slurping),无法让我的具体案例工作

时间:2012-02-22 05:10:33

标签: groovy xml-parsing screen-scraping gpath

好的,这就是我要找的东西。

我想进入DOM并查找以“thread_title_”开头的<a id>。以下是我尝试的一些事情:

// setup
def slurper = new XmlSlurper(new org.ccil.cowan.tagsoup.Parser())
def gurl = new URL("url")
gurl.withReader { gReader ->

  def try1 = gHTML.body.find { it['@id'].startsWith("thread_title_") }
  // fails: Caught: groovy.lang.MissingMethodException: No signature of method: groovy.util.slurpersupport.Attributes.startsWith() is applicable for argument types: (java.lang.String) values: [thread_title_]

  def try2 = gHTML.body.find { it['@id'] =~ /thread_title_/ }
  // fails: Caught: groovy.lang.MissingMethodException: No signature of method: groovy.util.slurpersupport.Attributes.startsWith() is applicable for argument types: (java.lang.String) values: [thread_title_]

  def try3 = gHTML.body.find { it['@id'].name.startsWith("thread_title_") }
  // fails: Caught: groovy.lang.MissingMethodException: No signature of method: groovy.util.slurpersupport.NodeChildren.startsWith() is applicable for argument types: (java.lang.String) values: [thread_title_]

  def try4 = gHTML.body.find { it['@id'] == 'thread_title_745429' }
  // doesn't fail, but doesn't return anything either

  def try5 = gHTML.body.findAll { it.name() == 'a' && it.@id.startsWith('thread_title_') }
  try5.eachWithIndex { row, i ->
    println "rn: $i"
  }
  // no output

}

Here是属性的gdoc。我真的不想要“名字”,我想要“价值”。 The gpath page暗示node.character.find { it['@id'] == '2' }有效,这看起来很像是..starts与我同在。 This stackoverflow answer是类似的,但是启动与不同,似乎对整个事情产生了影响。第五个条目是inspired by this stackoverflow answer

如果您担心输入数据存在问题:     $ curl --silent http://www.advrider.com/forums/forumdisplay.php?f=18 | grep thread_title | wc -l     43

以下是一些示例输出,使用上面的curl | grep

<a href="foo" id="thread_title_705760">text</a>
<a href="foo" id="thread_title_753701">text</a>

我安装了Groovy 1.7.10。我可以更新,不知道是否会有所帮助。

1 个答案:

答案 0 :(得分:2)

这是怎么回事?

@Grab( 'org.ccil.cowan.tagsoup:tagsoup:1.2.1' )
import org.ccil.cowan.tagsoup.Parser

def gHTML = new URL( 'http://www.advrider.com/forums/forumdisplay.php?f=18' ).withReader { r ->
  new XmlSlurper( new Parser() ).parse( r )
}

def allLinks = gHTML.body.'**'.findAll { it.name() == 'a' && it.@id.text().startsWith( 'thread_title_' ) }
allLinks.each { link ->
  println "${link.text()} -> ${link.@href}"
}

如果您有任何问题或疑问,请告诉我们: - )