使用XPath从HTML / XML文档中分组兄弟姐妹?

时间:2011-10-19 20:14:52

标签: ruby xpath nokogiri

我想通过对以前未分组的兄弟节点进行分组来转换HTML或XML文档。

例如,我想采取以下片段:

<h2>Header</h2>
<p>First paragraph</p>
<p>Second paragraph</p>

<h2>Second header</h2>
<p>Third paragraph</p>
<p>Fourth paragraph</p>

进入这个:

<section>
  <h2>Header</h2>
  <p>First paragraph</p>
  <p>Second paragraph</p>
</section>

<section>
  <h2>Second header</h2>
  <p>Third paragraph</p>
  <p>Fourth paragraph</p>
</section>

这是否可以使用简单的Xpath选择器和像Nokogiri这样的XML解析器?或者我是否需要为此任务实现SAX解析器?

3 个答案:

答案 0 :(得分:2)

更新答案

这是一个通用解决方案,它根据标题级别及其后续兄弟信息创建<section>元素的层次结构:

class Nokogiri::XML::Node
  # Create a hierarchy on a document based on heading levels
  #   wrap   : e.g. "<section>" or "<div class='section'>"
  #   stops  : array of tag names that stop all sections; use nil for none
  #   levels : array of tag names that control nesting, in order
  def auto_section(wrap='<section>', stops=%w[hr], levels=%w[h1 h2 h3 h4 h5 h6])
    levels = Hash[ levels.zip(0...levels.length) ]
    stops  = stops && Hash[ stops.product([true]) ]
    stack = []
    children.each do |node|
      unless level = levels[node.name]
        level = stops && stops[node.name] && -1
      end
      stack.pop while (top=stack.last) && top[:level]>=level if level
      stack.last[:section].add_child(node) if stack.last
      if level && level >=0
        section = Nokogiri::XML.fragment(wrap).children[0]
        node.replace(section); section << node
        stack << { :section=>section, :level=>level }
      end
    end
  end
end

以下是使用中的代码及其提供的结果。

原始HTML

<body>
<h1>Main Section 1</h1>
<p>Intro</p>
<h2>Subhead 1.1</h2>
<p>Meat</p><p>MOAR MEAT</p>
<h2>Subhead 1.2</h2>
<p>Meat</p>
<h3>Caveats</h3>
<p>FYI</p>
<h4>ProTip</h4>
<p>Get it done</p>
<h2>Subhead 1.3</h2>
<p>Meat</p>

<h1>Main Section 2</h1>
<h3>Jumpin' in it!</h3>
<p>Level skip!</p>
<h2>Subhead 2.1</h2>
<p>Back up...</p>
<h4>Dive! Dive!</h4>
<p>...and down</p>

<hr /><p id="footer">Copyright &copy; All Done</p>
</body>

转换代码

# Use XML only so that we can pretty-print the results; HTML works fine, too
doc = Nokogiri::XML(html,&:noblanks) # stripping whitespace allows indentation
doc.at('body').auto_section          # make the magic happen
puts doc.to_xhtml                    # show the result with indentation

结果

<body>
  <section>
    <h1>Main Section 1</h1>
    <p>Intro</p>
    <section>
      <h2>Subhead 1.1</h2>
      <p>Meat</p>
      <p>MOAR MEAT</p>
    </section>
    <section>
      <h2>Subhead 1.2</h2>
      <p>Meat</p>
      <section>
        <h3>Caveats</h3>
        <p>FYI</p>
        <section>
          <h4>ProTip</h4>
          <p>Get it done</p>
        </section>
      </section>
    </section>
    <section>
      <h2>Subhead 1.3</h2>
      <p>Meat</p>
    </section>
  </section>
  <section>
    <h1>Main Section 2</h1>
    <section>
      <h3>Jumpin' in it!</h3>
      <p>Level skip!</p>
    </section>
    <section>
      <h2>Subhead 2.1</h2>
      <p>Back up...</p>
      <section>
        <h4>Dive! Dive!</h4>
        <p>...and down</p>
      </section>
    </section>
  </section>
  <hr />
  <p id="footer">Copyright  All Done</p>
</body>

原始答案

这是一个不使用XPath的答案,但Nokogiri。我冒昧地使解决方案有点灵活,处理任意启动/停止(但不是嵌套部分)。

html = "<h2>Header</h2>
<p>First paragraph</p>
<p>Second paragraph</p>

<h2>Second header</h2>
<p>Third paragraph</p>
<p>Fourth paragraph</p>

<hr>
<p id='footer'>All done!</p>"

require 'nokogiri'
class Nokogiri::XML::Node
  # Provide a block that returns:
  #  true  - for nodes that should start a new section
  #  false - for nodes that should not start a new section
  #  :stop - for nodes that should stop any current section but not start a new one
  def group_under(name="section")
    group = nil
    element_children.each do |child|
      case yield(child)
        when false, nil
          group << child if group
        when :stop
          group = nil 
        else
          group = document.create_element(name)
          child.replace(group)
          group << child
      end
    end
  end
end

doc = Nokogiri::HTML(html)
doc.at('body').group_under do |node|
  if node.name == 'hr'
    :stop
  else
    %w[h1 h2 h3 h4 h5 h6].include?(node.name)
  end
end

puts doc
#=> <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN" "http://www.w3.org/TR/REC-html40/loose.dtd">
#=> <html><body>
#=> <section><h2>Header</h2>
#=> <p>First paragraph</p>
#=> <p>Second paragraph</p></section>
#=> 
#=> <section><h2>Second header</h2>
#=> <p>Third paragraph</p>
#=> <p>Fourth paragraph</p></section>
#=> 
#=> <hr>
#=> <p id="footer">All done!</p>
#=> </body></html>

对于XPath,请参阅XPath : select all following siblings until another sibling

答案 1 :(得分:2)

使用xpath的一种方法是选择跟随你的h2的所有p元素,并从中减去也跟随下一个h2的p元素:

doc = Nokogiri::HTML.fragment(html)
doc.css('h2').each do |h2|
    nodeset = h2.xpath('./following-sibling::p')
    next_h2 = h2.at('./following-sibling::h2')
    nodeset -= next_h2.xpath('./following-sibling::p') if next_h2
    section_tag = h2.add_previous_sibling Nokogiri::XML::Node.new('section',doc)
    h2.parent = section_tag
    nodeset.each {|n| n.parent = section_tag}
end

答案 2 :(得分:1)

XPath只能从输入文档中选择内容,不能将其转换为新文档。为此,您需要XSLT或其他一些转换语言。我想如果你进入Nokogiri那么之前的答案将是有用的,但为了完整性,这就是XSLT 2.0中的样子:

<xsl:for-each-group select="*" group-starting-with="h2">
  <section>
    <xsl:copy-of select="current-group()"/>
  </section>
</xsl:for-each-group>