Question

我试图从一堆本地html文件中获取原始html。在获取原始文件时，我从这篇文章得到了一些帮助：

但我目前的代码生成整个文件而不是子集。现在我似乎错过了一条线，我可以选择一条我想要抓住的xpath。

以下是我目前的代码：

def stringify_children(node):
    from lxml.etree import tostring
    from itertools import chain
    parts = ([node.text] +
            list(chain(*([c.text, tostring(c), c.tail] for c
            in node.getchildren()))) +
            [node.tail])
    # filter removes possible Nones in texts and tails
    return ''.join(filter(None, parts))

for filename in os.listdir('../news/article/'):
    if (filename.endswith('.html') and not filename.startswith('._')):
        print filename;
        with open('../news/article/' + filename, "r") as f:
            page=f.read();
        tree=html.fromstring(page);
        maincontent = stringify_children(tree);
        print maincontent;

我的最终目标是能够在字符串中获取并输出到本地文件，只作为div。

以下是一个示例文件：

<html>

<head>
    <title>Title</title>
    <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.6/css/bootstrap.min.css">
</head>

<body>
    <div class="container">
        <div class="row">
            <div class="col-xs-4">
                <div class="left-bar"></div>
            </div>
            <div class="col-xs-4">
                <div class="middle-bar"></div>
            </div>
            <div class="col-xs-4">
                <div class="right-bar"></div>
            </div>
        </div>
        <div class="row">
            <div class="col-xs-3">
                <div class="navigation"></div>
            </div>
            <div class="col-xs-9">
                <div class="main-content">
                    Hello
                    <br>
                    <br><a href="http://www.stackexchange.com">Click here to visit stack exchange</a>
                    <h1>This is an introduction</h1>
                    <h3>This is the third header</h3>
                    <p>Lorem ipsum dolor sit amet.....</p>
                    <p>Lorem ipsum dolor sit amet.....</p>
                    <p>Lorem ipsum dolor sit amet.....</p>
                    <ul>
                        <li>list text</li>
                        <li>list text</li>
                        <li>list text</li>
                        <li>list text</li>
                    </ul>
                    <div class="row">
                        <div class="col-xs-4"><img src="#">More content 1</div>
                        <div class="col-xs-4"><img src="#">More content 2</div>
                        <div class="col-xs-4"><img src="#">More content 3</div>
                    </div>

                </div>
            </div>
        </div>
    </div>

</body>

</html>

我想获取maincontent类下面的所有内容。以下是此文件中该类的xpath：

XPath：/ html / body / div / div [2] / div [2] / div

该程序应输出以下内容：

                    Hello
                    <br>
                    <br><a href="http://www.stackexchange.com">Click here to visit stack exchange</a>
                    <h1>This is an introduction</h1>
                    <h3>This is the third header</h3>
                    <p>Lorem ipsum dolor sit amet.....</p>
                    <p>Lorem ipsum dolor sit amet.....</p>
                    <p>Lorem ipsum dolor sit amet.....</p>
                    <ul>
                        <li>list text</li>
                        <li>list text</li>
                        <li>list text</li>
                        <li>list text</li>
                    </ul>
                    <div class="row">
                        <div class="col-xs-4"><img src="#">More content 1</div>
                        <div class="col-xs-4"><img src="#">More content 2</div>
                        <div class="col-xs-4"><img src="#">More content 3</div>
                    </div>

Answer 1

您可以尝试使用BeautifulSoup。我不熟悉它，但你可以做这样的事情（或者更清洁，如果你读到BeautifulSoup：）

from bs4 import BeautifulSoup
soup = BeautifulSoup(open("input.html"), 'html')
x = soup.find_all(class_="main-content")
for line in x[0].contents:
    print line,

您将获得如下输出：

        Hello
         <br/>
<br/> <a href="http://www.stackexchange.com">Click here to visit stack exchange</a>
<h1>This is an introduction</h1>
<h3>This is the third header</h3>
<p>Lorem ipsum dolor sit amet.....</p>
<p>Lorem ipsum dolor sit amet.....</p>
<p>Lorem ipsum dolor sit amet.....</p>
<ul>
<li>list text</li>
<li>list text</li>
<li>list text</li>
<li>list text</li>
</ul>
<div class="row">
<div class="col-xs-4"><img src="#"/>More content 1</div>
<div class="col-xs-4"><img src="#"/>More content 2</div>
<div class="col-xs-4"><img src="#"/>More content 3</div>
</div>

BeautifulSoup将＆＃34;修复＆＃34; HTML语法，如从更改为
到它的内容，它将保持元素内部的间距。请参阅以下文档：https://www.crummy.com/software/BeautifulSoup/bs4/doc/

Answer 2

使用lxml：

from lxml import html

xm = html.fromstring(h)
div = xm.xpath("//div[@class='main-content']")[0]
print(div.text  + "".join(map(html.tostring, div.xpath("./*"))))

或者：

from lxml import html

xm = html.fromstring(h)
eles  = xm.xpath("//div[@class='main-content']/text() | //div[@class='main-content']/*")
print("".join([ele if  isinstance(ele, str) else html.tostring(ele) for ele in eles]))

如何从Python中的本地文件中获取某个XPath中的原始所有原始html

2 个答案: