Question

我将对此网站上的文章进行网络浏览。

这是我到目前为止所做的：

# HR Version
# the entire crawling process

openfile = open("data/HR.csv", "rb")
r = csv.reader(openfile)
HR_data = []

for i in r:
    url = i[0]
    print url  # to know the status of web crawling
    r = requests.get(url)
    data = html.fromstring(r.text)
    #Inspect line with text
    #//*[@id="article-details"]
    #<section class="entry-content clearfix" itemprop="articleBody"></section>
    texts = data.xpath("//*[@id="article-details"]/p/text()") 
    raw = ''.join(str(i.encode("utf-8")) for i in texts)
    finaldata = raw.replace('\r','').replace('\n','').replace('\r','').replace('\t','')    
    HR_data.append([finaldata])

openfile.close()

有问题的命令如下

texts = data.xpath("//*[@id="article-details"]/p/text()")

它来自这个特定的网页：http://hrmagazine.co.uk/article-details/internal-entrepreneurship-can-boost-your-business

在Firefox上使用Inspect Element，我发现“text”在以下段中，包含在以下部分中：

<article id="article-details">
#One <h2> element, followed by multiple <p> elements.
</article>

从文章中仅提取段落文本的正确XPath是什么？

Answer 1

你几乎写了正确的XPath。您需要替换p

上的h2

texts = data.xpath("//*[@id="article-details"]/h2/text()")

Xpath从网页中提取文章文本

1 个答案: