Question

我正在使用cl /E ..

来删除数据

这是单一帖子的检查元素

lxml

对于每个这样的帖子，我想提取标题，帖子内容和发布时间。

例如上面的细节将是

<article id="post-4855" class="post-4855 post type-post status-publish format-standard hentry category-uncategorized">


<header class="entry-header">
    <h1 class="entry-title"><a href="http://aitplacements.com/uncategorized/cybage/" rel="bookmark">Cybage..</a></h1>
            <div class="entry-meta">
        <span class="byline"> Posted by <span class="author vcard"><a class="url fn n" href="http://aitplacements.com/author/tpoait/">TPO</a></span></span><span class="posted-on"> on <a href="http://aitplacements.com/uncategorized/cybage/" rel="bookmark"><time class="entry-date published updated" datetime="2017-09-13T11:02:32+00:00">September 13, 2017</time></a></span><span class="comments-link"> with <a href="http://aitplacements.com/uncategorized/cybage/#respond">0 Comment</a></span>      </div><!-- .entry-meta -->
        </header><!-- .entry-header -->

<div class="entry-content">
    <p>cybage placement details shared <a href="http://aitplacements.com/uncategorized/cybage/" class="read-more">READ MORE</a></p>
        </div><!-- .entry-content -->

直到我能够实现的目标：该网站需要登录，我是成功的。

用于提取信息：

{title : "Cybage..",
 post : "cybage placement details shared"
 datetime="2017-09-13T11:02:32+00:00"
}

raw_title给出空值headers = {'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) Chrome/42.0.2311.90'} url = 'http://aitplacements.com/news/' page = requests.get(url,headers=headers) doc = html.fromstring(page.content) #print doc # it prints <Element html at 0x7f59c38d2260> raw_title = doc.xpath('//h1[@class="entry-title"]/a/@href/text()') print raw_title？

我做错了什么？

Answer 1

@href是指href属性的值：

In [14]: doc.xpath('//h1[@class="entry-title"]/a/@href')
Out[14]: ['http://aitplacements.com/uncategorized/cybage/']

您想要<a>元素的文字：

In [16]: doc.xpath('//h1[@class="entry-title"]/a/text()')
Out[16]: ['Cybage..']

因此，请使用

raw_title = doc.xpath('//h1[@class="entry-title"]/a/text()')
if len(raw_title) > 0:
    raw_title = raw_title[0]
else:
    # handle the case of missing title
    raise ValueError('Missing title')

html使用lxml进行抓取

1 个答案: