Question

我尝试用scrapy废弃一页。我用FireXpath（一个firefox插件）找到了xpath，看起来不错。但是使用Scrapy，我没有结果。

我的python程序看起来像这样：

#!/usr/bin/python -tt
# -*- coding: utf-8 -*-

from scrapy.selector import Selector
from scrapy.contrib.spiders import CrawlSpider
from datetime import datetime
from scrapy.spider import BaseSpider

class robtex(BaseSpider):
    # Crawling Start
    CrawlSpider.started_on = datetime.now()
    # CrawlSpider
    name = 'robtex'

    DOWNLOAD_DELAY = 3
    start_urls = [ "https://www.whois.com/en/advisory/dns/com/stackoverflow/whois.html"]


    def parse(self, response):
        # Selector
        sel = Selector(response)
        print sel.xpath(".//*[@id='datawhois']/div[2]/table[3]/tbody/tr[3]/td[2]/a/text()").extract()

如何解决？

提前致谢。

Answer 1

您只需要从XPath表达式中删除tbody：

.//*[@id='datawhois']/div[2]/table[3]/tr[3]/td[2]/a/text()

演示：

$ scrapy shell https://www.robtex.com/en/advisory/dns/com/stackoverflow/whois.html
In [1]: response.xpath(".//*[@id='datawhois']/div[2]/table[3]/tbody/tr[3]/td[2]/a/text()").extract()
Out[1]: []

In [2]: response.xpath(".//*[@id='datawhois']/div[2]/table[3]/tr[3]/td[2]/a/text()").extract()
Out[2]: [u'STACKOVERFLOW.COM']

我的xpath很好，但我对scrapy一无所知

1 个答案: