scrapy输出标题和相关链接

Question

我的scrapy蜘蛛向我展示了所有网页的标题。 请告诉我如何显示与该标题相关的标题和链接？ 我想要解析this页面。 我的代码：

from scrapy.contrib.spiders import CrawlSpider, Rule
from scrapy.contrib.linkextractors.sgml import SgmlLinkExtractor
from scrapy.selector import HtmlXPathSelector
from probe1.items import SpiderItem

class SpiderSpider(CrawlSpider):
    name = "spider"
    allowed_domains = ["WEB_PAGE"]
    start_urls = [
    "http://www.WEB_PAGE"
    ]

    rules = (
        Rule(
            SgmlLinkExtractor(allow_domains=("WEB_PAGE",)),
            callback='parse_page', follow=True
        ),
    )


    def parse_page(self, response):
      hxs = HtmlXPathSelector(response)
      print hxs
      sites = hxs.select('//title')
      items = []
      for s in sites:
      item = SpiderItem()
          item['title'] = s.select('//title').extract
          items.append(item)
      return items   

Answer 1

response.url包含您需要的内容：

  

URL

     

包含响应网址的字符串。