Question

<div class="region size2of3">
<h2>Mumbai</h2>
<strong>Fort</strong>
<div>Elphinstone building, Horniman Circle,</div>
<div>Veer Nariman Road, Fort</div>
<div>Mumbai 400001</div>
<div>Timings: 08:00-00:30 hrs (Mon-Sun)</div>
<div><br></div>
</div>

我想排除＆＃34;时间：08：00-00：30（周一至周日）＆＃34;解析时使用div标记。

这是我的代码：

import scrapy
from job.items import StarbucksItem

class StarbucksSpider(scrapy.Spider):
    name = "starbucks"
    allowed_domains = ["starbucks.in"]
    start_urls = ["http://www.starbucks.in/coffeehouse/store-locations/"]

    def parse(self, response):
        for sel in response.xpath('//div[@class="region size2of3"]'):
            item = StarbucksItem()
            item['title'] = sel.xpath('div/text()').extract()
        yield item

Answer 1

我会使用starts-with() XPath function来获取以“Timings”开头的div元素的文字：

sel.xpath('.//div[starts-with(., "Timings")]/text()').extract()

请注意，页面的HTML结构不容易区分彼此之间的位置 - 没有可以迭代的特定于位置的容器。在这种情况下，我会找到每个h2或strong代码并使用following-sibling，例如来自Scrapy Shell：

In [10]: for sel in response.xpath('//div[contains(@class, "region")]/*[self::h2 or self::strong]'):
            name = sel.xpath('text()').extract()[0]
            timings = sel.xpath('./following-sibling::div[starts-with(., "Timings")]/text()').extract()[0]
            print name, timings
   ....:     
Mumbai Timings: 08:00-00:30 hrs (Mon-Sun)
Fort Timings: 08:00-00:30 hrs (Mon-Sun)
Colaba Timings: 07:00-01:00 hrs (Mon-Sun)
Goregaon Timings: 10:00-23:30 hrs (Mon-Sun)
Powai Timings: 07:00-00:00 hrs (Mon-Sun)
...
Hi-Tech City Timings: 09:00 - 22:30 hrs (Mon - Sun)
Madhapur Timings: 11:00 -23:00 hrs (Mon - Sun)
Banjara Hills Timings: 10:00 -22:30 hrs (Mon - Sun)

另请注意，如果要提取时间范围值，可以使用.re()：

In [18]: for sel in response.xpath('//div[contains(@class, "region")]/*[self::h2 or self::strong]'):
        name = sel.xpath('text()').extract()[0]
        timings = sel.xpath('./following-sibling::div[starts-with(., "Timings")]/text()')[0].re(r'(\d+:\d+)\s*\-\s*(\d+:\d+)')[:2]
        print name, timings
Mumbai [u'08:00', u'00:30']
Fort [u'08:00', u'00:30']
Colaba [u'07:00', u'01:00']
Goregaon [u'10:00', u'23:30']
...
Hi-Tech City [u'09:00', u'22:30']
Madhapur [u'11:00', u'23:00']
Banjara Hills [u'10:00', u'22:30']

此外，请确保循环体内有yield（请参阅您发布的代码）。

如果您要排除Timings并获取其余的位置说明，请使用：

for sel in response.xpath('//div[contains(@class, "region")]/*[self::h2 or self::strong]'):
    print " ".join(item.strip() for item in sel.xpath('following-sibling::div[position() < 4 and not(starts-with(., "Timings"))]/text()').extract())

如何在使用scrapy时从多个标签中排除特定的html标签（没有任何id）？

1 个答案: