我正在尝试使用Python 3.x从Amazon收集评论,但什么都没得到

时间:2016-06-30 23:15:45

标签: python scrapy amazon screen-scraping review

我想废除亚马逊的所有评论。当我在下面运行我的代码时,它返回空列表而没有任何错误。我无法弄清楚为什么。你能帮我吗?

from __future__ import unicode_literals

import requests
from scrapy.selector import Selector

def fetch_page(url):
    r = requests.get(url)
    return r.text 

def  review_positive(url):
    #html = fetch_page(url)
    sel = Selector(text = url)
    review =  sel.css(' .a-section review').extract()
    return review


print (review_positive('https://www.amazon.com/Apple-iPhone-Unlocked-GB-Packaging/product-reviews/B01DAJT1AW/ref=cm_cr_arp_d_viewpnt_lft?ie=UTF8&showViewpoints=1&sortBy=helpful&filterByStar=positive&pageNumber=1'))

1 个答案:

答案 0 :(得分:1)

类之间的CSS选择器中不应该有空格。相反,使用.分隔类:

.css('.a-section.review').extract()

修复后,您的选择器将正常工作:

In [6]: rev = response.css('.a-section.review').extract()

In [7]: len(rev)
Out[7]: 10

In [8]: rev = response.css('.a-section review').extract()

In [9]: len(rev)
Out[9]: 0