网络评论家Metacritic:评论

时间:2019-04-01 22:44:35

标签: python web-scraping beautifulsoup

¿我已经为Metacritic开发了一个Web抓取工具,可以在其中获得评论,但是现在我必须更改抓取方式,并且必须在一个地方拥有良好的评论(8-10),中立的评论(5 -8)和差评(1-5)。当我抓取我的评论时,应该将其分为这三个类别!

urls = ['https://www.metacritic.com/game/pc/league-of-legends/user-reviews']

def scrape_metacritic(self, urls):
    """
    Public method that extracts all the reviews given a list of Metacritic URLS
    :param urls:
    :return:
    """
    data = []
    for url in urls:
        start = 0
        next = 1
        while start < next:
            html = self.__make_request(url + f'user-reviews?page={str(start)}')
            blocks = self.__div_blocks(html)
            reviews = self.__get_comments(blocks)
            if len(reviews) is 0:
                start = next
            else:
                for review in reviews:
                    data.append(review)
                start += 1
                next += 1
    return data

函数是什么?

1 个答案:

答案 0 :(得分:0)

最好使用列表字典。

data = {'bad': [], 'neutral': [], 'good': []}

然后,当您遍历评论时,将附加到正确的列表中。

if review.score < 8 and review.score > 4:
    data['neutral'].append(review)