Question

我试图写一个蜘蛛来从Steam's top-sellers列表中获取一些信息。但是我的代码存在一些问题。我认为这是关于＆＃39; re＆＃39;模块，因为我无法在for循环中打印这些代码。并运行代码，它总是写＆＃34; []＆＃34;到我提供的文件。

def getDetail(self, url):
    source = self.getSource(url)
    pattern = re.compile('<div class="col search_name ellipsis"><span class="title">(.*?)</span>', re.S)
    items = re.findall(pattern, source)
    print(re.findall(pattern, source))
    number = 1
    for item in items:
        print('Crawling No.%d game' % number)
        print('Name: %s' % item[0])
        number += 1
        time.sleep(0.1)
    return items

这是我的全部代码。

import requests
import re
import time


class Spider(object):
    def __init__(self):
        self.siteURL = 'http://store.steampowered.com/search/?filter=topsellers'

    def getSource(self, url):
        user_agent = 'Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) ' \
                 'Chrome/45.0.2454.101 Safari/537.36'
        headers = {'User_agent': user_agent}
        r = requests.get(url, headers=headers)
        return r.text

    def getDetail(self, url):
        source = self.getSource(url)
        pattern = re.compile('<div class="col search_name ellipsis"><span class="title">(.*?)</span>', re.S)
        items = re.findall(pattern, source)
        print(re.findall(pattern, source))
        number = 1
        for item in items:
            print('Crawling No.%d game' % number)
            print('Name: %s' % item[0])
            number += 1
            time.sleep(0.1)
        return items

    def saveDetail(self):
        data = str(self.getDetail(self.siteURL))
        fileName = 'SteamTopseller.txt'
        f = open(fileName, 'wb')
        f.write(data.encode('utf-8'))
        print('Successfully written!')
        f.close()

if __name__ == '__main__':
    spider = Spider()
    spider.saveDetail()

请帮我解决这个小问题，谢谢！顺便说一句，我在python3中编码。

Answer 1

...

re.findall（pattern，string，flags = 0）

返回字符串中pattern的所有非重叠匹配，作为字符串列表。

因此，如果字符串中没有匹配项，则返回一个空列表，如[]。

+++++++++++++++++++++++++++++++++++++++++++++++ ++++++++

要跳过“[]”，您可以编码为

items = re.findall(pattern, source)
if items:
    print(items)

+++++++++++++++++++++++++++++++++++++++++++++++ +++++++++

删除换行符

def getDetail(self, url):
    source = self.getSource(url).replace("\r", "").replace("\n", "").replace("\t", "")

Python3中一点蜘蛛代码中的问题

1 个答案:

re.findall（pattern，string，flags = 0）