Question

我在Windows Vista 64位上运行Scrapy.org 2.7版64位。我有一些Scrapy代码尝试在包含在以下代码中包含的URL的表中包含的解析数据：

from scrapy.spider import Spider
from scrapy.selector import Selector
from scrapy.utils.markup import remove_tags
from scrapy.cmdline import execute
import re


class MySpider(Spider):
    name = "wiki"
    allowed_domains = ["whoscored.com"]
    start_urls = ["http://www.whoscored.com/Players/3859/Fixtures/Wayne-Rooney"]

def parse(self, response):

    for row in response.selector.xpath('//table[@id="player-fixture"]//tr[td[@class="tournament"]]'):
    # Is this row contains goal symbols?
        list_of_goals = row.xpath('//span[@title="Goal"')
        if list_of_goals:
            print remove_tags(list_of_goals).encode('utf-8')     

execute(['scrapy','crawl','wiki'])

然而，它引发了以下错误：

Traceback (most recent call last):
  File "c:\Python27\lib\site-packages\twisted\internet\base.py", line 1201, in mainLoop
    self.runUntilCurrent()
  File "c:\Python27\lib\site-packages\twisted\internet\base.py", line 824, in runUntilCurrent
    call.func(*call.args, **call.kw)
  File "c:\Python27\lib\site-packages\twisted\internet\defer.py", line 383, in callback
    self._startRunCallbacks(result)
  File "c:\Python27\lib\site-packages\twisted\internet\defer.py", line 491, in _startRunCallbacks
    self._runCallbacks()
--- <exception caught here> ---
  File "c:\Python27\lib\site-packages\twisted\internet\defer.py", line 578, in _runCallbacks
    current.result = callback(current.result, *args, **kw)
  File "c:\Python27\lib\site-packages\scrapy\spider.py", line 56, in parse
    raise NotImplementedError
exceptions.NotImplementedError:

谁能告诉我这里的问题是什么？我正在尝试获取表格中所有项目的屏幕打印，包括目标和助手列中的数据。

由于

Answer 1

你的缩进是错误的：

class MySpider(Spider):
    name = "wiki"
    allowed_domains = ["whoscored.com"]
    start_urls = ["http://www.whoscored.com/Players/3859/Fixtures/Wayne-Rooney"]

    def parse(self, response):

        for row in response.selector.xpath('//table[@id="player-fixture"]//tr[td[@class="tournament"]]'):
        # Is this row contains goal symbols?
            list_of_goals = row.xpath('//span[@title="Goal"')
            if list_of_goals:
                print remove_tags(list_of_goals).encode('utf-8')

当您使用parse时，需要实现Spider class方法，这就是源代码中的方法：

def parse(self, response):
        raise NotImplementedError

您的缩进错误，因此解析不属于该类，因此您没有实现所需的方法。

raise NotImplementedError用于确保在继承parse基类时编写所需的Spider方法。

您现在只需要找到正确的xpath;）

在尝试解析列表数据时Scrapy抛出Traceback

1 个答案: