AttributeError: 'Request' object has no attribute 'method'

时间:2015-09-01 22:08:38

标签: python web-crawler scrapy

I am getting a bizarre AttributeError: 'Request' object has no attribute 'method' Here is the original code:

class ModularSpider(CrawlSpider):
    name = 'modular_spider'
    allowed_domains = ['fatsecret.com']
    login_url = 'https://www.fatsecret.com/Auth.aspx?pa=s'
    start_urls = ['http://www.fatsecret.com/member/dorindam59']

    rules = [
    # main rule - process members page
        Rule(
            LinkExtractor(
                allow='member\/[^\/\?]+$',
                deny='inweb'  # deny my own user
            ),
            follow=True,
            callback='schedule'
        ),
    ]

    def schedule(self, response):
        print 10 * "$", response.url
        import pdb
        pdb.set_trace()
        r = Request(url=response.url
                    # callback=users.parse_user,
                    # dont_filter=True,
                    # priority=1
                    )
        print 10 * "END"
        print r
        return

... (code continues)

and here are some investigation I have made

> /Users/thalesfc/Envs/inweb/lib/python2.7/site-packages/scrapy/http/response/__init__.py(34)_get_url()
-> def _get_url(self):
(Pdb) s
> /Users/thalesfc/Envs/inweb/lib/python2.7/site-    packages/scrapy/http/response/__init__.py(35)_get_url()
-> return self._url
(Pdb) s
--Return--
> /Users/thalesfc/Envs/inweb/lib/python2.7/site-packages/scrapy/http/response/__init__.py(35)_get_url()-    >'http://www.f...onna+Mashburn'
-> return self._url
(Pdb) s
--Call--
> /Users/thalesfc/Envs/inweb/lib/python2.7/site-    packages/scrapy/utils/trackref.py(28)__new__()
-> def __new__(cls, *args, **kwargs):
(Pdb) args
cls = <class 'scrapy.http.request.Request'>
args = ()
kwargs = {'url': 'http://www.fatsecret.com/member/Donna+Mashburn'}
(Pdb) object.__new__(cls)
*** AttributeError: 'Request' object has no attribute 'method'
(Pdb) cls
<class 'scrapy.http.request.Request'>

I know maybe I just made some basic mistake, but could not find something related here or on StackOverflow.

  • PS: I am running scrapy version 1.0.3
  • PS2: I already tried reinstalling scrapy
  • PS3: When I open a normal ipython, I can create a Request without problem :p

1 个答案:

答案 0 :(得分:0)

对我来说它在1.0.3 scrapy版本中运行良好,也许它值得删除你的.pyc文件。在这种情况下,这可能是一个问题

包含已编译代码的.pyc文件通常会在与.py文件相同的目录中创建。