I am getting a bizarre AttributeError: 'Request' object has no attribute 'method' Here is the original code:
class ModularSpider(CrawlSpider):
name = 'modular_spider'
allowed_domains = ['fatsecret.com']
login_url = 'https://www.fatsecret.com/Auth.aspx?pa=s'
start_urls = ['http://www.fatsecret.com/member/dorindam59']
rules = [
# main rule - process members page
Rule(
LinkExtractor(
allow='member\/[^\/\?]+$',
deny='inweb' # deny my own user
),
follow=True,
callback='schedule'
),
]
def schedule(self, response):
print 10 * "$", response.url
import pdb
pdb.set_trace()
r = Request(url=response.url
# callback=users.parse_user,
# dont_filter=True,
# priority=1
)
print 10 * "END"
print r
return
... (code continues)
and here are some investigation I have made
> /Users/thalesfc/Envs/inweb/lib/python2.7/site-packages/scrapy/http/response/__init__.py(34)_get_url()
-> def _get_url(self):
(Pdb) s
> /Users/thalesfc/Envs/inweb/lib/python2.7/site- packages/scrapy/http/response/__init__.py(35)_get_url()
-> return self._url
(Pdb) s
--Return--
> /Users/thalesfc/Envs/inweb/lib/python2.7/site-packages/scrapy/http/response/__init__.py(35)_get_url()- >'http://www.f...onna+Mashburn'
-> return self._url
(Pdb) s
--Call--
> /Users/thalesfc/Envs/inweb/lib/python2.7/site- packages/scrapy/utils/trackref.py(28)__new__()
-> def __new__(cls, *args, **kwargs):
(Pdb) args
cls = <class 'scrapy.http.request.Request'>
args = ()
kwargs = {'url': 'http://www.fatsecret.com/member/Donna+Mashburn'}
(Pdb) object.__new__(cls)
*** AttributeError: 'Request' object has no attribute 'method'
(Pdb) cls
<class 'scrapy.http.request.Request'>
I know maybe I just made some basic mistake, but could not find something related here or on StackOverflow.
答案 0 :(得分:0)
对我来说它在1.0.3 scrapy版本中运行良好,也许它值得删除你的.pyc文件。在这种情况下,这可能是一个问题
包含已编译代码的.pyc文件通常会在与.py文件相同的目录中创建。