由于一些不寻常的原因，Scrapy不断投掷＆＃34;达到最大重定向＆＃34;即使它所说的重定向到的URL是相同的......
In [1]: fetch('http://www.website.com/ap/001/new')
2018-03-16 20:50:06 [scrapy.core.engine] INFO: Spider opened
2018-03-16 20:50:06 [scrapy.downloadermiddlewares.redirect] DEBUG: Redirecting (meta refresh) to <GET http://www.website.com/ap/001/new> from <GET http://www.website.com/ap/001/new>
2018-03-16 20:50:07 [scrapy.downloadermiddlewares.redirect] DEBUG: Redirecting (meta refresh) to <GET http://www.website.com/ap/001/new> from <GET http://www.website.com/ap/001/new>
2018-03-16 20:50:07 [scrapy.downloadermiddlewares.redirect] DEBUG: Redirecting (meta refresh) to <GET http://www.website.com/ap/001/new> from <GET http://www.website.com/ap/001/new>
2018-03-16 20:50:07 [scrapy.downloadermiddlewares.redirect] DEBUG: Redirecting (meta refresh) to <GET http://www.website.com/ap/001/new> from <GET http://www.website.com/ap/001/new>
2018-03-16 20:50:07 [scrapy.downloadermiddlewares.redirect] DEBUG: Redirecting (meta refresh) to <GET http://www.website.com/ap/001/new> from <GET http://www.website.com/ap/001/new>
2018-03-16 20:50:07 [scrapy.downloadermiddlewares.redirect] DEBUG: Redirecting (meta refresh) to <GET http://www.website.com/ap/001/new> from <GET http://www.website.com/ap/001/new>
2018-03-16 20:50:07 [scrapy.downloadermiddlewares.redirect] DEBUG: Redirecting (meta refresh) to <GET http://www.website.com/ap/001/new> from <GET http://www.website.com/ap/001/new>
2018-03-16 20:50:07 [scrapy.downloadermiddlewares.redirect] DEBUG: Redirecting (meta refresh) to <GET http://www.website.com/ap/001/new> from <GET http://www.website.com/ap/001/new>
2018-03-16 20:50:07 [scrapy.downloadermiddlewares.redirect] DEBUG: Redirecting (meta refresh) to <GET http://www.website.com/ap/001/new> from <GET http://www.website.com/ap/001/new>
2018-03-16 20:50:07 [scrapy.downloadermiddlewares.redirect] DEBUG: Redirecting (meta refresh) to <GET http://www.website.com/ap/001/new> from <GET http://www.website.com/ap/001/new>
2018-03-16 20:50:07 [scrapy.downloadermiddlewares.redirect] DEBUG: Redirecting (meta refresh) to <GET http://www.website.com/ap/001/new> from <GET http://www.website.com/ap/001/new>
2018-03-16 20:50:07 [scrapy.downloadermiddlewares.redirect] DEBUG: Redirecting (meta refresh) to <GET http://www.website.com/ap/001/new> from <GET http://www.website.com/ap/001/new>
2018-03-16 20:50:07 [scrapy.downloadermiddlewares.redirect] DEBUG: Redirecting (meta refresh) to <GET http://www.website.com/ap/001/new> from <GET http://www.website.com/ap/001/new>
2018-03-16 20:50:07 [scrapy.downloadermiddlewares.redirect] DEBUG: Redirecting (meta refresh) to <GET http://www.website.com/ap/001/new> from <GET http://www.website.com/ap/001/new>
2018-03-16 20:50:07 [scrapy.downloadermiddlewares.redirect] DEBUG: Redirecting (meta refresh) to <GET http://www.website.com/ap/001/new> from <GET http://www.website.com/ap/001/new>
2018-03-16 20:50:08 [scrapy.downloadermiddlewares.redirect] DEBUG: Redirecting (meta refresh) to <GET http://www.website.com/ap/001/new> from <GET http://www.website.com/ap/001/new>
2018-03-16 20:50:08 [scrapy.downloadermiddlewares.redirect] DEBUG: Redirecting (meta refresh) to <GET http://www.website.com/ap/001/new> from <GET http://www.website.com/ap/001/new>
2018-03-16 20:50:08 [scrapy.downloadermiddlewares.redirect] DEBUG: Redirecting (meta refresh) to <GET http://www.website.com/ap/001/new> from <GET http://www.website.com/ap/001/new>
2018-03-16 20:50:08 [scrapy.downloadermiddlewares.redirect] DEBUG: Redirecting (meta refresh) to <GET http://www.website.com/ap/001/new> from <GET http://www.website.com/ap/001/new>
2018-03-16 20:50:08 [scrapy.downloadermiddlewares.redirect] DEBUG: Redirecting (meta refresh) to <GET http://www.website.com/ap/001/new> from <GET http://www.website.com/ap/001/new>
2018-03-16 20:50:08 [scrapy.downloadermiddlewares.redirect] DEBUG: Discarding <GET http://www.website.com/ap/001/new>: max redirections reached
如果我满足要求＆amp; urllib2但scrapy它失败了...是什么导致这个＆amp;我如何解决它。我无法在stackoverflow / google上找到符合此问题的任何内容。
我已经在我的浏览器和＃34;重定向检查器＆＃34;中检查了URL。并且没有人认为该网站正在重定向，所以不确定为什么scrapy认为它是重定向的。