我从项目的顶级目录中运行scrapy shell myurl
。以下是日志输出:
$ scrapy shell "http://www.aufeminin.com/forum/show0_3/grossesse.html"
2013-11-07 12:02:08+0100 [scrapy] INFO: Scrapy 0.14.4 started (bot: auf)
2013-11-07 12:02:08+0100 [scrapy] DEBUG: Enabled extensions: TelnetConsole, CloseSpider, WebService, CoreStats, MemoryUsage, SpiderState
2013-11-07 12:02:09+0100 [scrapy] DEBUG: Enabled downloader middlewares: HttpAuthMiddleware, DownloadTimeoutMiddleware, UserAgentMiddleware, RetryMiddleware, DefaultHeadersMiddleware, RedirectMiddleware, CookiesMiddleware, HttpCompressionMiddleware, ChunkedTransferMiddleware, DownloaderStats
2013-11-07 12:02:09+0100 [scrapy] DEBUG: Enabled spider middlewares: HttpErrorMiddleware, OffsiteMiddleware, RefererMiddleware, UrlLengthMiddleware, DepthMiddleware
2013-11-07 12:02:09+0100 [scrapy] DEBUG: Enabled item pipelines:
2013-11-07 12:02:09+0100 [scrapy] DEBUG: Telnet console listening on 0.0.0.0:6023
2013-11-07 12:02:09+0100 [scrapy] DEBUG: Web service listening on 0.0.0.0:6080
2013-11-07 12:02:09+0100 [AufemininKeywordClassifierSpider] INFO: Spider opened
2013-11-07 12:02:09+0100 [AufemininKeywordClassifierSpider] DEBUG: Crawled (200) <GET http://www.aufeminin.com/forum/show0_3/grossesse.html> (referer: None)
[s] Available Scrapy objects:
[s] hxs <HtmlXPathSelector xpath=None data=u'<html>\r\n<head>\r\n<title>Forum Grossesse -'>
[s] item {}
[s] request <GET http://www.aufeminin.com/forum/show0_3/grossesse.html>
[s] response <200 http://www.aufeminin.com/forum/show0_3/grossesse.html>
[s] settings <CrawlerSettings module=<module 'auf.settings' from '/home/louist/Desktop/auf/auf/settings.pyc'>>
[s] spider <AufKeywordSpider 'AufemininKeywordClassifierSpider' at 0x3d41ed0>
[s] Useful shortcuts:
[s] shelp() Shell help (print this help)
[s] fetch(req_or_url) Fetch request (or URL) and update local objects
[s] view(response) View response in a browser
教程建议我应该有一个名为sel
的变量,它包含一个选择器。为什么不是这种情况?
答案 0 :(得分:2)
hxs
在您的案例中包含选择器。
答案 1 :(得分:1)
我认为这是因为您正在阅读最新版本(0.19
)的文档,并且您安装的版本较低。用法非常相似。