我正在尝试通过API将表单值发布到URL并获取结果。我能够传递表单数据,但检索空值。
$ scrapy crawl fake
2018-06-14 15:31:19 [scrapy.utils.log] INFO: Scrapy 1.5.0 started (bot: spin)
2018-06-14 15:31:19 [scrapy.utils.log] INFO: Versions: lxml 3.5.0.0, libxml2 2.9.3, cssselect 1.0.3, parsel 1.4.0, w3lib 1.19.0, Twisted 18.4.0, Python 3.5.2 (default, Nov 23 2017, 16:37:01) - [GCC 5.4.0 20160609], pyOpenSSL 18.0.0 (OpenSSL 1.0.2g 1 Mar 2016), cryptography 2.2.2, Platform Linux-4.4.0-127-generic-x86_64-with-Ubuntu-16.04-xenial
2018-06-14 15:31:19 [scrapy.crawler] INFO: Overridden settings: {'ROBOTSTXT_OBEY': True, 'BOT_NAME': 'spin', 'NEWSPIDER_MODULE': 'spin.spiders', 'SPIDER_MODULES': ['spin.spiders']}
2018-06-14 15:31:19 [scrapy.middleware] INFO: Enabled extensions:
['scrapy.extensions.corestats.CoreStats',
'scrapy.extensions.memusage.MemoryUsage',
'scrapy.extensions.telnet.TelnetConsole',
'scrapy.extensions.logstats.LogStats']
2018-06-14 15:31:19 [scrapy.middleware] INFO: Enabled downloader middlewares:
['scrapy.downloadermiddlewares.robotstxt.RobotsTxtMiddleware',
'scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
'scrapy.downloadermiddlewares.retry.RetryMiddleware',
'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
'scrapy.downloadermiddlewares.stats.DownloaderStats']
2018-06-14 15:31:19 [scrapy.middleware] INFO: Enabled spider middlewares:
['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',
'scrapy.spidermiddlewares.referer.RefererMiddleware',
'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
'scrapy.spidermiddlewares.depth.DepthMiddleware']
2018-06-14 15:31:19 [scrapy.middleware] INFO: Enabled item pipelines:
['scrapy.pipelines.images.ImagesPipeline']
2018-06-14 15:31:19 [scrapy.core.engine] INFO: Spider opened
2018-06-14 15:31:19 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2018-06-14 15:31:19 [scrapy.extensions.telnet] DEBUG: Telnet console listening on 127.0.0.1:6023
2018-06-14 15:31:19 [scrapy.core.engine] DEBUG: Crawled (404) <GET http://books.toscrape.com/robots.txt> (referer: None)
2018-06-14 15:31:19 [scrapy.core.engine] DEBUG: Crawled (200) <GET http://books.toscrape.com> (referer: None)
2018-06-14 15:31:19 [fake] DEBUG: Saved file books.toscrape.com/index.html
2018-06-14 15:31:19 [py.warnings] WARNING: /sne/home/shaanen/codeprojects/rp2/scraper/spin/spiders/fake_spider.py:68: ScrapyDeprecationWarning: Call to deprecated function select. Use .xpath() instead.
for url in hxs.select('//a/@href').extract():
2018-06-14 15:31:20 [scrapy.core.scraper] DEBUG: Scraped from <200 http://books.toscrape.com>
{'image_urls': ['media/cache/2c/da/2cdad67c44b002e7ead0cc35693c0e8b.jpg'],
'images': ['media/cache/2c/da/2cdad67c44b002e7ead0cc35693c0e8b.jpg']}
2018-06-14 15:31:20 [scrapy.core.scraper] DEBUG: Scraped from <200 http://books.toscrape.com>
{'image_urls': ['media/cache/26/0c/260c6ae16bce31c8f8c95daddd9f4a1c.jpg'],
'images': ['media/cache/26/0c/260c6ae16bce31c8f8c95daddd9f4a1c.jpg']}
2018-06-14 15:31:20 [scrapy.core.scraper] DEBUG: Scraped from <200 http://books.toscrape.com>
{'image_urls': ['media/cache/3e/ef/3eef99c9d9adef34639f510662022830.jpg'],
'images': ['media/cache/3e/ef/3eef99c9d9adef34639f510662022830.jpg']}
2018-06-14 15:31:20 [scrapy.core.scraper] DEBUG: Scraped from <200 http://books.toscrape.com>
{'image_urls': ['media/cache/32/51/3251cf3a3412f53f339e42cac2134093.jpg'],
'images': ['media/cache/32/51/3251cf3a3412f53f339e42cac2134093.jpg']}
2018-06-14 15:31:20 [scrapy.core.scraper] DEBUG: Scraped from <200 http://books.toscrape.com>
{'image_urls': ['media/cache/be/a5/bea5697f2534a2f86a3ef27b5a8c12a6.jpg'],
'images': ['media/cache/be/a5/bea5697f2534a2f86a3ef27b5a8c12a6.jpg']}
2018-06-14 15:31:20 [scrapy.core.scraper] DEBUG: Scraped from <200 http://books.toscrape.com>
{'image_urls': ['media/cache/68/33/68339b4c9bc034267e1da611ab3b34f8.jpg'],
'images': ['media/cache/68/33/68339b4c9bc034267e1da611ab3b34f8.jpg']}
2018-06-14 15:31:20 [scrapy.core.scraper] DEBUG: Scraped from <200 http://books.toscrape.com>
{'image_urls': ['media/cache/92/27/92274a95b7c251fea59a2b8a78275ab4.jpg'],
'images': ['media/cache/92/27/92274a95b7c251fea59a2b8a78275ab4.jpg']}
2018-06-14 15:31:20 [scrapy.core.scraper] DEBUG: Scraped from <200 http://books.toscrape.com>
{'image_urls': ['media/cache/3d/54/3d54940e57e662c4dd1f3ff00c78cc64.jpg'],
'images': ['media/cache/3d/54/3d54940e57e662c4dd1f3ff00c78cc64.jpg']}
2018-06-14 15:31:20 [scrapy.core.scraper] DEBUG: Scraped from <200 http://books.toscrape.com>
{'image_urls': ['media/cache/66/88/66883b91f6804b2323c8369331cb7dd1.jpg'],
'images': ['media/cache/66/88/66883b91f6804b2323c8369331cb7dd1.jpg']}
2018-06-14 15:31:20 [scrapy.core.scraper] DEBUG: Scraped from <200 http://books.toscrape.com>
{'image_urls': ['media/cache/58/46/5846057e28022268153beff6d352b06c.jpg'],
'images': ['media/cache/58/46/5846057e28022268153beff6d352b06c.jpg']}
2018-06-14 15:31:20 [scrapy.core.scraper] DEBUG: Scraped from <200 http://books.toscrape.com>
{'image_urls': ['media/cache/be/f4/bef44da28c98f905a3ebec0b87be8530.jpg'],
'images': ['media/cache/be/f4/bef44da28c98f905a3ebec0b87be8530.jpg']}
2018-06-14 15:31:20 [scrapy.core.scraper] DEBUG: Scraped from <200 http://books.toscrape.com>
{'image_urls': ['media/cache/10/48/1048f63d3b5061cd2f424d20b3f9b666.jpg'],
'images': ['media/cache/10/48/1048f63d3b5061cd2f424d20b3f9b666.jpg']}
2018-06-14 15:31:20 [scrapy.core.scraper] DEBUG: Scraped from <200 http://books.toscrape.com>
{'image_urls': ['media/cache/5b/88/5b88c52633f53cacf162c15f4f823153.jpg'],
'images': ['media/cache/5b/88/5b88c52633f53cacf162c15f4f823153.jpg']}
2018-06-14 15:31:20 [scrapy.core.scraper] DEBUG: Scraped from <200 http://books.toscrape.com>
{'image_urls': ['media/cache/94/b1/94b1b8b244bce9677c2f29ccc890d4d2.jpg'],
'images': ['media/cache/94/b1/94b1b8b244bce9677c2f29ccc890d4d2.jpg']}
2018-06-14 15:31:20 [scrapy.core.scraper] DEBUG: Scraped from <200 http://books.toscrape.com>
{'image_urls': ['media/cache/81/c4/81c4a973364e17d01f217e1188253d5e.jpg'],
'images': ['media/cache/81/c4/81c4a973364e17d01f217e1188253d5e.jpg']}
2018-06-14 15:31:20 [scrapy.core.scraper] DEBUG: Scraped from <200 http://books.toscrape.com>
{'image_urls': ['media/cache/54/60/54607fe8945897cdcced0044103b10b6.jpg'],
'images': ['media/cache/54/60/54607fe8945897cdcced0044103b10b6.jpg']}
2018-06-14 15:31:20 [scrapy.core.scraper] DEBUG: Scraped from <200 http://books.toscrape.com>
{'image_urls': ['media/cache/55/33/553310a7162dfbc2c6d19a84da0df9e1.jpg'],
'images': ['media/cache/55/33/553310a7162dfbc2c6d19a84da0df9e1.jpg']}
2018-06-14 15:31:20 [scrapy.core.scraper] DEBUG: Scraped from <200 http://books.toscrape.com>
{'image_urls': ['media/cache/09/a3/09a3aef48557576e1a85ba7efea8ecb7.jpg'],
'images': ['media/cache/09/a3/09a3aef48557576e1a85ba7efea8ecb7.jpg']}
2018-06-14 15:31:20 [scrapy.core.scraper] DEBUG: Scraped from <200 http://books.toscrape.com>
{'image_urls': ['media/cache/0b/bc/0bbcd0a6f4bcd81ccb1049a52736406e.jpg'],
'images': ['media/cache/0b/bc/0bbcd0a6f4bcd81ccb1049a52736406e.jpg']}
2018-06-14 15:31:20 [scrapy.core.scraper] DEBUG: Scraped from <200 http://books.toscrape.com>
{'image_urls': ['media/cache/27/a5/27a53d0bb95bdd88288eaf66c9230d7e.jpg'],
'images': ['media/cache/27/a5/27a53d0bb95bdd88288eaf66c9230d7e.jpg']}
2018-06-14 15:31:20 [scrapy.core.engine] INFO: Closing spider (finished)
2018-06-14 15:31:20 [scrapy.statscollectors] INFO: Dumping Scrapy stats:
{'downloader/request_bytes': 444,
'downloader/request_count': 2,
'downloader/request_method_count/GET': 2,
'downloader/response_bytes': 6204,
'downloader/response_count': 2,
'downloader/response_status_count/200': 1,
'downloader/response_status_count/404': 1,
'finish_reason': 'finished',
'finish_time': datetime.datetime(2018, 6, 14, 13, 31, 20, 12795),
'item_scraped_count': 20,
'log_count/DEBUG': 24,
'log_count/INFO': 7,
'log_count/WARNING': 1,
'memusage/max': 52318208,
'memusage/startup': 52318208,
'response_received_count': 2,
'scheduler/dequeued': 1,
'scheduler/dequeued/memory': 1,
'scheduler/enqueued': 1,
'scheduler/enqueued/memory': 1,
'start_time': datetime.datetime(2018, 6, 14, 13, 31, 19, 840581)}
2018-06-14 15:31:20 [scrapy.core.engine] INFO: Spider closed (finished)
我有什么遗失的吗?