用于抓取数据的scrapy请求表单

时间:2014-03-28 07:59:16

标签: html web-crawler scrapy

我正在使用此代码,我正在使用url = http://money.moneygram.com.au/forex-tools/currency-converter-widget-part

from __future__ import absolute_import
#import __init__
#from scrapy.spider import BaseSpider
from scrapy.contrib.spiders import CrawlSpider, Rule
from scrapy.http import Request
from scrapy.http import FormRequest
from scrapy.http import Response
from scrapy.selector import HtmlXPathSelector
import MySQLdb
class DmozSpider(CrawlSpider):
    name = "moneygram"
    allowed_domains = ["moneygram.com"]
    start_urls = ["http://money.moneygram.com.au/forex-tools/currency-converter-widget-part"]

    def parse(self,response):
           # yield FormRequest.from_response(response,formname='firstSelector',formdata="FromCurrency=USD&ToCurrency=INR&FromCurrency_dropDown=USD&ToCurrency_dropDown=INR",callback=self.parse1)
           # request_with_cookies = Request(url="http://money.moneygram.com.au",
           #                    cookies={'FromCurrency': 'USD', 'ToCurrency': 'INR'},callback=self.parse1)

            yield FormRequest.from_response(response,formname=None,formnumber=0,formpath=None,formdata="FromCurrency=AED&ToCurrency=VND&FromCurrency_dropDown=AED&ToCurrency_dropDown=VND&FromAmount=2561&ToAmount=&X-Requested-With=XMLHttpRequest",callback=self.parse1)

根据需要向我发送表单数据但是给出错误

raise ValueError("No <form> element found in %s" % response)
exceptions.ValueError: No <form> element found in <200 http://money.moneygram.com.au/forex-tools/currency-converter-widget-part> 

如何从usd转换为inr?

1 个答案:

答案 0 :(得分:0)

您提到的页面不是有效的HTML。看起来它是一个SSI块,它应该是另一个页面的一部分。

您的错误,因为此页面大量使用Javascript。它没有 FormRequest.from_response尝试使用现有表单发出请求,但表单不存在。

您应该使用整个页面,或者手动填写FormRequest填充它的属性,然后将其从整个页面提交到URL。