使用ruby从网站检索文档

时间:2013-01-29 14:59:29

标签: ruby rubygems nokogiri mechanize mechanize-ruby

我在ruby中编写了一个脚本,可以浏览网站并进入表单页面。填写表单页面后,脚本会点击提交按钮,然后会打开一个对话框,询问您在哪里保存它。我在尝试获取此文件时遇到问题。我在网上搜索过,无法找到任何东西。我将如何检索文档的文件名?

如果有人能帮助我,我真的很感激

我的代码如下:

browser = Mechanize.new

## CONSTANTS
LOGIN_URL = 'https://business.airtricity.com/ews/welcome.jsp'
HOME_PAGE_URL = 'https://business.airtricity.com/ews/welcome.jsp'
CONSUMPTION_REPORT_URL = 'https://business.airtricity.com/ews/touConsChart.jsp?custid=209495'
LOGIN = ""
PASS = ""
MPRN_GPRN_LCIS = "10000001534"
CONSUMPTION_DATE = "20/01/2013"
END_DATE = "27/01/2013"
DOWNLOAD = "DL"



### Login page
begin
    login_page = browser.get(LOGIN_URL)
rescue Mechanize::ResponseCodeError => exception
    login_page = exception.page
end

puts "+++++++++"
puts login_page.links
puts "+++++++++"

login_form = login_page.forms.first
login_form['userid'] = LOGIN
login_form['password'] = PASS
login_form['_login_form_'] = "yes"
login_form['ipAddress'] = "137.43.154.176"
login_form.submit



## home page
begin
    home_page = browser.get(HOME_PAGE_URL)
rescue Mechanize::ResponseCodeError => exception
    home_page = exception.page
end

puts "----------"
puts home_page.links
puts "----------"


# Consumption Report
begin
    Report_Page = browser.get(CONSUMPTION_REPORT_URL)
rescue Mechanize::ResponseCodeError => exception
    Report_Page = exception.page
end

puts "**********"
puts Report_Page.links
pp Report_Page
puts "**********"

Report_Form = Report_Page.forms.first
Report_Form['entity1'] = MPRN_GPRN_LCIS
Report_Form['start'] = CONSUMPTION_DATE
Report_Form['end'] = END_DATE
Report_Form['charttype'] = DOWNLOAD
Report_Form.submit

## Download Report

begin
    browser.pluggable_parser.csv = Mechanize::Download
    Download_Page = browser.get('https://business.airtricity.com/ews/touConsChart.jsp?custid=209495/meter_read_download_2013-1-20_2013-1-27.csv').save('Hello')
rescue Mechanize::ResponseCodeError => exception
    Download_Page = exception.page
end

2 个答案:

答案 0 :(得分:1)

http://mechanize.rubyforge.org/Mechanize.html#method-i-get_file
从url下载文件,机械化非常简单:

browser = Mechanize.new
file_url = 'https://raw.github.com/ragsagar/ragsagar.github.com/c5caa502f8dec9d5e3738feb83d86e9f7561bd5e/.html'
downloaded_file = browser.get_file file_url
File.open('new_file.txt', 'w') { |file| file.write downloaded_file }

答案 1 :(得分:0)

我看到自动化因浏览器代理而失败。也许你可以试试

browser.user_agent_alias = "Windows Mozilla"