我构建的应用程序是一个非常简单的网站监控工具,包含用户,警报和抓取功能。我创建了一个Rake任务,负责按照指定的时间间隔对网站进行爬网,并且工作正常,在使用rake crawl_next
手动运行时将爬网历史保存到数据库。
在整合逻辑以检查爬网是否超过用户指定的限制或存在错误之后&然后通过电子邮件发送给用户,我再也无法将抓取记录保存到数据库中了。我在运行rake任务时得到NoMethodError: undefined method 'clear' for false:FalseClass
,我无法确定其来源。根据控制台输出,我认为它有一些失败的验证,但我无法确定哪些验证可能导致它失败。我希望一个更有经验的Dev能指出我正确的方向。
我已将抓取记录保存到数据库中了解问题...我认为。
pry
检查输入和输入变量,似乎没什么不妥。after_save
操作,以消除可能的警报模型代码错误。.create
方法。它以同样的方式失败。控制台输出:
** Invoke crawl_next (first_time)
** Invoke environment (first_time)
** Execute environment
** Execute crawl_next
rake aborted!
NoMethodError: undefined method `clear' for false:FalseClass
/usr/local/rvm/gems/ruby-2.5.0/gems/activemodel-5.1.5/lib/active_model/validations.rb:334:in `valid?'
/usr/local/rvm/gems/ruby-2.5.0/gems/activerecord-5.1.5/lib/active_record/validations.rb:65:in `valid?'
/usr/local/rvm/gems/ruby-2.5.0/gems/activerecord-5.1.5/lib/active_record/validations.rb:82:in `perform_validations'
/usr/local/rvm/gems/ruby-2.5.0/gems/activerecord-5.1.5/lib/active_record/validations.rb:44:in `save'
/usr/local/rvm/gems/ruby-2.5.0/gems/activerecord-5.1.5/lib/active_record/attribute_methods/dirty.rb:35:in `save'
/usr/local/rvm/gems/ruby-2.5.0/gems/activerecord-5.1.5/lib/active_record/transactions.rb:308:in `block (2 levels) in save'
/usr/local/rvm/gems/ruby-2.5.0/gems/activerecord-5.1.5/lib/active_record/transactions.rb:384:in `block in with_transaction_returning_status'
/usr/local/rvm/gems/ruby-2.5.0/gems/activerecord-5.1.5/lib/active_record/connection_adapters/abstract/database_statements.rb:235:in `block in transaction'
/usr/local/rvm/gems/ruby-2.5.0/gems/activerecord-5.1.5/lib/active_record/connection_adapters/abstract/transaction.rb:194:in `block in within_new_transaction'
/usr/local/rvm/rubies/ruby-2.5.0/lib/ruby/2.5.0/monitor.rb:226:in `mon_synchronize'
/usr/local/rvm/gems/ruby-2.5.0/gems/activerecord-5.1.5/lib/active_record/connection_adapters/abstract/transaction.rb:191:in `within_new_transaction'
/usr/local/rvm/gems/ruby-2.5.0/gems/activerecord-5.1.5/lib/active_record/connection_adapters/abstract/database_statements.rb:235:in `transaction'
/usr/local/rvm/gems/ruby-2.5.0/gems/activerecord-5.1.5/lib/active_record/transactions.rb:210:in `transaction'
/usr/local/rvm/gems/ruby-2.5.0/gems/activerecord-5.1.5/lib/active_record/transactions.rb:381:in `with_transaction_returning_status'
/usr/local/rvm/gems/ruby-2.5.0/gems/activerecord-5.1.5/lib/active_record/transactions.rb:308:in `block in save'
/usr/local/rvm/gems/ruby-2.5.0/gems/activerecord-5.1.5/lib/active_record/transactions.rb:323:in `rollback_active_record_state!'
/usr/local/rvm/gems/ruby-2.5.0/gems/activerecord-5.1.5/lib/active_record/transactions.rb:307:in `save'
/usr/local/rvm/gems/ruby-2.5.0/gems/activerecord-5.1.5/lib/active_record/suppressor.rb:42:in `save'
/home/ubuntu/workspace/web_monitor/lib/tasks/scheduler.rake:19:in `block (2 levels) in <top (required)>'
/usr/local/rvm/gems/ruby-2.5.0/gems/activerecord-5.1.5/lib/active_record/relation/delegation.rb:39:in `each'
/usr/local/rvm/gems/ruby-2.5.0/gems/activerecord-5.1.5/lib/active_record/relation/delegation.rb:39:in `each'
/home/ubuntu/workspace/web_monitor/lib/tasks/scheduler.rake:5:in `block in <top (required)>'
/usr/local/rvm/rubies/ruby-2.5.0/lib/ruby/gems/2.5.0/gems/rake-12.3.0/lib/rake/task.rb:251:in `block in execute'
/usr/local/rvm/rubies/ruby-2.5.0/lib/ruby/gems/2.5.0/gems/rake-12.3.0/lib/rake/task.rb:251:in `each'
/usr/local/rvm/rubies/ruby-2.5.0/lib/ruby/gems/2.5.0/gems/rake-12.3.0/lib/rake/task.rb:251:in `execute'
/usr/local/rvm/rubies/ruby-2.5.0/lib/ruby/gems/2.5.0/gems/rake-12.3.0/lib/rake/task.rb:195:in `block in invoke_with_call_chain'
/usr/local/rvm/rubies/ruby-2.5.0/lib/ruby/2.5.0/monitor.rb:226:in `mon_synchronize'
/usr/local/rvm/rubies/ruby-2.5.0/lib/ruby/gems/2.5.0/gems/rake-12.3.0/lib/rake/task.rb:188:in `invoke_with_call_chain'
/usr/local/rvm/rubies/ruby-2.5.0/lib/ruby/gems/2.5.0/gems/rake-12.3.0/lib/rake/task.rb:181:in `invoke'
/usr/local/rvm/rubies/ruby-2.5.0/lib/ruby/gems/2.5.0/gems/rake-12.3.0/lib/rake/application.rb:160:in `invoke_task'
/usr/local/rvm/rubies/ruby-2.5.0/lib/ruby/gems/2.5.0/gems/rake-12.3.0/lib/rake/application.rb:116:in `block (2 levels) in top_level'
/usr/local/rvm/rubies/ruby-2.5.0/lib/ruby/gems/2.5.0/gems/rake-12.3.0/lib/rake/application.rb:116:in `each'
/usr/local/rvm/rubies/ruby-2.5.0/lib/ruby/gems/2.5.0/gems/rake-12.3.0/lib/rake/application.rb:116:in `block in top_level'
/usr/local/rvm/rubies/ruby-2.5.0/lib/ruby/gems/2.5.0/gems/rake-12.3.0/lib/rake/application.rb:125:in `run_with_threads'
/usr/local/rvm/rubies/ruby-2.5.0/lib/ruby/gems/2.5.0/gems/rake-12.3.0/lib/rake/application.rb:110:in `top_level'
/usr/local/rvm/rubies/ruby-2.5.0/lib/ruby/gems/2.5.0/gems/rake-12.3.0/lib/rake/application.rb:83:in `block in run'
/usr/local/rvm/rubies/ruby-2.5.0/lib/ruby/gems/2.5.0/gems/rake-12.3.0/lib/rake/application.rb:186:in `standard_exception_handling'
/usr/local/rvm/rubies/ruby-2.5.0/lib/ruby/gems/2.5.0/gems/rake-12.3.0/lib/rake/application.rb:80:in `run'
/usr/local/rvm/gems/ruby-2.5.0@global/gems/rake-12.3.0/exe/rake:27:in `<top (required)>'
/usr/local/rvm/rubies/ruby-2.5.0/bin/rake:29:in `load'
/usr/local/rvm/rubies/ruby-2.5.0/bin/rake:29:in `<main>'
Tasks: TOP => crawl_next
LIB /任务/ scheduler.rake:
desc "perform crawls for active alerts if their interval has passed"
task :crawl_next => :environment do
alerts = Alert.where(active: true).includes(:crawls)
alerts.each do |alert|
last = alert.crawls.last
last_crawl_time = last.crawl_time if !last.nil?
if last.nil? ||
(last_crawl_time + alert.crawl_interval_mins*60) < Time.now + 1
crawl_stats = alert.crawl
end
if crawl_stats
crawl = Crawl.new(crawl_stats)
crawl.save
if crawl.exceeds_limits? || crawl.errors
UserMailer.crawl_alert(alert, crawl).deliver_later
end
else
alert.deactivate
end
end
end
crawl.rb模型:
class Crawl < ActiveRecord::Base
belongs_to :alert
after_save :update_alert_last_crawl
def update_alert_last_crawl
alert = self.alert
alert.update(last_crawl: self.crawl_time)
end
def exceeds_limits?
self.resp_time_ms > self.alert.response_time_threshold_ms
end
def errors
self.resp_code != "200"
end
end
alert.rb型号:
class Alert < ActiveRecord::Base
has_many :user_alerts
has_many :users, through: :user_alerts, dependent: :destroy
has_many :crawls, dependent: :destroy
before_save :activate # TODO: don't activate all alerts before save
validate :valid_url? # Using custom method instead of valid url gem
validates :crawl_interval_mins, presence: true, inclusion: {in: [10, 30, 60]}
validates :notify_emails, presence: true, length: {minimum: 6}
validates :name, presence: true, length: {minimum: 2}
def activate
self.active = true
end
def deactivate
self.active = false
UserMailer.alert_deactivated(self).deliver_later
end
# Return a hash corresponding to a Crawl's schema, to be used in creating
# a new crawl record
def crawl
data = {}
resp = nil
time = Benchmark.measure do
begin
resp = HTTParty.get(self.url)
rescue => e
data = crawl_error_info(e)
end
end
{
alert_id: self.id,
crawl_time: Time.now.to_s,
resp_code: data[:resp_code] || resp.code,
resp_time_ms: data[:resp_time_ms] || time.real * 1_000,
resp_status: data[:resp_status] || resp.message,
resp_size_kb: data[:resp_size_kb] || resp.size # TODO: convert to mb
}
end
def valid_url?
if !self.url.match(/^(((http|https):\/\/|)?[a-z0-9]+([\-\.]{1}[a-z0-9]+)*\.[a-z]{2,6}(:[0-9]{1,5})?(\/.*)?)$/i)
errors.add(:url, "not valid")
end
end
def crawl_error_info(e)
if e.class == SocketError
return {
resp_code: 443,
resp_time_ms: 0,
resp_status: "NAME/SVC NOT KNOWN",
resp_size_kb: 0 # TODO: convert to mb
}
end
end
end
答案 0 :(得分:1)
这不好......
def errors
self.resp_code != "200"
end
errors
是由ActiveRecord::Base
提供的方法,您已使用仅返回true
或false
的on方法覆盖了该方法。当Rails尝试使用errors.clear
清除错误时,它不会期望返回布尔值false
而布尔值false
不支持#clear
。
将errors
方法的名称更改为其他内容,例如bad_response_code?
,然后将rake文件中的行更改为...
if crawl.exceeds_limits? || crawl.bad_response_code?
答案 1 :(得分:0)
错误发生在crawl_next任务中,在此行中:
crawl.save
然后就像你提到的那样,在这个方法中的活动模型验证失败了,在errors.clear中:
def valid?(context = nil)
current_context, self.validation_context = validation_context, context
errors.clear
run_validations!
ensure
self.validation_context = current_context
end
在抓取保存之前放置一个pry,以查看正在传递给Crawl.new的数据以清除问题。