第二次尝试

Question

简要说明：

在Heroku上的Rails 4.0应用
通过延迟工作的后台工作
工人dyno通过Workless
大多数作业成功，一些失败（带有以下异常跟踪）
仅在生产中作业失败（我无法在本地重现此问题）
在被延迟工作接收后约6秒钟内就业失败（见迹线的前三行）
在工作人员/应用程序初始化期间作业失败（我的代码甚至没有机会执行）
我认为错误在于Devise或其中一个模块，在后台工作的背景下

Gemfile（一些相关的行）：

gem 'devise', '~> 3.2.4'
gem 'delayed_job_active_record', '4.0.1'
gem 'workless', '~> 1.2.3'
gem 'daemons', '~> 1.1.9'

异常追踪：

16.566936 heroku[worker.2]: Starting process with command `bundle exec rake jobs:work`
17.201669 heroku[worker.2]: State changed from starting to up
23.375876 heroku[worker.2]: State changed from up to down
25.734990 heroku[worker.2]: Stopping all processes with SIGTERM
26.315290 app[worker.2]: rake aborted!
26.315342 app[worker.2]: SignalException: SIGTERM
26.333413 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/activesupport-4.0.5/lib/active_support/dependencies.rb:229:in `require'
26.333417 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/activesupport-4.0.5/lib/active_support/dependencies.rb:229:in `block in require'
26.333419 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/activesupport-4.0.5/lib/active_support/dependencies.rb:214:in `load_dependency'
26.333421 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/activesupport-4.0.5/lib/active_support/dependencies.rb:229:in `require'
26.333422 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/bcrypt-3.1.7/lib/bcrypt.rb:16:in `rescue in <top (required)>'
26.333424 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/bcrypt-3.1.7/lib/bcrypt.rb:12:in `<top (required)>'
26.333440 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/activesupport-4.0.5/lib/active_support/dependencies.rb:229:in `require'
26.333442 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/activesupport-4.0.5/lib/active_support/dependencies.rb:229:in `block in require'
26.333444 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/activesupport-4.0.5/lib/active_support/dependencies.rb:214:in `load_dependency'
26.333445 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/activesupport-4.0.5/lib/active_support/dependencies.rb:229:in `require'
26.333447 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/devise-3.2.4/lib/devise/models/database_authenticatable.rb:2:in `<top (required)>'
26.333448 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/devise-3.2.4/lib/devise/models.rb:88:in `const_get'
26.333450 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/devise-3.2.4/lib/devise/models.rb:88:in `block (2 levels) in devise'
26.333451 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/devise-3.2.4/lib/devise/models.rb:87:in `each'
26.333453 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/devise-3.2.4/lib/devise/models.rb:87:in `block in devise'
26.333454 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/devise-3.2.4/lib/devise/models.rb:114:in `devise_modules_hook!'
26.333456 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/devise-3.2.4/lib/devise/models.rb:84:in `devise'
26.333458 app[worker.2]: /app/app/models/user.rb:6:in `<class:User>'
26.333459 app[worker.2]: /app/app/models/user.rb:1:in `<top (required)>'
26.333461 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/activesupport-4.0.5/lib/active_support/dependencies.rb:229:in `require'
26.333462 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/activesupport-4.0.5/lib/active_support/dependencies.rb:229:in `block in require'
26.333464 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/activesupport-4.0.5/lib/active_support/dependencies.rb:214:in `load_dependency'
26.333465 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/activesupport-4.0.5/lib/active_support/dependencies.rb:229:in `require'
26.333466 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/activesupport-4.0.5/lib/active_support/dependencies.rb:330:in `require_or_load'
26.333468 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/activesupport-4.0.5/lib/active_support/dependencies.rb:463:in `load_missing_constant'
26.333469 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/activesupport-4.0.5/lib/active_support/dependencies.rb:184:in `const_missing'
26.333470 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/activesupport-4.0.5/lib/active_support/inflector/methods.rb:226:in `const_get'
26.333472 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/activesupport-4.0.5/lib/active_support/inflector/methods.rb:226:in `block in constantize'
26.333473 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/activesupport-4.0.5/lib/active_support/inflector/methods.rb:224:in `each'
26.333474 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/activesupport-4.0.5/lib/active_support/inflector/methods.rb:224:in `inject'
26.333476 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/activesupport-4.0.5/lib/active_support/inflector/methods.rb:224:in `constantize'
26.333477 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/activesupport-4.0.5/lib/active_support/dependencies.rb:535:in `get'
26.333478 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/activesupport-4.0.5/lib/active_support/dependencies.rb:566:in `constantize'
26.333480 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/devise-3.2.4/lib/devise.rb:297:in `get'
26.333481 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/devise-3.2.4/lib/devise/mapping.rb:77:in `to'
26.333483 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/devise-3.2.4/lib/devise/mapping.rb:72:in `modules'
26.333484 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/devise-3.2.4/lib/devise/mapping.rb:89:in `routes'
26.333508 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/devise-3.2.4/lib/devise/mapping.rb:156:in `default_used_route'
26.333509 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/devise-3.2.4/lib/devise/mapping.rb:66:in `initialize'
26.333510 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/devise-3.2.4/lib/devise.rb:331:in `new'
26.333512 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/devise-3.2.4/lib/devise.rb:331:in `add_mapping'
26.333519 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/devise-3.2.4/lib/devise/rails/routes.rb:221:in `block in devise_for'
26.333520 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/devise-3.2.4/lib/devise/rails/routes.rb:220:in `each'
26.333522 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/devise-3.2.4/lib/devise/rails/routes.rb:220:in `devise_for'
26.333523 app[worker.2]: /app/config/routes.rb:31:in `block in <top (required)>'
26.333524 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/actionpack-4.0.5/lib/action_dispatch/routing/route_set.rb:335:in `instance_exec' 
26.333526 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/actionpack-4.0.5/lib/action_dispatch/routing/route_set.rb:335:in `eval_block'
26.333527 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/actionpack-4.0.5/lib/action_dispatch/routing/route_set.rb:313:in `draw'
26.333528 app[worker.2]: /app/config/routes.rb:1:in `<top (required)>'
26.333530 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/activesupport-4.0.5/lib/active_support/dependencies.rb:223:in `load'
26.333531 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/activesupport-4.0.5/lib/active_support/dependencies.rb:223:in `block in load'
26.333542 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/activesupport-4.0.5/lib/active_support/dependencies.rb:214:in `load_dependency'
26.333543 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/activesupport-4.0.5/lib/active_support/dependencies.rb:223:in `load'
26.333545 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/railties-4.0.5/lib/rails/application/routes_reloader.rb:40:in `block in load_paths'
26.333546 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/railties-4.0.5/lib/rails/application/routes_reloader.rb:40:in `each'
26.333548 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/railties-4.0.5/lib/rails/application/routes_reloader.rb:40:in `load_paths'
26.333549 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/railties-4.0.5/lib/rails/application/routes_reloader.rb:16:in `reload!'
26.333551 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/railties-4.0.5/lib/rails/application/routes_reloader.rb:26:in `block in updater'
26.333552 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/activesupport-4.0.5/lib/active_support/file_update_checker.rb:75:in `call'
26.333553 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/activesupport-4.0.5/lib/active_support/file_update_checker.rb:75:in `execute'
26.333555 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/railties-4.0.5/lib/rails/application/routes_reloader.rb:27:in `updater'
26.333556 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/railties-4.0.5/lib/rails/application/routes_reloader.rb:6:in `execute_if_updated'
26.333557 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/railties-4.0.5/lib/rails/application/finisher.rb:69:in `block in <module:Finisher>'
26.333559 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/railties-4.0.5/lib/rails/initializable.rb:30:in `instance_exec'
26.333560 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/railties-4.0.5/lib/rails/initializable.rb:30:in `run'
26.333562 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/railties-4.0.5/lib/rails/initializable.rb:55:in `block in run_initializers'
26.333563 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/railties-4.0.5/lib/rails/initializable.rb:54:in `run_initializers'
26.333577 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/railties-4.0.5/lib/rails/application.rb:215:in `initialize!'
26.333579 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/railties-4.0.5/lib/rails/railtie/configurable.rb:30:in `method_missing'
26.333580 app[worker.2]: /app/config/environment.rb:9:in `<top (required)>'
26.333581 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/activesupport-4.0.5/lib/active_support/dependencies.rb:229:in `require'
26.333583 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/activesupport-4.0.5/lib/active_support/dependencies.rb:229:in `block in require'
26.333584 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/activesupport-4.0.5/lib/active_support/dependencies.rb:214:in `load_dependency'
26.333585 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/activesupport-4.0.5/lib/active_support/dependencies.rb:229:in `require'
26.333587 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/railties-4.0.5/lib/rails/application.rb:189:in `require_environment!'
26.333588 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/railties-4.0.5/lib/rails/application.rb:250:in `block in run_tasks_blocks'
26.333589 app[worker.2]: Tasks: TOP => jobs:work => jobs:environment_options => environment
26.333591 app[worker.2]: (See full trace by running task with --trace)
28.113879 heroku[worker.2]: Process exited with status 1

应用/模型/ user.rb：6

devise :database_authenticatable, :registerable, :confirmable, :omniauthable, :recoverable, :rememberable, :trackable, :validatable

配置/ routes.rb中：31

devise_for :users, controllers: {omniauth_callbacks: 'omniauth_callbacks'}

第二次尝试

我在omniauthable注释了user.rb：

devise :database_authenticatable, :registerable, :confirmable, #:omniauthable,
       :recoverable, :rememberable, :trackable, :validatable

{p}和omniauth_callbacks中的routes.rb：

devise_for :users#, controllers: {omniauth_callbacks: 'omniauth_callbacks'}

我跑了20个工作岗位，最后一个人崩溃了。这次异常跟踪略有不同：

异常追踪：

24.233695 app[worker.2]: rake aborted!
24.233841 app[worker.2]: SignalException: SIGTERM
24.234293 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/activesupport-4.0.5/lib/active_support/dependencies.rb:229:in `block in require'
24.234297 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/activesupport-4.0.5/lib/active_support/dependencies.rb:214:in `load_dependency'
24.234300 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/activesupport-4.0.5/lib/active_support/dependencies.rb:229:in `require'
24.234302 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/newrelic_rpm-3.9.0.229/lib/new_relic/agent/transaction_sampler.rb:11:in `<top (required)>'
24.234303 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/activesupport-4.0.5/lib/active_support/dependencies.rb:229:in `require'
24.234305 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/activesupport-4.0.5/lib/active_support/dependencies.rb:229:in `block in require'
24.234307 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/activesupport-4.0.5/lib/active_support/dependencies.rb:214:in `load_dependency'
24.234309 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/activesupport-4.0.5/lib/active_support/dependencies.rb:229:in `require'
24.234311 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems /newrelic_rpm-3.9.0.229/lib/new_relic/agent.rb:94:in `<module:Agent>'
24.234313 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/newrelic_rpm-3.9.0.229/lib/new_relic/agent.rb:69:in `<module:NewRelic>'
24.234315 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/newrelic_rpm-3.9.0.229/lib/new_relic/agent.rb:40:in `<top (required)>'
24.234317 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/activesupport-4.0.5/lib/active_support/dependencies.rb:229:in `require'
24.234319 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/activesupport-4.0.5/lib/active_support/dependencies.rb:229:in `block in require'
24.234321 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/activesupport-4.0.5/lib/active_support/dependencies.rb:214:in `load_dependency'
24.234323 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/activesupport-4.0.5/lib/active_support/dependencies.rb:229:in `require'
24.234325 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/newrelic_rpm-3.9.0.229/lib/new_relic/control.rb:24:in `<top (required)>'
24.234327 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/activesupport-4.0.5/lib/active_support/dependencies.rb:229:in `require'
24.234329 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/activesupport-4.0.5/lib/active_support /dependencies.rb:229:in `block in require'
24.234332 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/activesupport-4.0.5/lib/active_support/dependencies.rb:214:in `load_dependency'
24.234334 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/activesupport-4.0.5/lib/active_support/dependencies.rb:229:in `require'
24.234336 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/newrelic_rpm-3.9.0.229/lib/newrelic_rpm.rb:25:in `<top (required)>'
24.234338 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/bundler-1.6.3/lib/bundler/runtime.rb:76:in `require'
24.234340 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/bundler-1.6.3/lib/bundler/runtime.rb:76:in `block (2 levels) in require'
24.234343 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/bundler-1.6.3/lib/bundler/runtime.rb:72:in `each'
24.234345 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/bundler-1.6.3/lib/bundler/runtime.rb:72:in `block in require'
24.234346 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/bundler-1.6.3/lib/bundler/runtime.rb:61:in `each'
24.234349 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/bundler-1.6.3/lib/bundler/runtime.rb:61:in `require'
24.234351 app[worker.2]: /app/vendor/bundle/ruby/2.0.0/gems/bundler-1.6.3/lib/bundler.rb:132:in `require'
24.234354 app[worker.2]: /app/config/application.rb:12:in `<top (required)>'
24.234356 app[worker.2]: /app/Rakefile:4:in `require'
24.234358 app[worker.2]: /app/Rakefile:4:in `<top (required)>'
24.234365 app[worker.2]: (See full trace by running task with --trace)
23.305036 heroku[worker.2]: Stopping all processes with SIGTERM
25.200583 heroku[worker.2]: Process exited with status 1

Rake文件：4

require File.expand_path('../config/application', __FILE__)

配置/ application.rb中：12

Bundler.require(:default, Rails.env)

Answer 1

如果你没有调用SIGTERM，Heroku每天至少会通过发送一个SIGTERM来重启每个dyno。 Sidekiq会自动处理这个问题，但有时候Heroku会把它关闭得比它预期更早，并导致问题 - 这意味着将暂停从5更改为8秒。延迟作业和Resque要求您在5左右的时间内处理信号。

Resque说明 https://devcenter.heroku.com/articles/queuing-ruby-resque#job-termination

延迟工作 Long running delayed_job jobs stay locked after a restart on Heroku

Answer 2

你的dynos被发送SIGTERM命令。然后，在SIGKILL发送到您的进程之前，您的进程有10秒钟关闭。如果必须用力杀死进程，则抛出R12 error。除此之外，我们（Heroku）不会强制执行任何超时。

如果您对此有任何疑问，请与我们联系。或者，非常欢迎您打开support request以获取有关此内容的更多信息。

来源：Dynos and the Dyno Manager：Graceful shutdown with SIGTERM

为什么我的一些后台工作失败了？

第二次尝试

2 个答案: