ActiveRecord :: ConnectionTimeoutError:无法在5.000秒内获取数据库连接(等待5.000秒)

时间:2015-01-06 14:53:14

标签: mysql ruby-on-rails nginx puma airbrake

我有一个生产中的rails应用程序,我前几天部署了一些更改。突然之间,我每天多次收到错误ActiveRecord::ConnectionTimeoutError: could not obtain a database connection within 5.000 seconds (waited 5.000 seconds),并且必须重新启动puma才能解决问题。

我完全不知道造成这种情况的原因。我没有更改服务器上的任何内容,我所做的更改非常简单(添加到视图并添加到控制器方法)。

我在日志文件中看不到任何内容。

我使用的是rails 4.1.4和ruby 2.0.0p481

关于为什么我的关系填满的任何想法?我的连接池设置为10,我使用默认的puma配置。

这是一个堆栈跟踪:

ActiveRecord::ConnectionTimeoutError (could not obtain a database connection within 5.000 seconds (waited 5.000 seconds)):
  activerecord (4.1.4) lib/active_record/connection_adapters/abstract/connection_pool.rb:190:in `block in wait_poll'
  activerecord (4.1.4) lib/active_record/connection_adapters/abstract/connection_pool.rb:181:in `loop'
  activerecord (4.1.4) lib/active_record/connection_adapters/abstract/connection_pool.rb:181:in `wait_poll'
  activerecord (4.1.4) lib/active_record/connection_adapters/abstract/connection_pool.rb:136:in `block in poll'
  /usr/local/rvm/rubies/ruby-2.0.0-p481/lib/ruby/2.0.0/monitor.rb:211:in `mon_synchronize'
  activerecord (4.1.4) lib/active_record/connection_adapters/abstract/connection_pool.rb:146:in `synchronize'
  activerecord (4.1.4) lib/active_record/connection_adapters/abstract/connection_pool.rb:134:in `poll'
  activerecord (4.1.4) lib/active_record/connection_adapters/abstract/connection_pool.rb:418:in `acquire_connection'
  activerecord (4.1.4) lib/active_record/connection_adapters/abstract/connection_pool.rb:351:in `block in checkout'
  /usr/local/rvm/rubies/ruby-2.0.0-p481/lib/ruby/2.0.0/monitor.rb:211:in `mon_synchronize'
  activerecord (4.1.4) lib/active_record/connection_adapters/abstract/connection_pool.rb:350:in `checkout'
  activerecord (4.1.4) lib/active_record/connection_adapters/abstract/connection_pool.rb:265:in `block in connection'
  /usr/local/rvm/rubies/ruby-2.0.0-p481/lib/ruby/2.0.0/monitor.rb:211:in `mon_synchronize'
  activerecord (4.1.4) lib/active_record/connection_adapters/abstract/connection_pool.rb:264:in `connection'
  activerecord (4.1.4) lib/active_record/connection_adapters/abstract/connection_pool.rb:541:in `retrieve_connection'
  activerecord (4.1.4) lib/active_record/connection_handling.rb:113:in `retrieve_connection'
  activerecord (4.1.4) lib/active_record/connection_handling.rb:87:in `connection'
  activerecord (4.1.4) lib/active_record/query_cache.rb:51:in `restore_query_cache_settings'
  activerecord (4.1.4) lib/active_record/query_cache.rb:43:in `rescue in call'
  activerecord (4.1.4) lib/active_record/query_cache.rb:32:in `call'
  activerecord (4.1.4) lib/active_record/connection_adapters/abstract/connection_pool.rb:621:in `call'
  actionpack (4.1.4) lib/action_dispatch/middleware/callbacks.rb:29:in `block in call'
  activesupport (4.1.4) lib/active_support/callbacks.rb:82:in `run_callbacks'
  actionpack (4.1.4) lib/action_dispatch/middleware/callbacks.rb:27:in `call'
  actionpack (4.1.4) lib/action_dispatch/middleware/remote_ip.rb:76:in `call'
  airbrake (4.1.0) lib/airbrake/rails/middleware.rb:13:in `call'
  actionpack (4.1.4) lib/action_dispatch/middleware/debug_exceptions.rb:17:in `call'
  actionpack (4.1.4) lib/action_dispatch/middleware/show_exceptions.rb:30:in `call'
  railties (4.1.4) lib/rails/rack/logger.rb:38:in `call_app'
  railties (4.1.4) lib/rails/rack/logger.rb:20:in `block in call'
  activesupport (4.1.4) lib/active_support/tagged_logging.rb:68:in `block in tagged'
  activesupport (4.1.4) lib/active_support/tagged_logging.rb:26:in `tagged'
  activesupport (4.1.4) lib/active_support/tagged_logging.rb:68:in `tagged'
  railties (4.1.4) lib/rails/rack/logger.rb:20:in `call'
  actionpack (4.1.4) lib/action_dispatch/middleware/request_id.rb:21:in `call'
  rack (1.5.2) lib/rack/methodoverride.rb:21:in `call'
  dragonfly (1.0.5) lib/dragonfly/cookie_monster.rb:9:in `call'
  rack (1.5.2) lib/rack/runtime.rb:17:in `call'
  activesupport (4.1.4) lib/active_support/cache/strategy/local_cache_middleware.rb:26:in `call'
  rack (1.5.2) lib/rack/sendfile.rb:112:in `call'
  airbrake (4.1.0) lib/airbrake/user_informer.rb:16:in `_call'
  airbrake (4.1.0) lib/airbrake/user_informer.rb:12:in `call'
  railties (4.1.4) lib/rails/engine.rb:514:in `call'
  railties (4.1.4) lib/rails/application.rb:144:in `call'
  railties (4.1.4) lib/rails/railtie.rb:194:in `public_send'
  railties (4.1.4) lib/rails/railtie.rb:194:in `method_missing'
  puma (2.9.0) lib/puma/configuration.rb:71:in `call'
  puma (2.9.0) lib/puma/server.rb:490:in `handle_request'
  puma (2.9.0) lib/puma/server.rb:361:in `process_client'
  puma (2.9.0) lib/puma/server.rb:254:in `block in run'
  puma (2.9.0) lib/puma/thread_pool.rb:92:in `call'
  puma (2.9.0) lib/puma/thread_pool.rb:92:in `block in spawn_thread'

Puma init.d脚本

#!/bin/sh
# Starts and stops puma
#


case "$1" in
        start)
                su myuser -c  "source /etc/profile && cd /var/www/myapp/current && rvm gemset use myapp && puma -d -e production -b unix:///var/www/myapp/myapp_app.sock -S /var/www/myapp/myapp_app.state"
        ;;

        stop)
                su myuser -c "source /etc/profile && cd /var/www/myapp/current &&  rvm gemset use myapp && RAILS_ENV=production bundle exec pumactl -S /var/www/myapp/myapp_app.state stop"
        ;;

        restart)
                $0 stop
                $0 start
        ;;

        *)
                echo "Usage: $0 {start|stop|restart}"
                exit 1
esac

修改

我认为我最终使用airbrake方法current_user或{{1}将问题缩小为 devise gem和 } user_signed_in?中的application_controller.rb

这是我的应用程序控制器:

before_action

如果我删除class ApplicationController < ActionController::Base protect_from_forgery before_filter :authenticate_user!, :get_new_messages # Gets the unread messages for the logged in user def get_new_messages @num_new_messages = 0 # Initially set to 0 so login page, etc works # If the user is signed in, fetch the new messages if user_signed_in? # I also tried !current_user.nil? @num_new_messages = Message.where(:created_for => current_user.id).where(:viewed => false).count end end ... end 块,我没有问题。自从我介绍了该代码以来,我的应用程序似乎耗尽了连接。如果我保留if块并删除airbrake gem,我的应用程序似乎运行得很好,只有我的if文件池中默认的5个连接设置。

修改

我终于弄明白,如果我在database.yml文件config/environments/production.rb中注释掉这一行,我就不会收到错误消息。似乎应用程序控制器before_action中的自定义路由+设计是原因。我在github上创建了一个问题和一个可重现的项目。

https://github.com/plataformatec/devise/issues/3422 https://github.com/toymachiner62/devise-connection-failure/blob/master/config/environments/production.rb#L84

3 个答案:

答案 0 :(得分:6)

我遇到了同样的问题,这些问题是由于数据库的打开连接太多造成的。当您在控制器之外(模型,邮件程序,pdf生成器......)中有数据库查询时,就会发生这种情况。

我可以通过在这个自动关闭连接的块中包装这些查询来修复它。

ActiveRecord::Base.connection_pool.with_connection do
  # your code
end

由于Puma工作多线程,因此池大小(如提到的eabraham)也可能是一个限制。尝试增加它(一点点)......

我希望这有帮助!

答案 1 :(得分:2)

在设计家伙的帮助下,我想我终于弄明白了这个问题。似乎通过使用自己的控制器自定义错误页面,我没有跳过before_action get_new_messages。所以非常简单的解决方法是添加:

skip_before_filter :get_new_messages

到我的自定义错误控制器。

此问题详细说明了这背后的原因:https://github.com/plataformatec/devise/issues/3422

答案 2 :(得分:0)

最终这个问题仍然困扰着我一年左右。我终于通过与美洲狮人一起工作得到了很好的解决方案。

将你的美洲狮升级到至少2.15.x