Question

Rack Middleware和Heroku Request Queue偶尔会出现在我的Ruby / Sinatra应用程序中，导致应用程序挂起，直到我手动重启服务器。我已经对Rack :: Timeout，Postgres Timeout进行了调整，并更新了Postgres调用，以便它们大多运行不到100ms。通常，当请求队列增加时，服务器将自动重新启动，然后继续而不会出现问题，但是当中间件激增时，应用程序无法恢复，应用程序将继续挂起请求队列，直到超时为止。是什么导致这种情况发生？

以下是New Relic的屏幕截图，显示了中间件的高峰：

New Relic chart showing time spent in Request Queuing increase to 20s, Middleware increase to 40s, and Ruby increase to 14s, then crashing. After that, time spent in Request Queue is always 20s

以下是我设置中间件和数据库连接的文件：

配置/ puma.rb

threads_count = Integer(5)
threads threads_count, threads_count

preload_app!

rackup      DefaultRackup
port        ENV['PORT']     || 3000
environment ENV['RACK_ENV'] || 'development'

on_worker_boot do
    ActiveSupport.on_load(:active_record) do
        db = URI.parse(ENV['DATABASE_URL'])
        ActiveRecord::Base.establish_connection(
            :adapter  => db.scheme == 'postgres' ? 'postgresql' : db.scheme,
            :host     => db.host,
            :username => db.user,
            :password => db.password,
            :database => db.path[1..-1],
            :port => db.port,
            :encoding => 'utf8',
            :reaping_frequency => 10
            :pool => 5
            :timeout => 11
        )
    end
end

config.ru

require "rack-timeout"
use Rack::Timeout, service_timeout: 16, wait_timeout: 23

require './web'

run Sinatra::Application
use ActiveRecord::ConnectionAdapters::ConnectionManagement

rakefile.rb

require "./web"
require "sinatra/activerecord/rake"

的Gemfile

source 'https://rubygems.org'
ruby '2.3.1'
gem 'rack-timeout'
gem 'rake'
gem 'sinatra'
gem 'puma', "~> 2.16.0"
gem 'pg'
gem "activerecord", "< 5.0.0" # h/t https://github.com/janko-m/sinatra-activerecord/pull/66
gem 'activesupport'
gem 'sinatra-activerecord'
gem 'redis'
gem 'tilt'

应用程序崩溃时的日志文件

app/web.2:  Rack app error: #<Rack::Timeout::RequestTimeoutError: Request waited 783ms, then ran for longer than 15000ms> 
app/web.2: /app/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/connection_adapters/postgresql/database_statements.rb:155:in `async_exec' 
app/web.2:  #<Rack::Timeout::RequestExpiryError: Request older than 30000ms.>

在应用崩溃之前偶尔会出现错误的其他日志文件

app/web.2: /app/vendor/ruby-2.3.1/lib/ruby/2.3.0/monitor.rb:187:in `lock', deadlock; recursive locking

Answer 1

基于此行的超时错误回溯：

/app/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/connection_adapters/postgresql/database_statements.rb:155:in `async_exec'

当请求被强制超时（Rack :: Timeout）时，似乎活动记录正在对您的数据库进行async_exec调用。

根据您发布的其他错误：

/app/vendor/ruby-2.3.1/lib/ruby/2.3.0/monitor.rb:187:in `lock', deadlock; recursive locking

我会说明显的是postgres连接适配器遇到了死锁。

在postgres中是否发生死锁并且冒泡到活动记录或者连接适配器代码中是否发生死锁，这一点并不明显。你有什么慢的查询登录postgres？死锁错误可能只是symptom of using Rack::Timeout (or just the Timeout ruby class) while using threads。可能是由于查询速度慢，线程达到超时，引发了Timeout :: Error，连接管理中的连接/互斥变得不稳定/管理不善。

是什么导致我的Rack中间件飙升？

1 个答案: