Heroku dev上的Postgres进程fork错误

时间:2013-03-28 03:53:17

标签: ruby-on-rails postgresql heroku

我目前在使用Heroku RoR应用程序时出现问题,导致无法与Postgres数据库通信。在尝试加载我的应用程序时,我得到了通用的“出错”问题。

运行heroku logs会返回以下内容:

2013-03-28T03:46:01+00:00 heroku[web.1]: Stopping all processes with SIGTERM
2013-03-28T03:46:01+00:00 heroku[web.1]: State changed from starting to up
2013-03-28T03:46:01+00:00 app[web.1]: [2013-03-28 03:46:01] ERROR SignalException: SIGTERM
2013-03-28T03:46:01+00:00 app[web.1]:   /usr/local/lib/ruby/1.9.1/webrick/server.rb:90:in `select'
2013-03-28T03:46:05+00:00 heroku[web.1]: Error R12 (Exit timeout) -> At least one process failed to exit within 10 seconds of SIGTERM
2013-03-28T03:46:05+00:00 heroku[web.1]: Stopping remaining processes with SIGKILL
2013-03-28T03:46:07+00:00 heroku[web.1]: Process exited with status 137
2013-03-28T03:48:19+00:00 app[web.1]: Started GET "/projects" for 0.0.0.0 at 2013-03-28 03:48:19 +0000
2013-03-28T03:48:19+00:00 app[web.1]: PG::Error (could not fork new process for connection: Cannot allocate memory
2013-03-28T03:48:19+00:00 app[web.1]: 
2013-03-28T03:48:19+00:00 app[web.1]: could not fork new process for connection: Cannot allocate memory
2013-03-28T03:48:19+00:00 app[web.1]: 
2013-03-28T03:48:19+00:00 app[web.1]: ):
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.11/lib/active_record/connection_adapters/postgresql_adapter.rb:1208:in `initialize'
2013-03-28T03:48:19+00:00 app[web.1]: could not fork new process for connection: Cannot allocate memory
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.11/lib/active_record/connection_adapters/postgresql_adapter.rb:1208:in `new'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.11/lib/active_record/connection_adapters/postgresql_adapter.rb:326:in `initialize'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.11/lib/active_record/connection_adapters/postgresql_adapter.rb:28:in `new'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.11/lib/active_record/connection_adapters/postgresql_adapter.rb:1208:in `connect'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.11/lib/active_record/connection_adapters/postgresql_adapter.rb:28:in `postgresql_connection'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.11/lib/active_record/connection_adapters/abstract/connection_pool.rb:315:in `new_connection'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.11/lib/active_record/connection_adapters/abstract/connection_pool.rb:325:in `checkout_new_connection'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.11/lib/active_record/connection_adapters/abstract/connection_pool.rb:242:in `loop'
2013-03-28T03:48:19+00:00 app[web.1]:   /usr/local/lib/ruby/1.9.1/monitor.rb:201:in `mon_synchronize'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.11/lib/active_record/connection_adapters/abstract/connection_pool.rb:239:in `checkout'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.11/lib/active_record/connection_adapters/abstract/connection_pool.rb:102:in `block in connection'
2013-03-28T03:48:19+00:00 app[web.1]:   /usr/local/lib/ruby/1.9.1/monitor.rb:201:in `mon_synchronize'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.11/lib/active_record/connection_adapters/abstract/connection_pool.rb:101:in `connection'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.11/lib/active_record/connection_adapters/abstract/connection_pool.rb:247:in `block (2 levels) in checkout'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.11/lib/active_record/connection_adapters/abstract/connection_pool.rb:242:in `block in checkout'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.11/lib/active_record/connection_adapters/abstract/connection_specification.rb:171:in `retrieve_connection'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.11/lib/active_record/connection_adapters/abstract/connection_specification.rb:145:in `connection'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.11/lib/active_record/connection_adapters/abstract/connection_pool.rb:410:in `retrieve_connection'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.11/lib/active_record/query_cache.rb:67:in `rescue in call'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.11/lib/active_record/query_cache.rb:61:in `call'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/actionpack-3.2.11/lib/action_dispatch/middleware/callbacks.rb:28:in `block in call'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/activesupport-3.2.11/lib/active_support/callbacks.rb:405:in `_run__3713961688065884330__call__3295828753336647570__callbacks'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/activerecord-3.2.11/lib/active_record/connection_adapters/abstract/connection_pool.rb:479:in `call'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/activesupport-3.2.11/lib/active_support/callbacks.rb:405:in `__run_callback'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/activesupport-3.2.11/lib/active_support/callbacks.rb:385:in `_run_call_callbacks'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/activesupport-3.2.11/lib/active_support/callbacks.rb:81:in `run_callbacks'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/actionpack-3.2.11/lib/action_dispatch/middleware/show_exceptions.rb:56:in `call'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/actionpack-3.2.11/lib/action_dispatch/middleware/remote_ip.rb:31:in `call'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/actionpack-3.2.11/lib/action_dispatch/middleware/callbacks.rb:27:in `call'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/railties-3.2.11/lib/rails/rack/logger.rb:32:in `call_app'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/actionpack-3.2.11/lib/action_dispatch/middleware/debug_exceptions.rb:16:in `call'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/railties-3.2.11/lib/rails/rack/logger.rb:16:in `block in call'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/activesupport-3.2.11/lib/active_support/tagged_logging.rb:22:in `tagged'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/railties-3.2.11/lib/rails/rack/logger.rb:16:in `call'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/actionpack-3.2.11/lib/action_dispatch/middleware/request_id.rb:22:in `call'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/rack-1.4.5/lib/rack/methodoverride.rb:21:in `call'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/activesupport-3.2.11/lib/active_support/cache/strategy/local_cache.rb:72:in `call'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/rack-1.4.5/lib/rack/runtime.rb:17:in `call'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/rack-1.4.5/lib/rack/lock.rb:15:in `call'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/actionpack-3.2.11/lib/action_dispatch/middleware/static.rb:62:in `call'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/rack-cache-1.2/lib/rack/cache/context.rb:136:in `forward'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/rack-cache-1.2/lib/rack/cache/context.rb:185:in `lookup'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/rack-cache-1.2/lib/rack/cache/context.rb:66:in `call!'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/rack-cache-1.2/lib/rack/cache/context.rb:245:in `fetch'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/rack-cache-1.2/lib/rack/cache/context.rb:51:in `call'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/railties-3.2.11/lib/rails/application.rb:223:in `call'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/railties-3.2.11/lib/rails/engine.rb:479:in `call'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/rack-1.4.5/lib/rack/content_length.rb:14:in `call'
2013-03-28T03:48:19+00:00 app[web.1]:   /usr/local/lib/ruby/1.9.1/webrick/httpserver.rb:70:in `run'
2013-03-28T03:48:19+00:00 app[web.1]:   /usr/local/lib/ruby/1.9.1/webrick/httpserver.rb:111:in `service'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/railties-3.2.11/lib/rails/rack/log_tailer.rb:17:in `call'
2013-03-28T03:48:19+00:00 app[web.1]:   vendor/bundle/ruby/1.9.1/gems/rack-1.4.5/lib/rack/handler/webrick.rb:59:in `service'
2013-03-28T03:48:19+00:00 app[web.1]:   /usr/local/lib/ruby/1.9.1/webrick/server.rb:183:in `block in start_thread'

关键部分在开头:

2013-03-28T03:48:19+00:00 app[web.1]: PG::Error (could not fork new process for connection: Cannot allocate memory
2013-03-28T03:48:19+00:00 app[web.1]: 
2013-03-28T03:48:19+00:00 app[web.1]: could not fork new process for connection: Cannot allocate memory
2013-03-28T03:48:19+00:00 app[web.1]: 
2013-03-28T03:48:19+00:00 app[web.1]: ):

我发现了this问题,我试图限制退回的项目数量,但这似乎不是问题。我的数据库中只有两个项目,两个项目都具有以下模型:

class Project < ActiveRecord::Base
  attr_accessible :description, :image, :name, :url
end

哪些是文本/字符串字段。

2 个答案:

答案 0 :(得分:3)

我注意到你大约15分钟前发布了这个帖子。我已经开始在heroku的最后半小时看到这个错误(连接到dev / free postgres db)。我想知道他们最后是否有问题?他们的状态页面没有给出任何提示。

答案 1 :(得分:1)

您知道,此错误的关键部分是:

could not fork new process for connection: Cannot allocate memory

这表明主机,容器或ulimit上下文没有更多可用内存。创建新进程需要为堆栈和各种其他基本数据结构分配内存。它的内存不多,因此无法分叉进程表明你完全没有内存。

在普通主机上你会登录(如果可以的话),检查正在运行的进程,杀死有问题的进程,设置问题服务的内存限制等等。这不是Heroku的一个选项,但你可以

  • 减少您正在使用的并发连接数;和
  • 为每个连接减少work_memmaintenance_work_mem

如果超出容器中可用的RAM,这将有所帮助。如果整个主机内存不足,那对你没有任何帮助。

我为那些在升级到更大的Heroku计划后遇到类似问题的客户提供了一些紧急支持。我们从来没有得到关于发生了什么的可靠答案,但客户最终 - 经过几次尝试和几天 - 创建了他们数据库的追随者(副本),然后故障转移到它。这解决了问题,表明问题可能与托管该数据库实例的容器有关。如果您可以成功创建关注者,请尝试此操作。您可能需要减少连接数和work_mem等才能执行此操作。

一般情况下,请考虑PostgreSQL在相对较少的主动工作连接和良好的连接池中表现最佳。对于20或30个池连接,通常可以获得比使用200或300并且没有使用池的连接更好的吞吐量。减少连接数,直到问题消失为止,您可能也会看到性能提升。

(披露:我为2ndQuadrant,另一家PostgreSQL支持和服务公司工作)。