Heroku错误R14（超出内存配额）：我该如何解决这个问题？

时间：2012-01-16 02:43:38

标签： ruby-on-rails ruby-on-rails-3 memory heroku memory-management

我在Heroku上有一个Rails 3.1应用程序。我看到了很多这些错误：

  

错误R14（超出内存配额）

通常，前面的日志条目显示：

  

处理运行mem = 522M（102.1％）

这确实有所不同，但从来没有太多，并且可能在几乎任何URL请求之后发生，因此就我所知，它与特定的控制器操作无关。

这是一个经典的日志条目块：

2012-01-16T02:35:57+00:00 heroku[router]: PUT prizequiz.herokuapp.com/mobile_users/1 dyno=web.1 queue=0 wait=0ms service=55ms status=401 bytes=27
2012-01-16T02:35:58+00:00 heroku[router]: PUT prizequiz.herokuapp.com/mobile_users/1 dyno=web.1 queue=0 wait=0ms service=155ms status=200 bytes=1
2012-01-16T02:36:02+00:00 heroku[router]: PUT prizequiz.herokuapp.com/mobile_users/1 dyno=web.1 queue=0 wait=0ms service=13ms status=401 bytes=27
2012-01-16T02:36:02+00:00 heroku[router]: PUT prizequiz.herokuapp.com/mobile_users/1 dyno=web.1 queue=0 wait=0ms service=147ms status=200 bytes=1
2012-01-16T02:36:09+00:00 heroku[router]: POST prizequiz.herokuapp.com/mobile_users dyno=web.1 queue=0 wait=0ms service=87ms status=201 bytes=624
2012-01-16T02:36:11+00:00 heroku[router]: GET prizequiz.herokuapp.com/quizzes/1/questions dyno=web.1 queue=0 wait=0ms service=5ms status=401 bytes=27
2012-01-16T02:36:11+00:00 heroku[router]: GET prizequiz.herokuapp.com/quizzes/1/questions dyno=web.1 queue=0 wait=0ms service=290ms status=200 bytes=81412
2012-01-16T02:36:15+00:00 heroku[router]: PUT prizequiz.herokuapp.com/mobile_users/1 dyno=web.1 queue=0 wait=0ms service=10ms status=401 bytes=27
2012-01-16T02:36:16+00:00 heroku[router]: PUT prizequiz.herokuapp.com/mobile_users/1 dyno=web.1 queue=0 wait=0ms service=67ms status=200 bytes=1
2012-01-16T02:36:33+00:00 heroku[router]: POST prizequiz.herokuapp.com/quizzes/1/scores dyno=web.1 queue=0 wait=0ms service=10ms status=401 bytes=27
2012-01-16T02:36:33+00:00 heroku[router]: POST prizequiz.herokuapp.com/quizzes/1/scores dyno=web.1 queue=0 wait=0ms service=132ms status=201 bytes=230
2012-01-16T02:36:55+00:00 heroku[web.1]: Process running mem=522M(102.1%)
2012-01-16T02:36:55+00:00 heroku[web.1]: Error R14 (Memory quota exceeded)
2012-01-16T02:37:17+00:00 app[web.1]: 
2012-01-16T02:37:17+00:00 app[web.1]: 
2012-01-16T02:37:17+00:00 app[web.1]: Started POST "/quizzes/1/scores" for 177.53.80.25 at 2012-01-16 02:37:17 +0000
2012-01-16T02:37:17+00:00 app[web.1]: cache: [POST /quizzes/1/scores] invalidate, pass
2012-01-16T02:37:17+00:00 app[web.1]: 
2012-01-16T02:37:17+00:00 app[web.1]: 
2012-01-16T02:37:17+00:00 app[web.1]: Started POST "/quizzes/1/scores" for 177.53.80.25 at 2012-01-16 02:37:17 +0000
2012-01-16T02:37:17+00:00 app[web.1]: cache: [POST /quizzes/1/scores] invalidate, pass
2012-01-16T02:37:17+00:00 heroku[router]: POST prizequiz.herokuapp.com/quizzes/1/scores dyno=web.1 queue=0 wait=0ms service=44ms status=201 bytes=230
2012-01-16T02:37:17+00:00 heroku[web.1]: Process running mem=522M(102.1%)
2012-01-16T02:37:17+00:00 heroku[web.1]: Error R14 (Memory quota exceeded)
2012-01-16T02:37:17+00:00 heroku[router]: POST prizequiz.herokuapp.com/quizzes/1/scores dyno=web.1 queue=0 wait=0ms service=16ms status=401 bytes=27
2012-01-16T02:37:20+00:00 app[web.1]: 
2012-01-16T02:37:20+00:00 app[web.1]: 
2012-01-16T02:37:20+00:00 app[web.1]: Started GET "/quizzes/1/scores/current_game" for 177.53.80.25 at 2012-01-16 02:37:20 +0000
2012-01-16T02:37:20+00:00 app[web.1]: cache: [GET /quizzes/1/scores/current_game] miss
2012-01-16T02:37:20+00:00 app[web.1]: 
2012-01-16T02:37:20+00:00 app[web.1]: 
2012-01-16T02:37:20+00:00 app[web.1]: Started GET "/quizzes/1/scores/current_game" for 177.53.80.25 at 2012-01-16 02:37:20 +0000
2012-01-16T02:37:20+00:00 heroku[router]: GET prizequiz.herokuapp.com/quizzes/1/scores/current_game dyno=web.1 queue=0 wait=0ms service=8ms status=401 bytes=27
2012-01-16T02:37:20+00:00 app[web.1]: cache: [GET /quizzes/1/scores/current_game] miss
2012-01-16T02:37:20+00:00 heroku[router]: GET prizequiz.herokuapp.com/quizzes/1/scores/current_game dyno=web.1 queue=0 wait=0ms service=6ms status=401 bytes=27
2012-01-16T02:37:20+00:00 app[web.1]: 
2012-01-16T02:37:20+00:00 app[web.1]: 
2012-01-16T02:37:20+00:00 app[web.1]: Started POST "/quizzes/1/scores" for 177.53.80.25 at 2012-01-16 02:37:20 +0000
2012-01-16T02:37:20+00:00 heroku[router]: POST prizequiz.herokuapp.com/quizzes/1/scores dyno=web.1 queue=0 wait=0ms service=33ms status=401 bytes=27
2012-01-16T02:37:20+00:00 app[web.1]: cache: [POST /quizzes/1/scores] invalidate, pass
2012-01-16T02:37:20+00:00 app[web.1]: 
2012-01-16T02:37:20+00:00 app[web.1]: 
2012-01-16T02:37:20+00:00 app[web.1]: Started GET "/quizzes/1/scores/current_game" for 177.53.80.25 at 2012-01-16 02:37:20 +0000
2012-01-16T02:37:20+00:00 app[web.1]: 
2012-01-16T02:37:20+00:00 app[web.1]:

我安装了New Relic但无法识别任何用途。

如果需要，很乐意提供更多信息。我也对Heroku提出了支持请求，但到目前为止（2天标记为紧急）我没有回应。

添加网络动态不会产生任何影响。

UPDATE ，我添加了oink gem，这是一个示例结果

2012-01-26T08:24:25+00:00 app[web.1]: 
2012-01-26T08:24:25+00:00 app[web.1]: 
2012-01-26T08:24:25+00:00 app[web.1]: Started PUT "/mobile_users/1" for 110.49.234.219 at 2012-01-26 08:24:25 +0000
2012-01-26T08:24:26+00:00 app[web.1]: Oink Action: mobile_users#update
2012-01-26T08:24:26+00:00 app[web.1]: Memory usage: 286276 | PID: 16
2012-01-26T08:24:26+00:00 app[web.1]: Instantiation Breakdown: Total: 2 | MobileUser: 2
2012-01-26T08:24:26+00:00 app[web.1]: Oink Log Entry Complete
2012-01-26T08:24:26+00:00 app[web.1]: cache: [PUT /mobile_users/1] invalidate, pass
2012-01-26T08:24:26+00:00 heroku[router]: PUT prizequiz.herokuapp.com/mobile_users/1 dyno=web.1 queue=0 wait=0ms service=460ms status=200 bytes=1
2012-01-26T08:24:38+00:00 heroku[web.1]: Process running mem=537M(104.9%)
2012-01-26T08:24:38+00:00 heroku[web.1]: Error R14 (Memory quota exceeded)
2012-01-26T08:24:43+00:00 app[web.1]: 
2012-01-26T08:24:43+00:00 app[web.1]: 
2012-01-26T08:24:43+00:00 app[web.1]: Started PUT "/mobile_users/1" for 103.1.165.23 at 2012-01-26 08:24:43 +0000
2012-01-26T08:24:43+00:00 heroku[router]: PUT prizequiz.herokuapp.com/mobile_users/1 dyno=web.1 queue=0 wait=0ms service=544ms status=401 bytes=27
2012-01-26T08:24:43+00:00 app[web.1]: Oink Action: mobile_users#update
2012-01-26T08:24:43+00:00 app[web.1]: Memory usage: 288876 | PID: 19
2012-01-26T08:24:43+00:00 app[web.1]: Instantiation Breakdown: Total: 0
2012-01-26T08:24:43+00:00 app[web.1]: Oink Log Entry Complete
2012-01-26T08:24:43+00:00 app[web.1]: cache: [PUT /mobile_users/1] invalidate, pass
2012-01-26T08:24:47+00:00 app[web.1]: 
2012-01-26T08:24:47+00:00 app[web.1]: 
2012-01-26T08:24:47+00:00 app[web.1]: Started PUT "/mobile_users/1" for 103.1.165.23 at 2012-01-26 08:24:47 +0000
2012-01-26T08:24:48+00:00 app[web.1]: Oink Action: mobile_users#update
2012-01-26T08:24:48+00:00 app[web.1]: Memory usage: 286412 | PID: 16
2012-01-26T08:24:48+00:00 app[web.1]: Instantiation Breakdown: Total: 2 | MobileUser: 2
2012-01-26T08:24:48+00:00 app[web.1]: Oink Log Entry Complete
2012-01-26T08:24:48+00:00 heroku[router]: PUT prizequiz.herokuapp.com/mobile_users/1 dyno=web.1 queue=0 wait=0ms service=432ms status=200 bytes=1
2012-01-26T08:24:48+00:00 app[web.1]: cache: [PUT /mobile_users/1] invalidate, pass
2012-01-26T08:24:59+00:00 heroku[web.1]: Process running mem=537M(104.9%)
2012-01-26T08:24:59+00:00 heroku[web.1]: Error R14 (Memory quota exceeded)
2012-01-26T08:25:20+00:00 heroku[web.1]: Process running mem=537M(104.9%)
2012-01-26T08:25:20+00:00 heroku[web.1]: Error R14 (Memory quota exceeded)
2012-01-26T08:25:41+00:00 heroku[web.1]: Process running mem=537M(104.9%)
2012-01-26T08:25:41+00:00 heroku[web.1]: Error R14 (Memory quota exceeded)
2012-01-26T08:26:32+00:00 heroku[router]: PUT prizequiz.herokuapp.com/mobile_users/1 dyno=web.1 queue=0 wait=0ms service=34ms status=401 bytes=27
2012-01-26T08:27:04+00:00 app[web.1]: 
2012-01-26T08:27:04+00:00 app[web.1]: 
2012-01-26T08:27:04+00:00 app[web.1]: Started PUT "/mobile_users/1" for 103.1.165.23 at 2012-01-26 08:27:04 +0000
2012-01-26T08:27:04+00:00 app[web.1]: Oink Action: mobile_users#update
2012-01-26T08:27:04+00:00 app[web.1]: Memory usage: 288876 | PID: 19
2012-01-26T08:27:04+00:00 app[web.1]: Instantiation Breakdown: Total: 2 | MobileUser: 2
2012-01-26T08:27:04+00:00 app[web.1]: Oink Log Entry Complete
2012-01-26T08:27:04+00:00 app[web.1]: cache: [PUT /mobile_users/1] invalidate, pass
2012-01-26T08:27:04+00:00 heroku[router]: PUT prizequiz.herokuapp.com/mobile_users/1 dyno=web.1 queue=0 wait=0ms service=337ms status=200 bytes=1
2012-01-26T08:27:05+00:00 heroku[web.1]: Process running mem=537M(104.9%)
2012-01-26T08:27:05+00:00 heroku[web.1]: Error R14 (Memory quota exceeded)
2012-01-26T08:27:26+00:00 heroku[web.1]: Process running mem=537M(104.9%)
2012-01-26T08:27:26+00:00 heroku[web.1]: Error R14 (Memory quota exceeded)
2012-01-26T08:27:48+00:00 heroku[web.1]: Process running mem=537M(104.9%)
2012-01-26T08:27:48+00:00 heroku[web.1]: Error R14 (Memory quota exceeded)
2012-01-26T08:28:08+00:00 heroku[web.1]: Process running mem=537M(105.0%)
2012-01-26T08:28:08+00:00 heroku[web.1]: Error R14 (Memory quota exceeded)
2012-01-26T08:28:29+00:00 heroku[web.1]: Process running mem=537M(105.0%)
2012-01-26T08:28:29+00:00 heroku[web.1]: Error R14 (Memory quota exceeded)
2012-01-26T08:28:51+00:00 heroku[web.1]: Process running mem=537M(105.0%)
2012-01-26T08:28:51+00:00 heroku[web.1]: Error R14 (Memory quota exceeded)
2012-01-26T08:29:08+00:00 app[web.1]: 
2012-01-26T08:29:08+00:00 app[web.1]: 
2012-01-26T08:29:08+00:00 app[web.1]: Started PUT "/mobile_users/1" for 85.26.234.218 at 2012-01-26 08:29:08 +0000
2012-01-26T08:29:08+00:00 app[web.1]: Oink Action: mobile_users#update
2012-01-26T08:29:08+00:00 app[web.1]: Memory usage: 382404 | PID: 13
2012-01-26T08:29:08+00:00 app[web.1]: Instantiation Breakdown: Total: 0
2012-01-26T08:29:08+00:00 app[web.1]: Oink Log Entry Complete
2012-01-26T08:29:08+00:00 app[web.1]: cache: [PUT /mobile_users/1] invalidate, pass
2012-01-26T08:29:08+00:00 heroku[router]: PUT prizequiz.herokuapp.com/mobile_users/1 dyno=web.1 queue=0 wait=0ms service=86ms status=401 bytes=27
2012-01-26T08:29:09+00:00 app[web.1]: 
2012-01-26T08:29:09+00:00 app[web.1]: 
2012-01-26T08:29:09+00:00 app[web.1]: Started PUT "/mobile_users/1" for 85.26.234.218 at 2012-01-26 08:29:09 +0000
2012-01-26T08:29:09+00:00 app[web.1]: Oink Action: mobile_users#update
2012-01-26T08:29:09+00:00 app[web.1]: Memory usage: 382404 | PID: 13
2012-01-26T08:29:09+00:00 app[web.1]: Instantiation Breakdown: Total: 2 | MobileUser: 2
2012-01-26T08:29:09+00:00 app[web.1]: Oink Log Entry Complete
2012-01-26T08:29:09+00:00 app[web.1]: cache: [PUT /mobile_users/1] invalidate, pass
2012-01-26T08:29:09+00:00 heroku[router]: PUT prizequiz.herokuapp.com/mobile_users/1 dyno=web.1 queue=0 wait=0ms service=160ms status=200 bytes=1
2012-01-26T08:29:11+00:00 app[web.1]: 
2012-01-26T08:29:11+00:00 app[web.1]: 
2012-01-26T08:29:11+00:00 app[web.1]: Started PUT "/mobile_users/1" for 85.26.234.218 at 2012-01-26 08:29:11 +0000
2012-01-26T08:29:11+00:00 heroku[router]: PUT prizequiz.herokuapp.com/mobile_users/1 dyno=web.1 queue=0 wait=0ms service=101ms status=401 bytes=27
2012-01-26T08:29:11+00:00 app[web.1]: Oink Action: mobile_users#update
2012-01-26T08:29:11+00:00 app[web.1]: Memory usage: 382404 | PID: 13
2012-01-26T08:29:11+00:00 app[web.1]: Instantiation Breakdown: Total: 0
2012-01-26T08:29:11+00:00 app[web.1]: Oink Log Entry Complete
2012-01-26T08:29:11+00:00 app[web.1]: cache: [PUT /mobile_users/1] invalidate, pass
2012-01-26T08:29:12+00:00 app[web.1]: 
2012-01-26T08:29:12+00:00 app[web.1]: 
2012-01-26T08:29:12+00:00 app[web.1]: Started PUT "/mobile_users/1" for 85.26.234.218 at 2012-01-26 08:29:12 +0000
2012-01-26T08:29:12+00:00 heroku[web.1]: Process running mem=537M(105.0%)
2012-01-26T08:29:12+00:00 heroku[web.1]: Error R14 (Memory quota exceeded)
2012-01-26T08:29:12+00:00 app[web.1]: Oink Action: mobile_users#update
2012-01-26T08:29:12+00:00 app[web.1]: Memory usage: 288876 | PID: 19
2012-01-26T08:29:12+00:00 app[web.1]: Instantiation Breakdown: Total: 2 | MobileUser: 2
2012-01-26T08:29:12+00:00 app[web.1]: Oink Log Entry Complete
2012-01-26T08:29:12+00:00 app[web.1]: cache: [PUT /mobile_users/1] invalidate, pass
2012-01-26T08:29:12+00:00 heroku[router]: PUT prizequiz.herokuapp.com/mobile_users/1 dyno=web.1 queue=0 wait=0ms service=169ms status=200 bytes=1
2012-01-26T08:29:33+00:00 heroku[web.1]: Process running mem=537M(105.0%)
2012-01-26T08:29:33+00:00 heroku[web.1]: Error R14 (Memory quota exceeded)

我不知道除了看起来我没有使用Heroku认为的那么多内存之外还证明了什么 内存使用情况：288876 vs进程运行mem = 537M（105.0％）

8 个答案:

答案 0 :(得分：40)

查找代码'Model.all.each do |something|'并替换为Model.find_each do |something|。这将通过将模型的块加载到内存而不是整个模型中来节省内存。

此外，寻找机会使用in_groups_of或:limit来减少一次保存在内存中的对象数量。

编辑：for_each - ＆gt; find_each。

答案 1 :(得分：7)

Heroku在Acceptable Use Policy中指定每个dyno的RAM限制。

您可以使用memprof，特别是Memprof::Middleware来帮助找出在请求期间分配应用中所有内存的内容。这实际上只适用于非生产环境，因为它会降低您的应用程序速度。

Rails应用程序通常会分配许多ActiveRecord对象，因此您可能会查找正在执行Model.find(:all).select{...}的位置，或类似的查询从数据库中提取大量记录然后处理它们红宝石。更好的方法是使用where来限制使用SQL返回Rails应用程序的内容。

答案 2 :(得分：6)

我有这个问题..使用find_in_batches解决。

如果有人仍有此错误，我会将代码放在此处。 由于运行需要很长时间，我找到了一个帮助用户的progress_bar gem。会让它在这里也让我觉得它几乎在所有情况下都是强制性的。

bar = ProgressBar.new( total )
Texto.find_in_batches(:batch_size => 100) do |textos|
    textos.each{| texto | 
        ...do_stuff...
    }
    bar.increment! textos.size
end

进度条： https://github.com/paul/progress_bar/issues

答案 3 :(得分：4)

我使用的是Mongoid，在我的情况下，它来自于在非Web请求过程中启用的“身份地图”（因此身份地图永远不会被清除并保持增长）。在该流程中设置Mongoid.identity_map_enabled = false解决了这个问题。

答案 4 :(得分：4)

此外，WEB_CONCURRENCY heroku配置可能导致此问题。例如，我有一个运行2个1x dynos的应用程序。如果我使用sensible_defaults（另一个heroku配置），它默认WEB_CONCURRENCY为2（基于我的dynos）。这导致我的日志中出现大量R14错误。如果我关闭sensible_defaults，并设置WEB_CONCURRENCY = 1 ......问题就会消失。

如果您没有设置WEB_CONCURRENCY，并且您没有使用sensible_defaults，并且您通过heroku docs设置了服务器（我正在使用Puma）...您可能在puma.rb中有一行看起来像像这样：

workers Integer(ENV['WEB_CONCURRENCY'] || 3)

如果是这种情况，则WEB_CONCURRENCY未设置...因此它将为3，即使用更多内存。

在控制台中：

//查看现有的配置设置。

  

heroku config

//如果启用了sensible_defaults。

  

heroku config：set sensible_defaults = disabled

//设置WEB_CONCURRENCY

  

heroku config：设置WEB_CONCURRENCY = 1

//监控R14错误的日志。

答案 5 :(得分：2)

也尝试一下

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net471</TargetFramework>
    <DebugType>portable</DebugType>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="16.4.0" />
    <PackageReference Include="NUnit" Version="3.12.0" />
    <PackageReference Include="Nunit3TestAdapter" Version="3.16.1">
      <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
      <PrivateAssets>all</PrivateAssets>
    </PackageReference>
    <Reference Include="System.Data.Linq" />
    <PackageReference Include="System.Net.Http" Version="4.3.4" />
    <PackageReference Include="Newtonsoft.Json" Version="12.0.3" />
  </ItemGroup>
</Project>

我有2个控制台进程已经运行了将近24个小时。我在控制台中需要做一些繁重的事情，谁知道发生了什么。我在控制台中所做的事情是对数据库的大量写入和读取，并持续了10分钟以上。所以这一定是把东西弄糟了。

在终端中运行heroku ps将向您显示所有过程，并且您可能会得到一个“ aha”。

使用

杀死僵尸进程 
heroku ps

不要忘记“跑步”。部分

使用heroku ps验证它们是否消失，并检查您的日志是否已解决问题。

答案 6 :(得分：1)

我遇到了同样的问题，并在尝试使用oink gem进行调试时将其升级为2x dynos作为停止间隙。在heroku restart之后，将dynos加倍的配置更改无效 也没有将他们带到0并备份。在添加oink gem之后执行git push升级工作，并且oink没有显示heroku日志最初显示的相同消耗。

答案 7 :(得分：0)

对我而言，用MiniMagick取代RMagick完成了这项工作。两个宝石都具有相同的调整大小和裁剪功能（我使用载波进行图像上传）

  1. Gemfile更新

    删除gem 'rmagick'

    并添加gem 'mini_magick'

    运行bundle

  2. 更新运营商波上传者

    替换include CarrierWave::RMagick

    include CarrierWave::MiniMagick

  3. 部署到Heroku，就是这样！

    4. -

    最后，我在每次请求后都没有R14错误。

相关问题
最新问题