我在Heroku上有一个Rails 3.1应用程序。我看到了很多这些错误:
错误R14(超出内存配额)
通常,前面的日志条目显示:
处理运行mem = 522M(102.1%)
这确实有所不同,但从来没有太多,并且可能在几乎任何URL请求之后发生,因此就我所知,它与特定的控制器操作无关。
这是一个经典的日志条目块:
2012-01-16T02:35:57+00:00 heroku[router]: PUT prizequiz.herokuapp.com/mobile_users/1 dyno=web.1 queue=0 wait=0ms service=55ms status=401 bytes=27
2012-01-16T02:35:58+00:00 heroku[router]: PUT prizequiz.herokuapp.com/mobile_users/1 dyno=web.1 queue=0 wait=0ms service=155ms status=200 bytes=1
2012-01-16T02:36:02+00:00 heroku[router]: PUT prizequiz.herokuapp.com/mobile_users/1 dyno=web.1 queue=0 wait=0ms service=13ms status=401 bytes=27
2012-01-16T02:36:02+00:00 heroku[router]: PUT prizequiz.herokuapp.com/mobile_users/1 dyno=web.1 queue=0 wait=0ms service=147ms status=200 bytes=1
2012-01-16T02:36:09+00:00 heroku[router]: POST prizequiz.herokuapp.com/mobile_users dyno=web.1 queue=0 wait=0ms service=87ms status=201 bytes=624
2012-01-16T02:36:11+00:00 heroku[router]: GET prizequiz.herokuapp.com/quizzes/1/questions dyno=web.1 queue=0 wait=0ms service=5ms status=401 bytes=27
2012-01-16T02:36:11+00:00 heroku[router]: GET prizequiz.herokuapp.com/quizzes/1/questions dyno=web.1 queue=0 wait=0ms service=290ms status=200 bytes=81412
2012-01-16T02:36:15+00:00 heroku[router]: PUT prizequiz.herokuapp.com/mobile_users/1 dyno=web.1 queue=0 wait=0ms service=10ms status=401 bytes=27
2012-01-16T02:36:16+00:00 heroku[router]: PUT prizequiz.herokuapp.com/mobile_users/1 dyno=web.1 queue=0 wait=0ms service=67ms status=200 bytes=1
2012-01-16T02:36:33+00:00 heroku[router]: POST prizequiz.herokuapp.com/quizzes/1/scores dyno=web.1 queue=0 wait=0ms service=10ms status=401 bytes=27
2012-01-16T02:36:33+00:00 heroku[router]: POST prizequiz.herokuapp.com/quizzes/1/scores dyno=web.1 queue=0 wait=0ms service=132ms status=201 bytes=230
2012-01-16T02:36:55+00:00 heroku[web.1]: Process running mem=522M(102.1%)
2012-01-16T02:36:55+00:00 heroku[web.1]: Error R14 (Memory quota exceeded)
2012-01-16T02:37:17+00:00 app[web.1]:
2012-01-16T02:37:17+00:00 app[web.1]:
2012-01-16T02:37:17+00:00 app[web.1]: Started POST "/quizzes/1/scores" for 177.53.80.25 at 2012-01-16 02:37:17 +0000
2012-01-16T02:37:17+00:00 app[web.1]: cache: [POST /quizzes/1/scores] invalidate, pass
2012-01-16T02:37:17+00:00 app[web.1]:
2012-01-16T02:37:17+00:00 app[web.1]:
2012-01-16T02:37:17+00:00 app[web.1]: Started POST "/quizzes/1/scores" for 177.53.80.25 at 2012-01-16 02:37:17 +0000
2012-01-16T02:37:17+00:00 app[web.1]: cache: [POST /quizzes/1/scores] invalidate, pass
2012-01-16T02:37:17+00:00 heroku[router]: POST prizequiz.herokuapp.com/quizzes/1/scores dyno=web.1 queue=0 wait=0ms service=44ms status=201 bytes=230
2012-01-16T02:37:17+00:00 heroku[web.1]: Process running mem=522M(102.1%)
2012-01-16T02:37:17+00:00 heroku[web.1]: Error R14 (Memory quota exceeded)
2012-01-16T02:37:17+00:00 heroku[router]: POST prizequiz.herokuapp.com/quizzes/1/scores dyno=web.1 queue=0 wait=0ms service=16ms status=401 bytes=27
2012-01-16T02:37:20+00:00 app[web.1]:
2012-01-16T02:37:20+00:00 app[web.1]:
2012-01-16T02:37:20+00:00 app[web.1]: Started GET "/quizzes/1/scores/current_game" for 177.53.80.25 at 2012-01-16 02:37:20 +0000
2012-01-16T02:37:20+00:00 app[web.1]: cache: [GET /quizzes/1/scores/current_game] miss
2012-01-16T02:37:20+00:00 app[web.1]:
2012-01-16T02:37:20+00:00 app[web.1]:
2012-01-16T02:37:20+00:00 app[web.1]: Started GET "/quizzes/1/scores/current_game" for 177.53.80.25 at 2012-01-16 02:37:20 +0000
2012-01-16T02:37:20+00:00 heroku[router]: GET prizequiz.herokuapp.com/quizzes/1/scores/current_game dyno=web.1 queue=0 wait=0ms service=8ms status=401 bytes=27
2012-01-16T02:37:20+00:00 app[web.1]: cache: [GET /quizzes/1/scores/current_game] miss
2012-01-16T02:37:20+00:00 heroku[router]: GET prizequiz.herokuapp.com/quizzes/1/scores/current_game dyno=web.1 queue=0 wait=0ms service=6ms status=401 bytes=27
2012-01-16T02:37:20+00:00 app[web.1]:
2012-01-16T02:37:20+00:00 app[web.1]:
2012-01-16T02:37:20+00:00 app[web.1]: Started POST "/quizzes/1/scores" for 177.53.80.25 at 2012-01-16 02:37:20 +0000
2012-01-16T02:37:20+00:00 heroku[router]: POST prizequiz.herokuapp.com/quizzes/1/scores dyno=web.1 queue=0 wait=0ms service=33ms status=401 bytes=27
2012-01-16T02:37:20+00:00 app[web.1]: cache: [POST /quizzes/1/scores] invalidate, pass
2012-01-16T02:37:20+00:00 app[web.1]:
2012-01-16T02:37:20+00:00 app[web.1]:
2012-01-16T02:37:20+00:00 app[web.1]: Started GET "/quizzes/1/scores/current_game" for 177.53.80.25 at 2012-01-16 02:37:20 +0000
2012-01-16T02:37:20+00:00 app[web.1]:
2012-01-16T02:37:20+00:00 app[web.1]:
我安装了New Relic但无法识别任何用途。
如果需要,很乐意提供更多信息。我也对Heroku提出了支持请求,但到目前为止(2天标记为紧急)我没有回应。
添加网络动态不会产生任何影响。
UPDATE ,我添加了oink gem,这是一个示例结果
2012-01-26T08:24:25+00:00 app[web.1]:
2012-01-26T08:24:25+00:00 app[web.1]:
2012-01-26T08:24:25+00:00 app[web.1]: Started PUT "/mobile_users/1" for 110.49.234.219 at 2012-01-26 08:24:25 +0000
2012-01-26T08:24:26+00:00 app[web.1]: Oink Action: mobile_users#update
2012-01-26T08:24:26+00:00 app[web.1]: Memory usage: 286276 | PID: 16
2012-01-26T08:24:26+00:00 app[web.1]: Instantiation Breakdown: Total: 2 | MobileUser: 2
2012-01-26T08:24:26+00:00 app[web.1]: Oink Log Entry Complete
2012-01-26T08:24:26+00:00 app[web.1]: cache: [PUT /mobile_users/1] invalidate, pass
2012-01-26T08:24:26+00:00 heroku[router]: PUT prizequiz.herokuapp.com/mobile_users/1 dyno=web.1 queue=0 wait=0ms service=460ms status=200 bytes=1
2012-01-26T08:24:38+00:00 heroku[web.1]: Process running mem=537M(104.9%)
2012-01-26T08:24:38+00:00 heroku[web.1]: Error R14 (Memory quota exceeded)
2012-01-26T08:24:43+00:00 app[web.1]:
2012-01-26T08:24:43+00:00 app[web.1]:
2012-01-26T08:24:43+00:00 app[web.1]: Started PUT "/mobile_users/1" for 103.1.165.23 at 2012-01-26 08:24:43 +0000
2012-01-26T08:24:43+00:00 heroku[router]: PUT prizequiz.herokuapp.com/mobile_users/1 dyno=web.1 queue=0 wait=0ms service=544ms status=401 bytes=27
2012-01-26T08:24:43+00:00 app[web.1]: Oink Action: mobile_users#update
2012-01-26T08:24:43+00:00 app[web.1]: Memory usage: 288876 | PID: 19
2012-01-26T08:24:43+00:00 app[web.1]: Instantiation Breakdown: Total: 0
2012-01-26T08:24:43+00:00 app[web.1]: Oink Log Entry Complete
2012-01-26T08:24:43+00:00 app[web.1]: cache: [PUT /mobile_users/1] invalidate, pass
2012-01-26T08:24:47+00:00 app[web.1]:
2012-01-26T08:24:47+00:00 app[web.1]:
2012-01-26T08:24:47+00:00 app[web.1]: Started PUT "/mobile_users/1" for 103.1.165.23 at 2012-01-26 08:24:47 +0000
2012-01-26T08:24:48+00:00 app[web.1]: Oink Action: mobile_users#update
2012-01-26T08:24:48+00:00 app[web.1]: Memory usage: 286412 | PID: 16
2012-01-26T08:24:48+00:00 app[web.1]: Instantiation Breakdown: Total: 2 | MobileUser: 2
2012-01-26T08:24:48+00:00 app[web.1]: Oink Log Entry Complete
2012-01-26T08:24:48+00:00 heroku[router]: PUT prizequiz.herokuapp.com/mobile_users/1 dyno=web.1 queue=0 wait=0ms service=432ms status=200 bytes=1
2012-01-26T08:24:48+00:00 app[web.1]: cache: [PUT /mobile_users/1] invalidate, pass
2012-01-26T08:24:59+00:00 heroku[web.1]: Process running mem=537M(104.9%)
2012-01-26T08:24:59+00:00 heroku[web.1]: Error R14 (Memory quota exceeded)
2012-01-26T08:25:20+00:00 heroku[web.1]: Process running mem=537M(104.9%)
2012-01-26T08:25:20+00:00 heroku[web.1]: Error R14 (Memory quota exceeded)
2012-01-26T08:25:41+00:00 heroku[web.1]: Process running mem=537M(104.9%)
2012-01-26T08:25:41+00:00 heroku[web.1]: Error R14 (Memory quota exceeded)
2012-01-26T08:26:32+00:00 heroku[router]: PUT prizequiz.herokuapp.com/mobile_users/1 dyno=web.1 queue=0 wait=0ms service=34ms status=401 bytes=27
2012-01-26T08:27:04+00:00 app[web.1]:
2012-01-26T08:27:04+00:00 app[web.1]:
2012-01-26T08:27:04+00:00 app[web.1]: Started PUT "/mobile_users/1" for 103.1.165.23 at 2012-01-26 08:27:04 +0000
2012-01-26T08:27:04+00:00 app[web.1]: Oink Action: mobile_users#update
2012-01-26T08:27:04+00:00 app[web.1]: Memory usage: 288876 | PID: 19
2012-01-26T08:27:04+00:00 app[web.1]: Instantiation Breakdown: Total: 2 | MobileUser: 2
2012-01-26T08:27:04+00:00 app[web.1]: Oink Log Entry Complete
2012-01-26T08:27:04+00:00 app[web.1]: cache: [PUT /mobile_users/1] invalidate, pass
2012-01-26T08:27:04+00:00 heroku[router]: PUT prizequiz.herokuapp.com/mobile_users/1 dyno=web.1 queue=0 wait=0ms service=337ms status=200 bytes=1
2012-01-26T08:27:05+00:00 heroku[web.1]: Process running mem=537M(104.9%)
2012-01-26T08:27:05+00:00 heroku[web.1]: Error R14 (Memory quota exceeded)
2012-01-26T08:27:26+00:00 heroku[web.1]: Process running mem=537M(104.9%)
2012-01-26T08:27:26+00:00 heroku[web.1]: Error R14 (Memory quota exceeded)
2012-01-26T08:27:48+00:00 heroku[web.1]: Process running mem=537M(104.9%)
2012-01-26T08:27:48+00:00 heroku[web.1]: Error R14 (Memory quota exceeded)
2012-01-26T08:28:08+00:00 heroku[web.1]: Process running mem=537M(105.0%)
2012-01-26T08:28:08+00:00 heroku[web.1]: Error R14 (Memory quota exceeded)
2012-01-26T08:28:29+00:00 heroku[web.1]: Process running mem=537M(105.0%)
2012-01-26T08:28:29+00:00 heroku[web.1]: Error R14 (Memory quota exceeded)
2012-01-26T08:28:51+00:00 heroku[web.1]: Process running mem=537M(105.0%)
2012-01-26T08:28:51+00:00 heroku[web.1]: Error R14 (Memory quota exceeded)
2012-01-26T08:29:08+00:00 app[web.1]:
2012-01-26T08:29:08+00:00 app[web.1]:
2012-01-26T08:29:08+00:00 app[web.1]: Started PUT "/mobile_users/1" for 85.26.234.218 at 2012-01-26 08:29:08 +0000
2012-01-26T08:29:08+00:00 app[web.1]: Oink Action: mobile_users#update
2012-01-26T08:29:08+00:00 app[web.1]: Memory usage: 382404 | PID: 13
2012-01-26T08:29:08+00:00 app[web.1]: Instantiation Breakdown: Total: 0
2012-01-26T08:29:08+00:00 app[web.1]: Oink Log Entry Complete
2012-01-26T08:29:08+00:00 app[web.1]: cache: [PUT /mobile_users/1] invalidate, pass
2012-01-26T08:29:08+00:00 heroku[router]: PUT prizequiz.herokuapp.com/mobile_users/1 dyno=web.1 queue=0 wait=0ms service=86ms status=401 bytes=27
2012-01-26T08:29:09+00:00 app[web.1]:
2012-01-26T08:29:09+00:00 app[web.1]:
2012-01-26T08:29:09+00:00 app[web.1]: Started PUT "/mobile_users/1" for 85.26.234.218 at 2012-01-26 08:29:09 +0000
2012-01-26T08:29:09+00:00 app[web.1]: Oink Action: mobile_users#update
2012-01-26T08:29:09+00:00 app[web.1]: Memory usage: 382404 | PID: 13
2012-01-26T08:29:09+00:00 app[web.1]: Instantiation Breakdown: Total: 2 | MobileUser: 2
2012-01-26T08:29:09+00:00 app[web.1]: Oink Log Entry Complete
2012-01-26T08:29:09+00:00 app[web.1]: cache: [PUT /mobile_users/1] invalidate, pass
2012-01-26T08:29:09+00:00 heroku[router]: PUT prizequiz.herokuapp.com/mobile_users/1 dyno=web.1 queue=0 wait=0ms service=160ms status=200 bytes=1
2012-01-26T08:29:11+00:00 app[web.1]:
2012-01-26T08:29:11+00:00 app[web.1]:
2012-01-26T08:29:11+00:00 app[web.1]: Started PUT "/mobile_users/1" for 85.26.234.218 at 2012-01-26 08:29:11 +0000
2012-01-26T08:29:11+00:00 heroku[router]: PUT prizequiz.herokuapp.com/mobile_users/1 dyno=web.1 queue=0 wait=0ms service=101ms status=401 bytes=27
2012-01-26T08:29:11+00:00 app[web.1]: Oink Action: mobile_users#update
2012-01-26T08:29:11+00:00 app[web.1]: Memory usage: 382404 | PID: 13
2012-01-26T08:29:11+00:00 app[web.1]: Instantiation Breakdown: Total: 0
2012-01-26T08:29:11+00:00 app[web.1]: Oink Log Entry Complete
2012-01-26T08:29:11+00:00 app[web.1]: cache: [PUT /mobile_users/1] invalidate, pass
2012-01-26T08:29:12+00:00 app[web.1]:
2012-01-26T08:29:12+00:00 app[web.1]:
2012-01-26T08:29:12+00:00 app[web.1]: Started PUT "/mobile_users/1" for 85.26.234.218 at 2012-01-26 08:29:12 +0000
2012-01-26T08:29:12+00:00 heroku[web.1]: Process running mem=537M(105.0%)
2012-01-26T08:29:12+00:00 heroku[web.1]: Error R14 (Memory quota exceeded)
2012-01-26T08:29:12+00:00 app[web.1]: Oink Action: mobile_users#update
2012-01-26T08:29:12+00:00 app[web.1]: Memory usage: 288876 | PID: 19
2012-01-26T08:29:12+00:00 app[web.1]: Instantiation Breakdown: Total: 2 | MobileUser: 2
2012-01-26T08:29:12+00:00 app[web.1]: Oink Log Entry Complete
2012-01-26T08:29:12+00:00 app[web.1]: cache: [PUT /mobile_users/1] invalidate, pass
2012-01-26T08:29:12+00:00 heroku[router]: PUT prizequiz.herokuapp.com/mobile_users/1 dyno=web.1 queue=0 wait=0ms service=169ms status=200 bytes=1
2012-01-26T08:29:33+00:00 heroku[web.1]: Process running mem=537M(105.0%)
2012-01-26T08:29:33+00:00 heroku[web.1]: Error R14 (Memory quota exceeded)
我不知道除了看起来我没有使用Heroku认为的那么多内存之外还证明了什么 内存使用情况:288876 vs进程运行mem = 537M(105.0%)
答案 0 :(得分:40)
查找代码'Model.all.each do |something|'
并替换为Model.find_each do |something|
。这将通过将模型的块加载到内存而不是整个模型中来节省内存。
此外,寻找机会使用in_groups_of
或:limit
来减少一次保存在内存中的对象数量。
编辑:for_each - > find_each。
答案 1 :(得分:7)
Heroku在Acceptable Use Policy中指定每个dyno的RAM限制。
您可以使用memprof,特别是Memprof::Middleware
来帮助找出在请求期间分配应用中所有内存的内容。这实际上只适用于非生产环境,因为它会降低您的应用程序速度。
Rails应用程序通常会分配许多ActiveRecord对象,因此您可能会查找正在执行Model.find(:all).select{...}
的位置,或类似的查询从数据库中提取大量记录然后处理它们红宝石。更好的方法是使用where
来限制使用SQL返回Rails应用程序的内容。
答案 2 :(得分:6)
我有这个问题..使用find_in_batches解决。
如果有人仍有此错误,我会将代码放在此处。 由于运行需要很长时间,我找到了一个帮助用户的progress_bar gem。会让它在这里也让我觉得它几乎在所有情况下都是强制性的。
bar = ProgressBar.new( total )
Texto.find_in_batches(:batch_size => 100) do |textos|
textos.each{| texto |
...do_stuff...
}
bar.increment! textos.size
end
答案 3 :(得分:4)
我使用的是Mongoid,在我的情况下,它来自于在非Web请求过程中启用的“身份地图”(因此身份地图永远不会被清除并保持增长)。在该流程中设置Mongoid.identity_map_enabled = false
解决了这个问题。
答案 4 :(得分:4)
此外,WEB_CONCURRENCY heroku配置可能导致此问题。例如,我有一个运行2个1x dynos的应用程序。如果我使用sensible_defaults(另一个heroku配置),它默认WEB_CONCURRENCY为2(基于我的dynos)。这导致我的日志中出现大量R14错误。如果我关闭sensible_defaults,并设置WEB_CONCURRENCY = 1 ......问题就会消失。
如果您没有设置WEB_CONCURRENCY,并且您没有使用sensible_defaults,并且您通过heroku docs设置了服务器(我正在使用Puma)...您可能在puma.rb中有一行看起来像像这样:
workers Integer(ENV['WEB_CONCURRENCY'] || 3)
如果是这种情况,则WEB_CONCURRENCY未设置...因此它将为3,即使用更多内存。
//查看现有的配置设置。
heroku config
//如果启用了sensible_defaults。
heroku config:set sensible_defaults = disabled
//设置WEB_CONCURRENCY
heroku config:设置WEB_CONCURRENCY = 1
//监控R14错误的日志。
答案 5 :(得分:2)
也尝试一下
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>net471</TargetFramework>
<DebugType>portable</DebugType>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="16.4.0" />
<PackageReference Include="NUnit" Version="3.12.0" />
<PackageReference Include="Nunit3TestAdapter" Version="3.16.1">
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
<PrivateAssets>all</PrivateAssets>
</PackageReference>
<Reference Include="System.Data.Linq" />
<PackageReference Include="System.Net.Http" Version="4.3.4" />
<PackageReference Include="Newtonsoft.Json" Version="12.0.3" />
</ItemGroup>
</Project>
我有2个控制台进程已经运行了将近24个小时。我在控制台中需要做一些繁重的事情,谁知道发生了什么。我在控制台中所做的事情是对数据库的大量写入和读取,并持续了10分钟以上。所以这一定是把东西弄糟了。
在终端中运行heroku ps将向您显示所有过程,并且您可能会得到一个“ aha”。
使用
杀死僵尸进程heroku ps
不要忘记“跑步”。部分
使用heroku ps验证它们是否消失,并检查您的日志是否已解决问题。
答案 6 :(得分:1)
我遇到了同样的问题,并在尝试使用oink gem进行调试时将其升级为2x dynos作为停止间隙。在heroku restart
之后,将dynos加倍的配置更改无效
也没有将他们带到0并备份。在添加oink gem之后执行git push
升级工作,并且oink没有显示heroku日志最初显示的相同消耗。
答案 7 :(得分:0)
对我而言,用MiniMagick取代RMagick完成了这项工作。两个宝石都具有相同的调整大小和裁剪功能(我使用载波进行图像上传)
Gemfile更新
删除gem 'rmagick'
并添加gem 'mini_magick'
运行bundle
更新运营商波上传者
替换include CarrierWave::RMagick
include CarrierWave::MiniMagick
部署到Heroku,就是这样!
最后,我在每次请求后都没有R14错误。