Question

我一直很难调试内存泄漏。我在Sidekiq中使用HTTParty进行分页，内存不断增长和增长。我描述了sidekiq，这是我发现的：

 540 /home/user/.rvm/gems/ruby-2.2.3/gems/aws-sdk-core-2.4.2/lib/seahorse/client/configuration.rb:157:DATA
 540 /home/user/.rvm/gems/ruby-2.2.3/gems/aws-sdk-core-2.4.2/lib/seahorse/client/configuration.rb:157:NODE
 609 /home/user/.rvm/gems/ruby-2.2.3/gems/activesupport-4.1.14/lib/active_support/core_ext/class/attribute.rb:86:DATA
1376 /home/user/.rvm/gems/ruby-2.2.3/gems/activerecord-4.1.14/lib/active_record/connection_adapters/postgresql/database_statements.rb:148:STRING
1712 /home/user/.rvm/gems/ruby-2.2.3/gems/activesupport-4.1.14/lib/active_support/dependencies.rb:247:ARRAY
1713 /home/user/.rvm/rubies/ruby-2.2.3/lib/ruby/2.2.0/set.rb:290:STRING
1964 /home/user/.rvm/gems/ruby-2.2.3/gems/mime-types-3.1/lib/mime/types/_columnar.rb:29:OBJECT
1964 /home/user/.rvm/rubies/ruby-2.2.3/lib/ruby/2.2.0/set.rb:72:OBJECT
2572 /home/user/.rvm/gems/ruby-2.2.3/gems/mime-types-3.1/lib/mime/types/_columnar.rb:24:STRING
2987 /home/user/.rvm/gems/ruby-2.2.3/gems/activesupport-4.1.14/lib/active_support/dependencies.rb:247:NODE
3126 /home/user/.rvm/gems/ruby-2.2.3/gems/mime-types-3.1/lib/mime/types/container.rb:10:OBJECT
3506 /home/user/.rvm/gems/ruby-2.2.3/gems/activesupport-4.1.14/lib/active_support/dependencies.rb:247:DATA
3928 /home/user/.rvm/gems/ruby-2.2.3/gems/mime-types-3.1/lib/mime/type.rb:532:STRING
3928 /home/user/.rvm/gems/ruby-2.2.3/gems/mime-types-3.1/lib/mime/type.rb:543:STRING
5175 /home/user/.rvm/rubies/ruby-2.2.3/lib/ruby/2.2.0/set.rb:81:HASH
5234 /home/user/.rvm/gems/ruby-2.2.3/gems/activesupport-4.1.14/lib/active_support/dependencies.rb:247:STRING
5892 /home/user/.rvm/gems/ruby-2.2.3/gems/mime-types-3.1/lib/mime/type.rb:546:STRING
8120 /home/user/.rvm/gems/ruby-2.2.3/gems/json-1.8.3/lib/json/common.rb:155:ARRAY
87403 /home/user/.rvm/gems/ruby-2.2.3/gems/json-1.8.3/lib/json/common.rb:155:HASH
541817 /home/user/.rvm/gems/ruby-2.2.3/gems/json-1.8.3/lib/json/common.rb:155:STRING

注意json STRING和HASH。随着sidekiq继续经营越来越多的工作，这些底部2不断增长。

代码是这样的：

...
begin
      reactions = HTTParty.get("https://graph.facebook.com/v2.7/#{vid.external_id}/reactions?summary=true&limit=#{PAGE_SIZE}&access_token=#{ENV['at']}")
    rescue Exception => ex


      return
    end

    r_last_id = "vl/programs/#{vid.program_id}/vids/#{vid.id}/reactions/last_id"
    r_last_entry_idx = "vl/programs/#{vid.program_id}/vids/#{vid.id}/reactions/last_entry_idx"
    reactions_purged = []
    abort = false
    total_records = reactions['summary']['total_count'] || 0
    last_total_count = $redis.get(r_last_entry_idx).to_i

    need_to_run = total_records - last_total_count
    need_to_run = 0 if need_to_run < 0

    last_id = nil
    loop do

      break if reactions['data'].nil? || reactions['data'].empty?

      reactions['data'].each do |r|

        last_id = $redis.get(r_last_id)

        abort = true and break if !last_id.nil? && need_to_run <= 0

        need_to_run -= 1
        reactions_purged << r
      end

      GC.start # <----- Even this didnt solve it

      break if abort
      break if reactions['paging']['next'].nil?
      reactions = HTTParty.get(reactions['paging']['next'])

    end
...

即使是我添加的GC.start，如你所见，也没有解决它。是什么导致这种JSON泄漏？这来自HTTParty ......

由于

HTTParty和JSON内存泄漏？

0 个答案: