活跃的工作和Sidekiq并发会带来巨大的内存和性能影响

时间:2017-12-11 13:22:58

标签: ruby-on-rails redis sidekiq rails-activejob

我的sidekiq工作代码。 GetProcessMem gem用于获取进程前后的内存使用情况。它急剧增加。     class TestBatchJob< ApplicationJob       queue_as:test_builder       包括Math :: PeriodQuarters

  def print_usage(description)
    mb = GetProcessMem.new.mb
    logger.info "#{ description } - MEMORY USAGE(MB): #{ mb.round }"
  end

  def perform
    res = Benchmark.bm do |x|
      x.report("Perform task timeing") do
        print_usage("Before")
        names = []
        10_000_000.times do |item|
          names << "Name #{item}"
        end
        print_usage("After")
      end
    end
    logger.info "Benchmark result #{res.first.total.to_s}"
  end
end

运行单个工作人员时

[b890e994-582a-44ab-af45-22e59aee5455] Performing TestBatchJob (Job ID: b890e994-582a-44ab-af45-22e59aee5455) from Sidekiq(test_builder)
[b890e994-582a-44ab-af45-22e59aee5455] Before - MEMORY USAGE(MB): 1220
[b890e994-582a-44ab-af45-22e59aee5455] After - MEMORY USAGE(MB): 1220
[b890e994-582a-44ab-af45-22e59aee5455] Benchmark result 4.529999999999996
[b890e994-582a-44ab-af45-22e59aee5455] Performed TestBatchJob (Job ID: b890e994-582a-44ab-af45-22e59aee5455) from Sidekiq(test_builder) in 4546.21ms

在sidekiq中运行10个并发

[342e9a9f-dc4f-4eec-9fce-92f1f33517db] Before - MEMORY USAGE(MB): 724
[10b11254-6900-4dbd-b1e0-ddab3d384174] Before - MEMORY USAGE(MB): 724
[e6bbcc06-01bf-4d57-8d70-46fd076ec07f] Before - MEMORY USAGE(MB): 724
[09a03dee-28c7-4722-9637-5c70b319c584] Before - MEMORY USAGE(MB): 724
[31a2c2e3-47ca-48f5-95bb-45a55a54bc8f] Before - MEMORY USAGE(MB): 724
[5757965b-adab-4d91-ac90-ed66224c6bcc] Before - MEMORY USAGE(MB): 724
[018fbd1f-2379-4ebf-a7ca-f6f9335d9002] Before - MEMORY USAGE(MB): 724
[81121b45-1cf5-41ff-a0cb-d70eca6546e0] Before - MEMORY USAGE(MB): 724
[dc1c2ae5-abc9-444a-8ed8-ff2036930990] Before - MEMORY USAGE(MB): 732
[10b11254-6900-4dbd-b1e0-ddab3d384174] After - MEMORY USAGE(MB): 4379
[342e9a9f-dc4f-4eec-9fce-92f1f33517db] After - MEMORY USAGE(MB): 4431
[e6bbcc06-01bf-4d57-8d70-46fd076ec07f] After - MEMORY USAGE(MB): 4607
[342e9a9f-dc4f-4eec-9fce-92f1f33517db] Benchmark result 79.66999999999999
[10b11254-6900-4dbd-b1e0-ddab3d384174] Benchmark result 78.84000000000002
[e6bbcc06-01bf-4d57-8d70-46fd076ec07f] Benchmark result 80.36999999999999
[09a03dee-28c7-4722-9637-5c70b319c584] After - MEMORY USAGE(MB): 4763
[342e9a9f-dc4f-4eec-9fce-92f1f33517db] Performed TestBatchJob (Job ID: 342e9a9f-dc4f-4eec-9fce-92f1f33517db) from Sidekiq(test_builder) in 127633.75ms
[31a2c2e3-47ca-48f5-95bb-45a55a54bc8f] After - MEMORY USAGE(MB): 4820
[10b11254-6900-4dbd-b1e0-ddab3d384174] Performed TestBatchJob (Job ID: 10b11254-6900-4dbd-b1e0-ddab3d384174) from Sidekiq(test_builder) in 132140.78ms
[5757965b-adab-4d91-ac90-ed66224c6bcc] After - MEMORY USAGE(MB): 4902
[018fbd1f-2379-4ebf-a7ca-f6f9335d9002] After - MEMORY USAGE(MB): 4896
[e6bbcc06-01bf-4d57-8d70-46fd076ec07f] Performed TestBatchJob (Job ID: e6bbcc06-01bf-4d57-8d70-46fd076ec07f) from Sidekiq(test_builder) in 132739.1ms
[81121b45-1cf5-41ff-a0cb-d70eca6546e0] After - MEMORY USAGE(MB): 4910
[09a03dee-28c7-4722-9637-5c70b319c584] Benchmark result 82.94
[31a2c2e3-47ca-48f5-95bb-45a55a54bc8f] Benchmark result 83.18
[5757965b-adab-4d91-ac90-ed66224c6bcc] Benchmark result 84.99
[018fbd1f-2379-4ebf-a7ca-f6f9335d9002] Benchmark result 85.02999999999999
[09a03dee-28c7-4722-9637-5c70b319c584] Performed TestBatchJob (Job ID: 09a03dee-28c7-4722-9637-5c70b319c584) from Sidekiq(test_builder) in 137796.23ms
[dc1c2ae5-abc9-444a-8ed8-ff2036930990] After - MEMORY USAGE(MB): 4993
[81121b45-1cf5-41ff-a0cb-d70eca6546e0] Benchmark result 85.08
[31a2c2e3-47ca-48f5-95bb-45a55a54bc8f] Performed TestBatchJob (Job ID: 31a2c2e3-47ca-48f5-95bb-45a55a54bc8f) from Sidekiq(test_builder) in 137797.17ms
[5757965b-adab-4d91-ac90-ed66224c6bcc] Performed TestBatchJob (Job ID: 5757965b-adab-4d91-ac90-ed66224c6bcc) from Sidekiq(test_builder) in 137801.78ms
[018fbd1f-2379-4ebf-a7ca-f6f9335d9002] Performed TestBatchJob (Job ID: 018fbd1f-2379-4ebf-a7ca-f6f9335d9002) from Sidekiq(test_builder) in 137796.29ms
[dc1c2ae5-abc9-444a-8ed8-ff2036930990] Benchmark result 85.00999999999999
[81121b45-1cf5-41ff-a0cb-d70eca6546e0] Performed TestBatchJob (Job ID: 81121b45-1cf5-41ff-a0cb-d70eca6546e0) from Sidekiq(test_builder) in 137796.56ms
[dc1c2ae5-abc9-444a-8ed8-ff2036930990] Performed TestBatchJob (Job ID: dc1c2ae5-abc9-444a-8ed8-ff2036930990) from Sidekiq(test_builder) in 137797.2ms

我用redis-server运行它。我该如何调试此问题。

0 个答案:

没有答案