Question

当表delayed_jobs开始在几百个之上增长时，工人的表现开始呈指数级下降。

Answer 1

我一直在努力解决这个问题，所以我将我的发现揭示给未来的新人来到这场噩梦。

DelayedJobs项目中有几个问题涉及此问题：

问题在于DelayedJob在非常工作者运行中使用的查询：

UPDATE `delayed_jobs` SET `locked_at` = '2014-04-17 22:32:20', `locked_by` = 'host:b38f770a-f3f3-4b2a-8c66-7c8eebdb7fea pid:2' WHERE ((run_at <= '2014-04-17 22:32:20' AND (locked_at IS NULL OR locked_at < '2014-04-17 18:32:20') OR locked_by = 'host:b38f770a-f3f3-4b2a-8c66-7c8eebdb7fea pid:2') AND failed_at IS NULL) ORDER BY priority ASC, run_at ASC LIMIT 1

在我的情况下，可能需要将近1秒的时间不到1000个工作岗位......并且随着更多工作待定而呈指数增长。

我找到的唯一解决方案是在this blog中公开的一个，简而言之：由于问题是缺少适当的初始查询索引，解决方案是批量拆分表：

-- stop workers
select max(id) from delayed_jobs; -- -> 10010
create table delayed_jobs_backup like delayed_jobs;
insert into delayed_jobs_backup select * from delayed_jobs where id < 10010;
delete from delayed_jobs where id < 10010;
-- start workers
-- while jobs in delayed_jobs_backup do
  -- wait until the batch have been processed
  insert into delayed_jobs select * from delayed_jobs_backup limit 1000;
  delete from delayed_jobs_backup limit 1000;
-- end

Answer 2

延迟作业未针对大量作业进行优化。长期解决方案是转向类似sidekiq（https://github.com/mperham/sidekiq），但短期内你可以使用下面的策略来清除delayed_job队列：

http://aawaara.com/post/61798959791/break-it-down

Answer 3

对于MySQL 5.6，在failed_at列上添加索引有很大帮助。

Ruby，DelayedJob在有太多Jobs挂起时变得很慢

3 个答案: