多个项目将更新/连接到同一数据库。 “ 项目A ”负责对数据库进行更新,“ 项目B ”是客户端界面,负责显示更新的数据。
计划安排 Project A 更新记录时, Project B 将创建一个后台作业,运行15分钟以捕获所做的更新,每15秒更新一次界面通过ActionCable(webhooks)。
ActiveJob停留在“ 3 remaining records
”上,而没有注意到任何数据库更改。如果直接调用模型(在作业外部),则一切正常。数据库更改已注意到,客户端界面得到了更新,每个人都很高兴。
但是从作业中调用它(在后台运行任务)后,它就会卡住。
class BatchWatchJob < ApplicationJob
queue_as :default
def perform(**args)
if args[:batch_number]
p "=== Begin - BatchWatch for batch_number: #{args[:batch_number]} ==="
begin
# Set 15 minute timeout
# ReImport.watch_batch() not updating
# it's count when DB data changes :/
# BUT if I call the method directly (not from this Job), it works
Timeout.timeout(900) { ReImport.watch_batch(args[:batch_number]) }
rescue Timeout::Error
puts '=== TIMEOUT ERROR ==='
puts 'BatchWatch job failed to finish in the requisite amount of time (15 minutes)'
end
else
p "=== Unable to start BatchWatch, batch_number not specified ==="
end
end
end
class ReImport < ApplicationRecord
[...]
def self.watch_batch(num)
batch = ReImport.where(batch: num)
total = batch.count
remaining = batch.where(completed: false) # NOTE: starts with 3 remaining
remaining_count = remaining.count
p "initial remaining: #{remaining_count}"
while !remaining.empty? do
p 'wait 10 seconds...'
sleep(10) # wait 10 seconds
p '...seconds finished...'
# batch.reload # doesn't work
# remaining.reload # doesn't work
batch = ReImport.where(batch: num)
remaining = batch.where(completed: false) # NOTE: this should now be 2
p "remaining_count: #{remaining_count}"
p "remaining . count: #{remaining.count}"
p "(remaining_count > remaining.count): #{(remaining_count > remaining.count)}"
if remaining_count > remaining.count
p '=== WATCH_BATCH: update found! ==='
# Update count
remaining_count = remaining.count
# Broadcast DataTables to update records
ReImportBatchChannel.broadcast_to("re_import_batch_#{num}", {update: true})
else
p '=== WATCH_BATCH: nothing changed yet ==='
end
end
p '=== WATCH_BATCH COMPLETED (or timeout reached) ==='
end
[...]
end
> BatchWatchJob.perform_later(batch_number: 7)
Enqueued BatchWatchJob (Job ID: 7d1beeaf-c4d8-4489-885c-13b44d1037cf) to Async(default) with arguments: {:batch_number=>7}
=> #<BatchWatchJob:0x000000055c2100 @arguments=[{:batch_number=>7}], @job_id="7d1beeaf-c4d8-4489-885c-13b44d1037cf", @queue_name="default", @priority=nil, @executions=0, @provider_job_id="011e004c-34f5-4c7a-9925-052df1aa1774">
Performing BatchWatchJob (Job ID: 7d1beeaf-c4d8-4489-885c-13b44d1037cf) from Async(default) with arguments: {:batch_number=>7}
"=== Begin - BatchWatch for batch_number: 7 ==="
(2.1ms) SET @@SESSION.sql_mode = CONCAT(CONCAT(@@sql_mode, ',STRICT_ALL_TABLES'), ',NO_AUTO_VALUE_ON_ZERO'), @@SESSION.sql_auto_is_null = 0, @@SESSION.wait_timeout = 2147483
(0.9ms) SELECT COUNT(*) FROM `re_imports` WHERE `re_imports`.`batch` = 7
(0.7ms) SELECT COUNT(*) FROM `re_imports` WHERE `re_imports`.`batch` = 7 AND `re_imports`.`completed` = 0
"initial remaining: 3"
ReImport Exists (0.5ms) SELECT 1 AS one FROM `re_imports` WHERE `re_imports`.`batch` = 7 AND `re_imports`.`completed` = 0 LIMIT 1
"wait 10 seconds..."
"...seconds finished..."
"remaining_count: 3"
CACHE (0.0ms) SELECT COUNT(*) FROM `re_imports` WHERE `re_imports`.`batch` = 7 AND `re_imports`.`completed` = 0 [["batch", 7], ["completed", 0]]
"remaining . count: 3"
CACHE (0.0ms) SELECT COUNT(*) FROM `re_imports` WHERE `re_imports`.`batch` = 7 AND `re_imports`.`completed` = 0 [["batch", 7], ["completed", 0]]
"(remaining_count > remaining.count): false"
CACHE (0.0ms) SELECT COUNT(*) FROM `re_imports` WHERE `re_imports`.`batch` = 7 AND `re_imports`.`completed` = 0 [["batch", 7], ["completed", 0]]
"=== WATCH_BATCH: nothing changed yet ==="
CACHE ReImport Exists (0.0ms) SELECT 1 AS one FROM `re_imports` WHERE `re_imports`.`batch` = 7 AND `re_imports`.`completed` = 0 LIMIT 1 [["batch", 7], ["completed", 0], ["LIMIT", 1]]
"wait 10 seconds..."
"...seconds finished..."
"remaining_count: 3"
CACHE (0.0ms) SELECT COUNT(*) FROM `re_imports` WHERE `re_imports`.`batch` = 7 AND `re_imports`.`completed` = 0 [["batch", 7], ["completed", 0]]
"remaining . count: 3"
CACHE (0.0ms) SELECT COUNT(*) FROM `re_imports` WHERE `re_imports`.`batch` = 7 AND `re_imports`.`completed` = 0 [["batch", 7], ["completed", 0]]
"(remaining_count > remaining.count): false"
CACHE (0.0ms) SELECT COUNT(*) FROM `re_imports` WHERE `re_imports`.`batch` = 7 AND `re_imports`.`completed` = 0 [["batch", 7], ["completed", 0]]
"=== WATCH_BATCH: nothing changed yet ==="
CACHE ReImport Exists (0.0ms) SELECT 1 AS one FROM `re_imports` WHERE `re_imports`.`batch` = 7 AND `re_imports`.`completed` = 0 LIMIT 1 [["batch", 7], ["completed", 0], ["LIMIT", 1]]
"wait 10 seconds..."
[...]
Performed BatchWatchJob (Job ID: 7d1beeaf-c4d8-4489-885c-13b44d1037cf) from Async(default) in 22863.44ms
我不仅注意到CACHE ReImport Exists
,还想弄清楚如何将其关闭...
答案 0 :(得分:0)
我发现问题是 SQL缓存,并通过在 while循环中使用ActiveRecords未缓存的方法ReImport.uncached do
来解决此问题,
class ReImport < ApplicationRecord
[...]
def self.watch_batch(num)
batch = ReImport.where(batch: num)
total = batch.count
remaining = batch.where(completed: false)
remaining_count = remaining.count
p "initial remaining: #{remaining_count}"
while !remaining.empty? do
ReImport.uncached do # <--- UNCACHE SQL QUERIES
p 'wait 10 seconds...'
sleep(10) # wait 10 seconds
p '...seconds finished...'
batch = ReImport.where(batch: num)
remaining = batch.where(completed: false)
p "remaining_count: #{remaining_count}"
p "remaining . count: #{remaining.count}"
p "(remaining_count > remaining.count): #{(remaining_count > remaining.count)}"
if remaining_count > remaining.count
p '=== WATCH_BATCH: update found! ==='
# Update count
remaining_count = remaining.count
# Broadcast DataTables to update records
ReImportBatchChannel.broadcast_to("re_import_batch_#{num}", {update: true})
else
p '=== WATCH_BATCH: nothing changed yet ==='
end
end
end
p '=== WATCH_BATCH COMPLETED (or timeout reached) ==='
end
[...]
end