该Rails代码应该可以防止服务器在20秒内记录重复的记录:
@transit = Transit.new(tag: params[:tag])
if Transit.where(tag: @transit.tag).where("created_at > ?", 20.seconds.ago).first
logger.warn "Duplicate tag"
else
@transit.save!
end
但是,这不起作用。我可以在生产数据库(托管在Heroku中)中看到两个不同的记录,它们是用相距10秒的相同标签创建的。
日志显示在第二个请求上执行了正确的查询,但没有返回任何结果,并且仍然保存了一条新记录。
为什么会这样?我认为Postgres的默认隔离级别read_committed可以防止这种情况的发生。不返回任何记录的查询应该错过Rails的SQL缓存。日志显示这两个请求是由Heroku上的同一WEB.1 Dyno处理的,而我的Puma.rb设置为4个工作人员和5个线程。
我想念什么?
这是数据库中的两条记录:
=> #<Transit id: 1080116, tag: 33504,
created_at: "2019-01-30 12:36:11",
updated_at: "2019-01-30 12:41:23">
=> #<Transit id: 1080115, tag: 33504,
created_at: "2019-01-30 12:35:56",
updated_at: "2019-01-30 12:35:56">
第一次插入的日志:
30 Jan 2019 07:35:56.203132 <190>1 2019-01-30T12:35:56.050681+00:00 app web.1 - - [1m [36m (0.8ms) [0m [1mBEGIN [0m
30 Jan 2019 07:35:56.203396 <190>1 2019-01-30T12:35:56.055097+00:00 app web.1 - - [1m [35mSQL (1.0ms) [0m INSERT INTO "transits" ("tag", "created_at", "updated_at") VALUES ($1, $2, $3) RETURNING "id"
30 Jan 2019 07:35:56.269133 <190>1 2019-01-30T12:35:56.114572+00:00 app web.1 - - [1m [36m (2.0ms) [0m [1mCOMMIT [0m
在插入重复项之前的查询日志:
30 Jan 2019 07:36:12.160359 <190>1 2019-01-30T12:36:11.863973+00:00 app web.1 - - [1m [35mTransit Load (5.1ms) [0m SELECT "transits".* FROM "transits" WHERE "transits"."tag" = 33504 AND created_at > '2019-01-30 12:35:51.846431' ORDER BY "transits"."id" ASC LIMIT 1
这是postgres事务隔离级别,需要明确的是出现此问题后打开的另一个连接:
SHOW default_transaction_isolation;
default_transaction_isolation
-------------------------------
read committed
(1 row)
答案 0 :(得分:0)
防止Rails中重复的一种方法是进行验证: Correct way of prevent duplicate records in Rails
但是,您的条件更复杂,因为它涉及跨多个行。 我相信您的标准是,如果最近的运输记录创建时间少于20秒,则不允许输入运输记录。是吗?
在这里提到试图强制执行一个约束,该约束涉及从许多行查看数据,这是不希望的: SQL Sub queries in check constraint
触发器可用于在数据库级别强制执行约束。 一个人可能会抓住一个例外。 不确定是否有一个名为HairTrigger的宝石。
带有Postgresql触发器的示例:
bin/rails generate model transit tag:text
rails generate migration add_validation_trigger_for_transit_creation
class AddValidationTriggerForTransitCreation < ActiveRecord::Migration[5.2]
def up
execute <<-CODE
CREATE FUNCTION validate_transit_create_time() returns trigger as $$
DECLARE
age int;
BEGIN
age := (select extract(epoch from current_timestamp - t.created_at)
from transits t
where t.tag = NEW.tag
and t.id in (select id from transits u
where u.id = t.id
and u.tag = t.tag
and u.created_at = (select max(v.created_at) from transits v where v.tag = u.tag)
));
IF (age < 20) THEN
RAISE EXCEPTION 'created_at too early: %', NEW.created_at;
END IF;
RETURN NEW;
END;
$$ language plpgsql;
CREATE TRIGGER validate_transit_create_trigger BEFORE INSERT OR UPDATE ON transits
FOR EACH ROW EXECUTE PROCEDURE validate_transit_create_time();
CODE
end
def down
execute <<-CODE
drop function validate_transit_create_time() cascade;
CODE
end
end
user1@debian8 /home/user1/rails/dup_test > ../transit_test.rb ; sleep 20; ../transit_test.rb
dup_test_development=> select * from transits;
id | tag | created_at | updated_at
-----+----------+----------------------------+----------------------------
158 | test_tag | 2019-01-31 18:38:10.115891 | 2019-01-31 18:38:10.115891
159 | test_tag | 2019-01-31 18:38:30.609125 | 2019-01-31 18:38:30.609125
(2 rows)
这是我们查询中提供带有标签的最新公交条目的部分
dup_test_development=> select * from transits t
where t.tag = 'test_tag' and t.id in
(select id from transits u where u.id = t.id and u.tag = t.tag and u.created_at =
(select max(v.created_at) from transits v where v.tag = u.tag));
id | tag | created_at | updated_at
-----+----------+----------------------------+----------------------------
159 | test_tag | 2019-01-31 18:38:30.609125 | 2019-01-31 18:38:30.609125
(1 row)
使用我们的标签进行修改,以给出current_timestamp(现在)与最新的运输条目之间的差异。这种差异是PostgreSQL中的间隔。使用UTC匹配Rails:
dup_test_development=> select current_timestamp at time zone 'utc' - created_at
from transits t where t.tag = 'test_tag' and t.id in
(select id from transits u where u.id = t.id and u.tag = t.tag and u.created_at =
(select max(v.created_at) from transits v where v.tag = u.tag));
?column?
-----------------
00:12:34.146536
(1 row)
添加Extract(epoch)将其转换为秒:
dup_test_development=> select extract(epoch from current_timestamp at time zone 'utc' - created_at)
from transits t where t.tag = 'test_tag' and t.id in
(select id from transits u where u.id = t.id and u.tag = t.tag and u.created_at =
(select max(v.created_at) from transits v where v.tag = u.tag));
date_part
------------
868.783503
(1 row)
我们将秒存储为年龄,如果年龄小于20,则会引发数据库异常
运行2次插入,其第二次延迟小于20:
user1@debian8 /home/user1/rails/dup_test > ../transit_test.rb ; sleep 5; ../transit_test.rb
#<ActiveRecord::StatementInvalid: PG::RaiseException: ERROR: created_at too early: 2019-01-31 18:54:48.95695
: INSERT INTO "transits" ("tag", "created_at", "updated_at") VALUES ($1, $2, $3) RETURNING "id">
"ERROR: created_at too early: 2019-01-31 18:54:48.95695\n"
在铁轨外进行短路测试:
#!/usr/bin/env ruby
require 'active_record'
require 'action_view'
path = "/home/user1/rails/dup_test/app/models"
require "#{path}/application_record.rb"
Dir.glob(path + "/*.rb").sort.each do | file |
require file
end
ActiveRecord::Base.establish_connection(
:adapter => "postgresql",
:database => 'dup_test_development',
encoding: "unicode",
username: "user1",
password: nil
)
class Test
def initialize()
end
def go()
begin
t = Transit.new(tag: 'test_tag')
t.save
rescue ActiveRecord::StatementInvalid => e
p e
p e.cause.message
end
end
end
def main
begin
t = Test.new()
t.go()
rescue Exception => e
puts e.message
end
end
main
已经提到使用Redis之类的东西-可能对性能更好
答案 1 :(得分:0)
我认为这是一个并发问题。
ActiveRecord返回后,Rails事务将异步继续进行。只要提交需要15秒才能应用,就会导致此问题。这很长,不太可能,但是可能。
我无法证明这是事实,但这似乎是唯一的解释。要防止这种情况,将需要一个dB存储过程,或者像@PhilipWright建议的那样,或者像您和@kwerle这样的分布式锁。
答案 2 :(得分:-1)
这就是测试的目的。
class Transit < ActiveRecord::Base
def new_transit(tag: tag)
<your code>
end
end
您测试代码:
test 'it saves once' do
<save it once. check the count, etc>
end
test 'it does not save within 10 seconds' do
<save it once. Set the created at to 10 seconds ago. try to save again. check the count, etc>
end
等
p.s。考虑使用redis或类似的东西。否则,您想要执行诸如表锁之类的操作以确保您不会踩踏自己。而且您可能不想做表锁。