Question

As part of a daily process, I insert big chunks of data which I know for a fact do not invalidate any foreign key constraints. To speed things up, several resources suggest to temporarily remove any indexes or constraints and re-apply them after the insert is ready. I'm aware this is risky but would still like to give that a try using SqlAlchemy. I have not added any manual constraints and the only constraints are the ones defined by SqlAlchemy due to foreign and primary keys.

My question is: how can I, using SqlAlchemy API, temporarily remove any constraint/index, perform a set of transactions and then apply the auto-generated constraints again?

I'm hoping there's something a little more convenient than doing that manually.

Thanks!

Answer 1

我不了解SqlAlchemy，但这对SQL来说并不是很难。

对于表my_table，构建一组DROP CONSTRAINT / ADD CONSTRAINT语句：

SELECT
  format('ALTER TABLE %s DROP CONSTRAINT %s', conrelid::regclass, conname),
  format('ALTER TABLE %s ADD CONSTRAINT %s %s', conrelid::regclass, conname, pg_get_constraintdef(oid))
FROM pg_constraint
WHERE conrelid = 'my_table'::regclass

...和索引类似，跳过与约束相关的任何内容：

SELECT
  'DROP INDEX ' || indexrelid::regclass,
  pg_get_indexdef(indexrelid)
FROM pg_index
WHERE indrelid = 'my_table'::regclass AND
  NOT EXISTS (SELECT 1 FROM pg_constraint WHERE conindid = indexrelid)

然后只需运行删除脚本，加载数据，并在完成后运行添加脚本（当然，所有这些都在事务中）。

有一个缺点：当添加外键约束时，所有数据都将被重新检查，如果你对它有信心，这将是很多不必要的工作。全部有效。

有两种方法可以在保留FK约束的同时抑制检查（尽管两者都需要超级用户权限）：

ALTER TABLE ... DISABLE TRIGGER ALL，其中包含用于FK检查的系统触发器。（这是永久性的，当您完成后，您需要重新启用触发器。）
SET session_replication_role = 'replica'基本上宣称你是一个复制过程（假设已经在源头执行了FK检查）。

将这些特权之一传递给非超级用户的最简单方法是给它一个默认的session_replication_role：

ALTER ROLE data_loader SET session_replication_role = 'replica'

任何比这更细粒度的东西可能需要包装在SECURITY DEFINER函数中（使用其创建者的特权执行，而不是其调用者），加载过程可以随意调用。例如。禁用当前事务其余部分的触发器：

CREATE FUNCTION disable_triggers_for_current_transaction() RETURNS void AS $$
  SET LOCAL session_replication_role = 'replica'
$$
LANGUAGE sql
SECURITY DEFINER;
REVOKE ALL ON FUNCTION disable_triggers_for_current_transaction() FROM PUBLIC;
GRANT EXECUTE ON FUNCTION disable_triggers_for_current_transaction() TO data_loader;

或者禁用特定表格上的触发器：

CREATE FUNCTION disable_triggers(TableID REGCLASS) RETURNS void AS $$
BEGIN
  EXECUTE pg_catalog.format('ALTER TABLE %s DISABLE TRIGGER ALL', TableID);
END
$$
LANGUAGE plpgsql
SECURITY DEFINER;
REVOKE ALL ON FUNCTION disable_triggers(REGCLASS) FROM PUBLIC;
GRANT EXECUTE ON FUNCTION disable_triggers(REGCLASS) TO data_loader;

当然，如果您决定使用这些选项中的任何一个，那么在构建那些DROP脚本时，您将希望保持FK约束;只需将AND contype <> 'f'添加到第一个WHERE子句。

SqlAlchemy: Temporarily disable database constraints for big insert transactions

1 个答案: