Question

我有一个大型转储文件，我正在并行处理并插入到postgres 9.4.5数据库中。有大约10个进程都在启动一个事务，插入~X000对象，然后提交，重复，直到它们的文件块完成。 除非他们从未完成，因为数据库已锁定。

转储包含500万个左右的对象，每个对象代表一个专辑。一个对象有一个标题，一个发布日期，一个艺术家列表，一个轨道名称列表等。我有一个发布表，每个这些（谁的主键来自转储中的对象）然后连接表与他们的拥有release_artist，release_track等内容的主键。

表格如下：

Table: mdc_releases
  Column   |           Type           | Modifiers | Storage  | Stats target | Description 
-----------+--------------------------+-----------+----------+--------------+-------------
 id        | integer                  | not null  | plain    |              | 
 title     | text                     |           | extended |              | 
 released  | timestamp with time zone |           | plain    |              | 
Indexes:
    "mdc_releases_pkey" PRIMARY KEY, btree (id)

Table: mdc_release_artists
   Column   |  Type   |                            Modifiers                             | Storage | Stats target | Description 
------------+---------+------------------------------------------------------------------+---------+--------------+-------------
 id         | integer | not null default nextval('mdc_release_artists_id_seq'::regclass) | plain   |              | 
 release_id | integer |                                                                  | plain   |              | 
 artist_id  | integer |                                                                  | plain   |              | 
Indexes:
    "mdc_release_artists_pkey" PRIMARY KEY, btree (id)

并插入一个对象如下：

insert into release(...) values(...) returning id;  // refer to id below as $ID
insert into release_meta(release_id, ...) values ($ID, ...);
insert into release_artists(release_id, ...) values ($ID, ...), ($ID, ...), ...;
insert into release_tracks(release_id, ...) values ($ID, ...), ($ID, ...), ...;

所以交易看起来像BEGIN，上面的代码片段是5000次，COMMIT。我已经做了一些谷歌搜索，我不知道为什么看起来像独立插入的东西导致死锁。

这是来自pg_stat_activity的select *显示：

|         state_change          | waiting |        state        | backend_xid | backend_xmin |              query              
+-------------------------------+---------+---------------------+-------------+--------------+---------------------------------
| 2016-01-04 18:42:35.542629-08 | f       | active              |             |      2597876 | select * from pg_stat_activity;
| 2016-01-04 07:36:06.730736-08 | f       | idle in transaction |             |              | BEGIN
| 2016-01-04 07:37:36.066837-08 | f       | idle in transaction |             |              | BEGIN
| 2016-01-04 07:37:36.314909-08 | f       | idle in transaction |             |              | BEGIN
| 2016-01-04 07:37:49.491939-08 | f       | idle in transaction |             |              | BEGIN
| 2016-01-04 07:36:04.865133-08 | f       | idle in transaction |             |              | BEGIN
| 2016-01-04 07:38:39.344163-08 | f       | idle in transaction |             |              | BEGIN
| 2016-01-04 07:36:48.400621-08 | f       | idle in transaction |             |              | BEGIN
| 2016-01-04 07:34:37.802813-08 | f       | idle in transaction |             |              | BEGIN
| 2016-01-04 07:37:24.615981-08 | f       | idle in transaction |             |              | BEGIN
| 2016-01-04 07:37:10.887804-08 | f       | idle in transaction |             |              | BEGIN
| 2016-01-04 07:37:44.200148-08 | f       | idle in transaction |             |              | BEGIN

如何在不导致死锁的情况下对postgres执行多个并发插入事务？

0 个答案: