Question

我正在使用django迁移postgres数据。我对一个模式执行原始sql查询，并使用django ORM填充目标模式。原始查询如下所示：

query = '''
    SELECT {columns}
    FROM tbl_one one
    INNER JOIN tbl_two two on two.one_id=one.id
    WHERE one.filter = '{filter}'
        AND two._created_at >= '{start_date} 00:00:00'
    ORDER BY two._created_at;
    '''.format(columns=', '.join(columns), filter=filter,
               start_date=start_date)

if connections['old_schema'].connection is None:
    cursor = connections['old_schema'].cursor()

with transaction.atomic(using='old_schema'):
    cursor.execute(query)
    while True:
        rows = cursor.fetchmany(1000)

        if not rows:
            break
        for row in rows:
            yield dict(zip(columns, row)))

但是，在非常长的查询中，原始查询barfs 在之后产生所有行：

Traceback (most recent call last):
  File "~/.virtualenvs/venv/local/lib/python3.4/site-packages/django/db/backends/__init__.py", line 139, in _commit
    return self.connection.commit()
psycopg2.DatabaseError: SSL SYSCALL error: Connection timed out


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "~/.virtualenvs/venv/local/lib/python3.4/site-packages/django/core/management/__init__.py", line 399, in execute_from_command_line
    utility.execute()
  File "~/.virtualenvs/venv/local/lib/python3.4/site-packages/django/core/management/__init__.py", line 392, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "~/.virtualenvs/venv/local/lib/python3.4/site-packages/django/core/management/base.py", line 242, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "~/.virtualenvs/venv/local/lib/python3.4/site-packages/django/core/management/base.py", line 285, in execute
    output = self.handle(*args, **options)
  File "~/app/management/commands/migrate_cass_data.py", line 166, in handle
    for pg in self.get_from_postgres(filter, start_date):
  File "~/app/management/commands/migrate_cass_data.py", line 235, in get_from_postgres
    yield dict(zip(columns, row)))
  File "~/.virtualenvs/venv/local/lib/python3.4/site-packages/django/db/transaction.py", line 316, in __exit__
    connection.commit()
  File "~/.virtualenvs/venv/local/lib/python3.4/site-packages/django/db/backends/__init__.py", line 171, in commit
    self._commit()
  File "~/.virtualenvs/venv/local/lib/python3.4/site-packages/django/db/backends/__init__.py", line 139, in _commit
    return self.connection.commit()
  File "~/.virtualenvs/venv/local/lib/python3.4/site-packages/django/db/utils.py", line 99, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "~/.virtualenvs/venv/local/lib/python3.4/site-packages/django/utils/six.py", line 549, in reraise
    raise value.with_traceback(tb)
  File "~/.virtualenvs/venv/local/lib/python3.4/site-packages/django/db/backends/__init__.py", line 139, in _commit
    return self.connection.commit()
django.db.utils.DatabaseError: SSL SYSCALL error: Connection timed out

同样，当查询结果较小时，它可以正常工作。当结果很大时，查询似乎返回所有结果，但它似乎没有干净地结束。为什么会发生这种情况？我该如何解决？

为什么我的django在获取所有行后postgres游标查询超时？

0 个答案: