Question

我正在尝试查询数据库并将其按块加载到一系列数据帧中。这需要3个单独的查询，并且在我对数据帧执行操作并继续查询数据库并加载块时，还需要保持与数据库的连接保持打开状态。

以下是重现该错误的代码：

import sqlalchemy
engine = sqlalchemy.create_engine(self.conn_str)
query = "select something from something where something"
query = "select something-else from something-else where something-else"
with engine.connect() as conn:
    events_generator = pd.read_sql(query=query,conn=conn, chunksize=1000)
    df_generator = pd.read_sql(query= query2, conn=conn, chunksize=1000)
    trial_df = next(events_generator)

这是堆栈跟踪：

Traceback (most recent call last):
  File "/Users/borismetodiev/miniconda3/lib/python3.6/site-packages/sqlalchemy/engine/result.py", line 1239, in fetchmany
    l = self.process_rows(self._fetchmany_impl(size))
  File "/Users/borismetodiev/miniconda3/lib/python3.6/site-packages/sqlalchemy/engine/result.py", line 1155, in _fetchmany_impl
    return self.cursor.fetchmany(size)
pyodbc.Error: ('HY010', '[HY010] [unixODBC][Driver Manager]Function sequence error (0) (SQLFetch)')

上述异常是以下异常的直接原因：

Traceback (most recent call last):
  File "model.py", line 192, in <module>
    value_distribution_collector.populate_events()
  File "/Users/x/src/generator/generator/g/g/framework.py", line 87, in populate_events
    current_patient_event_df = next(patient_events_df_generator)
  File "/Users/borismetodiev/miniconda3/lib/python3.6/site-packages/pandas/io/sql.py", line 1155, in _query_iterator
    data = result.fetchmany(chunksize)
  File "/Users/borismetodiev/miniconda3/lib/python3.6/site-packages/sqlalchemy/engine/result.py", line 1245, in fetchmany
    e, None, None, self.cursor, self.context
  File "/Users/borismetodiev/miniconda3/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1473, in _handle_dbapi_exception
    util.raise_from_cause(sqlalchemy_exception, exc_info)
  File "/Users/borismetodiev/miniconda3/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 398, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
  File "/Users/borismetodiev/miniconda3/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 152, in reraise
    raise value.with_traceback(tb)
  File "/Users/borismetodiev/miniconda3/lib/python3.6/site-packages/sqlalchemy/engine/result.py", line 1239, in fetchmany
    l = self.process_rows(self._fetchmany_impl(size))
  File "/Users/x/miniconda3/lib/python3.6/site-packages/sqlalchemy/engine/result.py", line 1155, in _fetchmany_impl
    return self.cursor.fetchmany(size)
sqlalchemy.exc.DBAPIError: (pyodbc.Error) ('HY010', '[HY010] [unixODBC][Driver Manager]Function sequence error (0) (SQLFetch)')
(Background on this error at: http://sqlalche.me/e/dbapi)

我已经了解到，使用块大小进行查询需要调用以消耗read_sql返回的生成器，该生成器在engine.connect（）对象的范围内，因此我认为一种解决方法是打开多个连接，然后具有多个嵌套作用域，但是必须有更简单的方法来做到这一点。任何想法将不胜感激。

使用熊猫read_sql运行多个查询时函数序列错误

0 个答案: