Question

我正在尝试从数据仓库中将消息数据提取到我的EC2实例中，获取1.5亿行。我的磁盘上有90GB可用空间。当我将查询运行时，以下是我得到的错误：

Traceback (most recent call last):
  File "/usr/local/anaconda/lib/python3.5/site-packages/sqlalchemy/engine/result.py", line 964, in fetchall
    l = self.process_rows(self._fetchall_impl())
  File "/usr/local/anaconda/lib/python3.5/site-packages/sqlalchemy/engine/result.py", line 915, in _fetchall_impl
    return self.cursor.fetchall()
pyodbc.Error: ('08S01', '[08S01] [FreeTDS][SQL Server]Read from the server failed (20004) (SQLFetch)')

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "en_str.py", line 45, in <module>
    INNER JOIN notified.Message m ON c.MessageID = m.MessageID where c.CreateDate>'2016-01-01 00:00:00.0';''',db007_cnxn)
  File "/usr/local/anaconda/lib/python3.5/site-packages/pandas/io/sql.py", line 331, in read_sql_query
    parse_dates=parse_dates, chunksize=chunksize)
  File "/usr/local/anaconda/lib/python3.5/site-packages/pandas/io/sql.py", line 1092, in read_query
    data = result.fetchall()
  File "/usr/local/anaconda/lib/python3.5/site-packages/sqlalchemy/engine/result.py", line 970, in fetchall
    self.cursor, self.context)
  File "/usr/local/anaconda/lib/python3.5/site-packages/sqlalchemy/engine/base.py", line 1341, in _handle_dbapi_exception
    exc_info
  File "/usr/local/anaconda/lib/python3.5/site-packages/sqlalchemy/util/compat.py", line 202, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
  File "/usr/local/anaconda/lib/python3.5/site-packages/sqlalchemy/util/compat.py", line 185, in reraise
    raise value.with_traceback(tb)
  File "/usr/local/anaconda/lib/python3.5/site-packages/sqlalchemy/engine/result.py", line 964, in fetchall
    l = self.process_rows(self._fetchall_impl())
  File "/usr/local/anaconda/lib/python3.5/site-packages/sqlalchemy/engine/result.py", line 915, in _fetchall_impl
    return self.cursor.fetchall()
sqlalchemy.exc.DBAPIError: (pyodbc.Error) ('08S01', '[08S01] [FreeTDS][SQL Server]Read from the server failed (20004) (SQLFetch)')

我的磁盘空间已满，tmp文件夹中的odbc.log文件占用了90GB空间。我有另一个装有100TB空间的磁盘但odbc.log只在tmp文件夹中生成。这是什么解决方案？关于如何解决这个问题的任何想法？或者，如何将odbc.log文件位置从100 GB空间磁盘更改为具有100 TB空间的磁盘？

另外，如果我给出s3存储桶位置，而不是给出ec2实例位置，那么是否会避免使用这个odbc.log文件？

任何帮助将不胜感激！

Answer 1

看起来你可能有详细的FreeTDS登录。大多数情况下，配置文件位于config.Routes.MapHttpRoute( name: "DefaultApi", routeTemplate: "api/{controller}/{id}", defaults: new { id = RouteParameter.Optional } );或/etc/freetds.conf。

您需要在FreeTDS日志中查找/etc/freetds/freetds.conf。另外，请检查dump file，因为可以在此处启用日志记录。

您可能会发现指向odbcinst.ini的各种日志记录工具之一，这将是您的罪魁祸首。祝你好运！

PyODBC：/tmp/odbc.log文件超过90GB并停止从数据仓库

1 个答案: