用spellfix1表反映sqlite数据库

时间:2016-07-17 20:52:51

标签: python sqlite sqlalchemy

我正在尝试反映基于sqlite的数据库(更具体地说是plex media server database)。

我可以连接到DB:

import sqlalchemy as sa
from sqlalchemy.orm import create_session

from sqlalchemy.ext.automap import automap_base
from sqlalchemy.engine.reflection import Inspector

conn = 'sqlite:////..../com.plexapp.plugins.library.db-2016-07-17'
eng = sa.create_engine(conn)
plex = eng.connect()

我可以查询数据库:

s = create_session(plex)
r = s.execute('SELECT COUNT(*) FROM METADATA_ITEMS')

r.fetchall()

>>> [(55368,)]

我甚至可以使用SQLAlchemy的Inspector检查所有表格

i = Inspector.from_engine(plex)
i.get_sorted_table_and_fkc_names()
>>>> [('metadata_item_views', set()),
 ('cloudsync_files', set()),
 ('library_timeline_entries', set()),
 ('synced_ancestor_items', set()),
 ('schema_migrations', set()),
 ('fts4_tag_titles_stat', set()),
 ('spellfix_tag_titles', set()),
 ('metadata_items', set()),
 ('media_streams', set()),
 ('metadata_relations', set()),
 ('play_queue_generators', set()),
 ...

但是,我似乎无法反映整个数据库。我得到了这个非常丑陋的例外:

m = sa.MetaData(bind=plex)
m.reflect()
>>>>
   ---------------------------------------------------------------------------
OperationalError                          Traceback (most recent call last)
/opt/anaconda3/lib/python3.5/site-packages/sqlalchemy/engine/base.py in _execute_context(self, dialect, constructor, statement, parameters, *args)
   1138                         parameters,
-> 1139                         context)
   1140         except Exception as e:

/opt/anaconda3/lib/python3.5/site-packages/sqlalchemy/engine/default.py in do_execute(self, cursor, statement, parameters, context)
    449     def do_execute(self, cursor, statement, parameters, context=None):
--> 450         cursor.execute(statement, parameters)
    451 

OperationalError: no such module: spellfix1

The above exception was the direct cause of the following exception:

OperationalError                          Traceback (most recent call last)
<ipython-input-58-8cb90bbdb0ad> in <module>()
      1 m = sa.MetaData(bind=plex)
      2 
----> 3 m.reflect()

/opt/anaconda3/lib/python3.5/site-packages/sqlalchemy/sql/schema.py in reflect(self, bind, schema, views, only, extend_existing, autoload_replace, **dialect_kwargs)
   3653 
   3654             for name in load:
-> 3655                 Table(name, self, **reflect_opts)
   3656 
   3657     def append_ddl_listener(self, event_name, listener):

/opt/anaconda3/lib/python3.5/site-packages/sqlalchemy/sql/schema.py in __new__(cls, *args, **kw)
    414             except:
    415                 with util.safe_reraise():
--> 416                     metadata._remove_table(name, schema)
    417 
    418     @property

/opt/anaconda3/lib/python3.5/site-packages/sqlalchemy/util/langhelpers.py in __exit__(self, type_, value, traceback)
     58             exc_type, exc_value, exc_tb = self._exc_info
     59             self._exc_info = None   # remove potential circular references
---> 60             compat.reraise(exc_type, exc_value, exc_tb)
     61         else:
     62             if not compat.py3k and self._exc_info and self._exc_info[1]:

/opt/anaconda3/lib/python3.5/site-packages/sqlalchemy/util/compat.py in reraise(tp, value, tb, cause)
    184         if value.__traceback__ is not tb:
    185             raise value.with_traceback(tb)
--> 186         raise value
    187 
    188 else:

/opt/anaconda3/lib/python3.5/site-packages/sqlalchemy/sql/schema.py in __new__(cls, *args, **kw)
    409             metadata._add_table(name, schema, table)
    410             try:
--> 411                 table._init(name, metadata, *args, **kw)
    412                 table.dispatch.after_parent_attach(table, metadata)
    413                 return table

/opt/anaconda3/lib/python3.5/site-packages/sqlalchemy/sql/schema.py in _init(self, name, metadata, *args, **kwargs)
    482         # circular foreign keys
    483         if autoload:
--> 484             self._autoload(metadata, autoload_with, include_columns)
    485 
    486         # initialize all the column, etc. objects.  done after reflection to

/opt/anaconda3/lib/python3.5/site-packages/sqlalchemy/sql/schema.py in _autoload(self, metadata, autoload_with, include_columns, exclude_columns)
    494             autoload_with.run_callable(
    495                 autoload_with.dialect.reflecttable,
--> 496                 self, include_columns, exclude_columns
    497             )
    498         else:

/opt/anaconda3/lib/python3.5/site-packages/sqlalchemy/engine/base.py in run_callable(self, callable_, *args, **kwargs)
   1475 
   1476         """
-> 1477         return callable_(self, *args, **kwargs)
   1478 
   1479     def _run_visitor(self, visitorcallable, element, **kwargs):

/opt/anaconda3/lib/python3.5/site-packages/sqlalchemy/engine/default.py in reflecttable(self, connection, table, include_columns, exclude_columns)
    362             self, connection, table, include_columns, exclude_columns):
    363         insp = reflection.Inspector.from_engine(connection)
--> 364         return insp.reflecttable(table, include_columns, exclude_columns)
    365 
    366     def get_pk_constraint(self, conn, table_name, schema=None, **kw):

/opt/anaconda3/lib/python3.5/site-packages/sqlalchemy/engine/reflection.py in reflecttable(self, table, include_columns, exclude_columns)
    561 
    562         for col_d in self.get_columns(
--> 563                 table_name, schema, **table.dialect_kwargs):
    564             found_table = True
    565 

/opt/anaconda3/lib/python3.5/site-packages/sqlalchemy/engine/reflection.py in get_columns(self, table_name, schema, **kw)
    367         col_defs = self.dialect.get_columns(self.bind, table_name, schema,
    368                                             info_cache=self.info_cache,
--> 369                                             **kw)
    370         for col_def in col_defs:
    371             # make this easy and only return instances for coltype

<string> in get_columns(self, connection, table_name, schema, **kw)

/opt/anaconda3/lib/python3.5/site-packages/sqlalchemy/engine/reflection.py in cache(fn, self, con, *args, **kw)
     52     ret = info_cache.get(key)
     53     if ret is None:
---> 54         ret = fn(self, con, *args, **kw)
     55         info_cache[key] = ret
     56     return ret

/opt/anaconda3/lib/python3.5/site-packages/sqlalchemy/dialects/sqlite/base.py in get_columns(self, connection, table_name, schema, **kw)
   1181     def get_columns(self, connection, table_name, schema=None, **kw):
   1182         info = self._get_table_pragma(
-> 1183             connection, "table_info", table_name, schema=schema)
   1184 
   1185         columns = []

/opt/anaconda3/lib/python3.5/site-packages/sqlalchemy/dialects/sqlite/base.py in _get_table_pragma(self, connection, pragma, table_name, schema)
   1478         qtable = quote(table_name)
   1479         statement = "%s%s(%s)" % (statement, pragma, qtable)
-> 1480         cursor = connection.execute(statement)
   1481         if not cursor._soft_closed:
   1482             # work around SQLite issue whereby cursor.description

/opt/anaconda3/lib/python3.5/site-packages/sqlalchemy/engine/base.py in execute(self, object, *multiparams, **params)
    904         """
    905         if isinstance(object, util.string_types[0]):
--> 906             return self._execute_text(object, multiparams, params)
    907         try:
    908             meth = object._execute_on_connection

/opt/anaconda3/lib/python3.5/site-packages/sqlalchemy/engine/base.py in _execute_text(self, statement, multiparams, params)
   1052             statement,
   1053             parameters,
-> 1054             statement, parameters
   1055         )
   1056         if self._has_events or self.engine._has_events:

/opt/anaconda3/lib/python3.5/site-packages/sqlalchemy/engine/base.py in _execute_context(self, dialect, constructor, statement, parameters, *args)
   1144                 parameters,
   1145                 cursor,
-> 1146                 context)
   1147 
   1148         if self._has_events or self.engine._has_events:

/opt/anaconda3/lib/python3.5/site-packages/sqlalchemy/engine/base.py in _handle_dbapi_exception(self, e, statement, parameters, cursor, context)
   1339                 util.raise_from_cause(
   1340                     sqlalchemy_exception,
-> 1341                     exc_info
   1342                 )
   1343             else:

/opt/anaconda3/lib/python3.5/site-packages/sqlalchemy/util/compat.py in raise_from_cause(exception, exc_info)
    200     exc_type, exc_value, exc_tb = exc_info
    201     cause = exc_value if exc_value is not exception else None
--> 202     reraise(type(exception), exception, tb=exc_tb, cause=cause)
    203 
    204 if py3k:

/opt/anaconda3/lib/python3.5/site-packages/sqlalchemy/util/compat.py in reraise(tp, value, tb, cause)
    183             value.__cause__ = cause
    184         if value.__traceback__ is not tb:
--> 185             raise value.with_traceback(tb)
    186         raise value
    187 

/opt/anaconda3/lib/python3.5/site-packages/sqlalchemy/engine/base.py in _execute_context(self, dialect, constructor, statement, parameters, *args)
   1137                         statement,
   1138                         parameters,
-> 1139                         context)
   1140         except Exception as e:
   1141             self._handle_dbapi_exception(

/opt/anaconda3/lib/python3.5/site-packages/sqlalchemy/engine/default.py in do_execute(self, cursor, statement, parameters, context)
    448 
    449     def do_execute(self, cursor, statement, parameters, context=None):
--> 450         cursor.execute(statement, parameters)
    451 
    452     def do_execute_no_params(self, cursor, statement, context=None):

OperationalError: (sqlite3.OperationalError) no such module: spellfix1 [SQL: 'PRAGMA table_info("spellfix_metadata_titles")']

我知道spellfix1是sqlite的一个功能。但我找不到任何python支持/插件来正确反映它,或者我不完全理解引发的异常。

  1. 我喜欢解释为什么这种情况发生了(为什么会这样 表格特殊,不能正常反映?)
  2. 是否有任何图书馆/扩展名,以便sqlalchemy可以支持这个?
  3. 谢谢!

2 个答案:

答案 0 :(得分:1)

SQLite是一个嵌入式数据库,可以自定义。 Plex媒体服务器添加了spellfix1扩展名; documentation说:

  

spellfix1虚拟表未包含在SQLite合并中,并且不是任何标准SQLite构建的一部分。这是loadable extension

为了能够访问此表,您必须自己编译此扩展,并将其加载到Python数据库连接中。

如果您实际上不需要进行拼写检查,则可以创建没有该虚拟表的数据库副本:

  1. 使用sqlite3命令行shell将数据库转换为文本:

    sqlite3 some/where/...library.db ".dump" > library.sql
    

    (这可以在没有安装模块的情况下工作);

  2. 从SQL脚本中删除虚拟表的创建;这是一条看起来像这样的一条线:

    INSERT INTO sqlite_master(type,name,tbl_name,rootpage,sql)VALUES('table','xxx','xxx',0,'CREATE VIRTUAL TABLE xxx using spellfix1(...)');
    
  3. 从该脚本创建一个新数据库:

    sqlite3 new.db < library.sql
    

答案 1 :(得分:1)

我发现了一个很好的解决方法。

SQLAlchemy允许元数据反射为“仅”&#39;提供可调用性。旗。 这会忽略需要插件的不受支持的表。 (感谢@CL。暗示忽略那些!)

m = sa.MetaData(plex)
m.reflect(only=(lambda x, y: 'spellfix' not in x))
Base = automap_base(metadata=m)
Base.prepare(plex)

像魅力一样!