sqlalchemy BC与postgresql约会

时间:2014-06-19 13:09:32

标签: python-2.7 sqlalchemy postgresql-9.1

我正在通过sqlalchemy处理postgresql数据库。我需要使用postgresql" date"来处理BC日期。列类型(它支持BC日期http://www.postgresql.org/docs/9.2/static/datatype-datetime.html)。但是虽然我可以以字符串的形式插入BC日期(例如' 2000-01-01 BC'),但我无法检索它们。试图从数据库中获取BC日期引发值错误:年份超出范围。起初我认为原因是它的类型Date中的sqlalchemy使用python datetime.date模块,它不支持BC日期,但我不完全确定。我试图找到一种方法来映射" date"在数据库中键入自定义python类(绕过datetime.date模块的使用)但我还没有成功(我找到了在sqlalchemy中为各种处理步骤添加逻辑的方法,比如result_processor或者带有column_expression的bind_expression,但我没有& #39;设法让它发挥作用。)

以下是重现问题的示例。

    from sqlalchemy import create_engine, Column, Integer, String, Date
    from sqlalchemy.orm import sessionmaker
    from sqlalchemy.ext.declarative import declarative_base

    engine = create_engine('postgresql://database:database@localhost:5432/sqlalchemy_test', echo=True)
    Base = declarative_base()

    class User(Base):
        __tablename__ = 'users'

        id = Column(Integer, primary_key=True)
        name = Column(String)
        date = Column(Date)

        def __repr__(self):
            return "<User(name='%s', date='%s')>" % (self.name, self.date)

    Base.metadata.create_all(engine)
    Session = sessionmaker(bind=engine)
    session = Session()

    # remove objects from previous launches
    session.query(User).delete()

    import datetime
    a_date = datetime.date.today()
    # Insert valid date in datetime.date format, it works.
    ed_user = User(name='ed', date=a_date)
    # Insert BC date in string format, it works.
    al_user = User(name='al', date='1950-12-16 BC')

    session.add_all([ed_user, al_user])
    session.commit()

    # Insert data via raw sql, it works
    conn = engine.connect()
    conn.execute('INSERT INTO users (name, date) VALUES (%s, %s)', ("Lu", "0090-04-25 BC"))

    # Check that all 3 objects are present
    user_names = session.query(User.name).all()
    print user_names


    # Check entry with AD date, it works.
    user = session.query(User).filter_by(name='ed').first()
    print '-- user vith valid date: ', user, user.date

    # But when trying to fetch date fields in any way, it raises Value error

    # Trying to fetch model, it raises Value error
    users = session.query(User).all()
    print users

    # Trying to fetch only field, it raises Value error
    user_dates = session.query(User.date).all()
    print user_dates

    # Even trying to fetch dates in raw sql raises Value error
    a = conn.execute('SELECT * FROM users')
    print [c for c in a]

输出跟踪:

    2014-06-19 16:06:20,466 INFO sqlalchemy.engine.base.Engine select version()
    2014-06-19 16:06:20,466 INFO sqlalchemy.engine.base.Engine {}
    2014-06-19 16:06:20,468 INFO sqlalchemy.engine.base.Engine select current_schema()
    2014-06-19 16:06:20,468 INFO sqlalchemy.engine.base.Engine {}
    2014-06-19 16:06:20,470 INFO sqlalchemy.engine.base.Engine SELECT CAST('test plain returns' AS VARCHAR(60)) AS anon_1
    2014-06-19 16:06:20,470 INFO sqlalchemy.engine.base.Engine {}
    2014-06-19 16:06:20,471 INFO sqlalchemy.engine.base.Engine SELECT CAST('test unicode returns' AS VARCHAR(60)) AS anon_1
    2014-06-19 16:06:20,471 INFO sqlalchemy.engine.base.Engine {}
    2014-06-19 16:06:20,473 INFO sqlalchemy.engine.base.Engine show standard_conforming_strings
    2014-06-19 16:06:20,473 INFO sqlalchemy.engine.base.Engine {}
    2014-06-19 16:06:20,475 INFO sqlalchemy.engine.base.Engine select relname from pg_class c join pg_namespace n on n.oid=c.relnamespace where n.nspname=current_schema() and relname=%(name)s
    2014-06-19 16:06:20,475 INFO sqlalchemy.engine.base.Engine {'name': u'users'}
    2014-06-19 16:06:20,477 INFO sqlalchemy.engine.base.Engine BEGIN (implicit)
    2014-06-19 16:06:20,478 INFO sqlalchemy.engine.base.Engine DELETE FROM users
    2014-06-19 16:06:20,478 INFO sqlalchemy.engine.base.Engine {}
    2014-06-19 16:06:20,480 INFO sqlalchemy.engine.base.Engine INSERT INTO users (name, date) VALUES (%(name)s, %(date)s) RETURNING users.id
    2014-06-19 16:06:20,481 INFO sqlalchemy.engine.base.Engine {'date': datetime.date(2014, 6, 19), 'name': 'ed'}
    2014-06-19 16:06:20,482 INFO sqlalchemy.engine.base.Engine INSERT INTO users (name, date) VALUES (%(name)s, %(date)s) RETURNING users.id
    2014-06-19 16:06:20,482 INFO sqlalchemy.engine.base.Engine {'date': '1950-12-16 BC', 'name': 'al'}
    2014-06-19 16:06:20,483 INFO sqlalchemy.engine.base.Engine COMMIT
    2014-06-19 16:06:20,494 INFO sqlalchemy.engine.base.Engine INSERT INTO users (name, date) VALUES (%s, %s)
    2014-06-19 16:06:20,494 INFO sqlalchemy.engine.base.Engine ('Lu', '0090-04-25 BC')
    2014-06-19 16:06:20,495 INFO sqlalchemy.engine.base.Engine COMMIT
    2014-06-19 16:06:20,517 INFO sqlalchemy.engine.base.Engine BEGIN (implicit)
    2014-06-19 16:06:20,517 INFO sqlalchemy.engine.base.Engine SELECT users.name AS users_name 
    FROM users
    2014-06-19 16:06:20,517 INFO sqlalchemy.engine.base.Engine {}
    -- user names:  [(u'ed',), (u'al',), (u'Lu',)]
    2014-06-19 16:06:20,520 INFO sqlalchemy.engine.base.Engine SELECT users.id AS users_id, users.name AS users_name, users.date AS users_date 
    FROM users 
    WHERE users.name = %(name_1)s 
     LIMIT %(param_1)s
    2014-06-19 16:06:20,520 INFO sqlalchemy.engine.base.Engine {'name_1': 'ed', 'param_1': 1}
    -- user with valid date:  <User(name='ed', date='2014-06-19')> 2014-06-19
    2014-06-19 16:06:20,523 INFO sqlalchemy.engine.base.Engine SELECT users.id AS users_id, users.name AS users_name, users.date AS users_date 
    FROM users
    2014-06-19 16:06:20,523 INFO sqlalchemy.engine.base.Engine {}
    Traceback (most recent call last):
      File "test_script.py", line 49, in <module>
        users = session.query(User).all()
      File "/home/pavel/.virtualenvs/common/local/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2292, in all
        return list(self)
      File "/home/pavel/.virtualenvs/common/local/lib/python2.7/site-packages/sqlalchemy/orm/loading.py", line 65, in instances
        fetch = cursor.fetchall()
      File "/home/pavel/.virtualenvs/common/local/lib/python2.7/site-packages/sqlalchemy/engine/result.py", line 788, in fetchall
        self.cursor, self.context)
      File "/home/pavel/.virtualenvs/common/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1111, in _handle_dbapi_exception
        util.reraise(*exc_info)
      File "/home/pavel/.virtualenvs/common/local/lib/python2.7/site-packages/sqlalchemy/engine/result.py", line 782, in fetchall
        l = self.process_rows(self._fetchall_impl())
      File "/home/pavel/.virtualenvs/common/local/lib/python2.7/site-packages/sqlalchemy/engine/result.py", line 749, in _fetchall_impl
        return self.cursor.fetchall()
    ValueError: year is out of range

我使用postgresql 9.1.11,sqlalchemy 0.9.4,python 2.7

我的问题是:有没有办法在postgresql中使用BC日期&#34; date&#34;通过sqlalchemy键入列(至少将其作为字符串检索,但透视将其映射到可以使用BC日期的某个模块,如FlexiDate或astropy.time)?

1 个答案:

答案 0 :(得分:3)

我正在挖掘这个问题,我发现这个Value错误异常不是由sqlalchemy引发的,而是由数据库驱动程序(在本例中为psycopg2)引发的。 这里,如果使用从上面创建的数据库执行,此代码将引发相同的值错误异常。

    import psycopg2
    conn = psycopg2.connect("dbname=sqlalchemy_test user=database password=database host=localhost port=5432")
    cursor = conn.cursor()
    cursor.execute("SELECT * FROM users")
    cursor.fetchall()

输出:

    Traceback (most recent call last):
      File "test_script.py", line 5, in <module>
        values = cursor.fetchall()
    ValueError: year is out of range

看起来它试图从数据库中检索的值创建datetime.date对象并失败。

我也试过pg8000方言,它没有引发Value错误,但它吃了BC部分,返回datetime.date对象的年,月和日(我的意思是从数据库行中获取值&#39; 1990-公元前11-25&#39;它返回datetime.date(1990,11,25),实际上是1990-11-25)。 这里的例子:

    import pg8000
    conn = pg8000.connect(user='postgres', host='localhost', port=5432,
                          database='sqlalchemy_test', password='postgres')
    cursor = conn.cursor()
    cursor.execute('SELECT * FROM users')
    values = cursor.fetchall()
    print values
    print [str(val[2]) for val in values]

输出:

    ([1, u'ed', datetime.date(2014, 6, 19)], [2, u'al', datetime.date(1950, 12, 16)], [3, u'Lu', datetime.date(90, 4, 25)])
    ['2014-06-19', '1950-12-16', '0090-04-25']

我也想尝试py-postgresql方言,但它仅适用于python 3+,而不是一个选项(生产使用python 2.7.6进行此项目)。

所以,回答我的问题是:&#34;不,在sqlalchemy中是不可能的。&#34;,因为数据库文字和python对象之间的类型转换发生在数据库驱动程序中,而不是在sqlalchemy部分。

更新

虽然在sqlalchemy中确实无法做到这一点,但可以在psycopg2中定义自己的类型转换(并覆盖默认行为),并从数据库返回watever中创建日期类型。这是一个例子:

    # Cast PostgreSQL date into string
    # http://initd.org/psycopg/docs/advanced.html#type-casting-from-sql-to-python

    import psycopg2

    BcDate = None

    # This function dafines types cast, and as returned database literal is already a string, no
    # additional logic required.
    def cast_bc_date(value, cursor):
        return value

    def register_bc_date(connection):
        global BcDate
        if not BcDate:
            cursor = connection.cursor()
            cursor.execute('SELECT NULL::date')
            psql_date_oid = cursor.description[0][1]

            BcDate = psycopg2.extensions.new_type((psql_date_oid,), 'DATE', cast_bc_date)
            psycopg2.extensions.register_type(BcDate)

    conn = psycopg2.connect(database='sqlalchemy_test', user='database', password='database', host='localhost', port=5432)
    register_bc_date(conn)

这个问题的例子就像一个魅力。并在sqlalchemy中开辟了额外数据处理的方法。