Question

我试图实现以下目标。我想创建一个python类，将数据库中的所有表转换为pandas数据帧。

我就是这样做的，这不是很通用......

class sql2df():
    def __init__(self, db, password='123',host='127.0.0.1',user='root'):
        self.db = db

        mysql_cn= MySQLdb.connect(host=host,
                        port=3306,user=user, passwd=password, 
                        db=self.db)

        self.table1 = psql.frame_query('select * from table1', mysql_cn)
        self.table2 = psql.frame_query('select * from table2', mysql_cn)
        self.table3 = psql.frame_query('select * from table3', mysql_cn)

现在我可以像这样访问所有表：

my_db = sql2df('mydb')
my_db.table1

我想要类似的东西：

class sql2df():
    def __init__(self, db, password='123',host='127.0.0.1',user='root'):
        self.db = db

        mysql_cn= MySQLdb.connect(host=host,
                        port=3306,user=user, passwd=password, 
                        db=self.db)
        tables = (""" SELECT TABLE_NAME FROM information_schema.TABLES WHERE TABLE_SCHEMA = '%s' """ % self.db)
        <some kind of iteration that gives back all the tables in df as class attributes>

建议最受欢迎......

Answer 1

我会使用SQLAlchemy：

engine = sqlalchemy.create_engine("mysql+mysqldb://root:123@127.0.0.1/%s" % db)

注意syntax是dialect + driver：// username：password @ host：port / database。

def db_to_frames_dict(engine):
    meta = sqlalchemy.MetaData()
    meta.reflect(bind=engine)
    tables = meta.sorted_tables
    return {t: pd.read_sql('SELECT * FROM %s' % t.name,
                           engine.raw_connection())
                   for t in tables}
    # Note: frame_query is depreciated in favor of read_sql

这会返回一个字典，但您也可以将这些作为类属性（例如，通过更新类字典和__getitem__）

class SQLAsDataFrames:
    def __init__(self, engine):
        self.__dict__ = db_to_frames_dict(engine)  # allows .table_name access
    def __getitem__(self, key):                    # allows [table_name] access
        return self.__dict__[key]

在pandas 0.14中，sql代码已经被重写为引擎，IIRC有所有表的帮助器和读取所有表（使用read_sql(table_name)）。

Answer 2

这是我现在拥有的：进口

 import sqlalchemy
 from sqlalchemy import create_engine
 from sqlalchemy import Table, Column,Date, Integer, String, MetaData, ForeignKey
 from sqlalchemy.ext.declarative import declarative_base
 from sqlalchemy.orm import relationship, backref
 import pandas as pd

engine = sqlalchemy.create_engine("mysql+mysqldb://root:password@127.0.0.1/%s" % 'surveytest')
def db_to_frames_dict(engine):
    meta = sqlalchemy.MetaData()
    meta.reflect(bind=engine)
    tables = meta.sorted_tables
    return {t: pd.read_sql('SELECT * FROM %s' % t.name, engine.connect())
               for t in tables}
# Note: frame_query is depreciated in favor of read_sql

尚未开始摆弄这部分！下面：

class SQLAsDataFrames:
    def __init__(self, engine):
        self.__dict__ = db_to_frames_dict(engine)  # allows .table_name access
    def __getitem__(self, key):                    # allows [table_name] access
        return self.__dict__[key]

错误：看起来至少它试图得到表名......

frames=db_to_frames_dict(engine)
frames

Error on sql SELECT * FROM tbl_original_survey_master
---------------------------------------------------------------------------
 AttributeError                            Traceback (most recent call last)
 <ipython-input-4-6b0006e1ce47> in <module>()
 ----> 1 frames=db_to_frames_dict(engine)
 >>>> more tracebck
 ---> 53             con.rollback()
 54         except Exception:  # pragma: no cover
 55             pass

 AttributeError: 'Connection' object has no attribute 'rollback'

感谢您坚持这一点！

Answer 3

感谢所有帮助，这就是我最终使用的内容：

import sqlalchemy
from sqlalchemy import create_engine
from sqlalchemy import Table, Column,Date, Integer, String, MetaData, ForeignKey
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import relationship, backref
import pandas as pd

engine = sqlalchemy.create_engine("mysql+mysqldb://root:123@127.0.0.1/%s" % 'mydb')

def db_to_frames_dict(engine):
    meta = sqlalchemy.MetaData()
    meta.reflect(engine)
    tables = meta.tables.keys()
    cnx = engine.raw_connection()

    return {t: pd.read_sql('SELECT * FROM %s' % t, cnx )
               for t in tables}

class SQLAsDataFrames:
    def __init__(self, engine):
        self.__dict__ = db_to_frames_dict(engine)  # allows .table_name access
    def __getitem__(self, key):                    # allows [table_name] access
        return self.__dict__[key]

Python类将数据库中的所有表转换为pandas数据帧

3 个答案: