Question

我正在构建一个python数据库，以便在星型模式数据库之上进行分析，并且由于数据框中存在一些重复的列密钥而无法集成pandas和sqlalchemy。

这是班级：

class Student(Base):
    __tablename__ = 'DimStudent'

    id = Column('StudentKey', Integer, primary_key=True)
    srcstudentid = ('SrcStudentId', Integer)
    firstname = Column('FirstName', String)
    middlename = Column('MiddleName', String)
    lastname = Column('LastName', String)
    lep = Column('LimitedEnglishProficiency', String)
    frl = Column('FreeReducedLunch', String)
    sped = Column('SpecialEducation', String)

class School(Base):
    __tablename__ = 'DimSchool'

    id = Column('SchoolKey', Integer, primary_key=True)
    name = Column('SchoolName', String)
    district = Column('SchoolDistrict', String)
    statecode = Column('StateCode', String)

class StudentScore(Base):
    __tablename__ = 'FactStudentScore'

    studentkey = Column('StudentKey', Integer, ForeignKey('DimStudent.StudentKey'), primary_key=True)
    teacherkey = Column('TeacherKey', Integer, ForeignKey('DimTeacher.TeacherKey'), primary_key=True)    
    schoolkey = Column('SchoolKey', Integer, ForeignKey('DimSchool.SchoolKey'), primary_key = True)
    assessmentkey = Column('AssessmentKey', Integer, ForeignKey('DimAssessment.AssessmentKey'), primary_key=True)
    subjectkey = Column('SubjectKey', Integer, ForeignKey('DimSubject.SubjectKey'), primary_key=True)
    yearcyclekey = Column('YearCycleKey', Integer, ForeignKey('DimYearCycle.YearCycleKey'), primary_key=True)
    pointspossible = Column('PointsPossible', Integer)
    pointsreceived = Column('PointsReceived', Integer)

    student = relationship("Student", backref=backref('studentscore'))
    school = relationship("School", backref=backref('studentscore'))
    assessment = relationship("Assessment", backref='studentscore')
    teacher = relationship("Teacher", backref='studentscore')
    subject = relationship("Subject", backref='studentscore')
    yearcycle = relationship("YearCycle", backref='studentscore')

每当我查询我的数据时，我总是提出重复的数据列，例如，在此ORM调用中的学校键，然后从中构建数据帧。

school = session.query(StudentScore, School, Subject)\    
.join(StudentScore.school).join(StudentScore.subject)\
.filter(School.name.like('%Dever%'))\
.filter(Subject.code == 'Math')

 a = pd.read_sql(school.statement, school.session.bind)

这个SO thread提供了一个很好的转置技术来删除副本。

 a = a.T.drop_duplicates().T

但是，当我在IDE变量资源管理器中与此数据帧交互时，我仍然遇到错误。错误是：＆＃34;重新索引仅对具有唯一值的索引对象有效，并且＃34;

知道问题出在哪里？

Answer 1

找到正确的答案！而不是最简单的：

 a = a.T.drop_duplicates().T

我改为使用groupby删除重复项：

df.T.groupby(level=0).first().T

那就是说，我不确定原始错误的驱动因素是什么。此外，新代码行的速度比旧代码快10-100倍。

查询SQLAlchemy到Pandas DF时重复列？

1 个答案: