Question

我有两个使用SQLAlchemy ORM的子查询版本：

subq1 = session.query(su.DistCode,dr.RtrId, su.RtrCode, su.InvoiceNo, su.SlabId, sh.SchId, sslab.PurQty, sslab.DiscPer, sslab.FlatAmt).\
    join(sh).\
    join(dr, and_(dr.DistCode==su.DistCode, dr.RtrCode==su.RtrCode)).\
    join(sslab,and_(su.SlabId==sslab.SlabId, sh.SchId==sslab.SchId)).\
    subquery()

另一个是：

subq1 = session.query(su.DistCode,dr.RtrId, su.RtrCode, su.InvoiceNo, su.SlabId, sh.SchId).\
    join(sh).\
    join(dr, and_(dr.DistCode==su.DistCode, dr.RtrCode==su.RtrCode)).\
    subquery()

两者之间的唯一区别是加入了一个带有以下内容的联接：

.join(sslab,and_(su.SlabId==sslab.SlabId, sh.SchId==sslab.SchId))

我将两个版本都使用以下代码，一个接一个。

subq2 = session.query(ds.DistCode, ds.RtrId, ds.PrdCde, ds.SalInvDte, ds.SalInvNo,
                     (ds.SalInvQty*ds.SelRateBeforTax).label('SBT'), ds.SalInvSch, 
                     pdet.ProductId, dr.RtrChannelCode, dr.GeoName, dr.RtrClassCode, dr.RtrCode,
                     dr.RtrGroupCode).join(pdet).outerjoin(dr, and_(ds.DistCode==dr.DistCode, ds.RtrId==dr.RtrId)).subquery()

rset = session.query(subq2.c.DistCode, subq2.c.RtrId, subq2.c.RtrCode, subq2.c.SalInvNo,
                     subq2.c.SalInvDte, subq2.c.PrdCde, subq2.c.ProductId, subq2.c.SBT, subq2.c.SalInvSch,
                     subq2.c.RtrChannelCode, subq2.c.RtrClassCode, subq2.c.RtrGroupCode,
                     subq2.c.GeoName, subq1.c.SlabId, subq1.c.SchId).join(subq1,and_(subq1.c.DistCode==subq2.c.DistCode, subq1.c.RtrId==subq2.c.RtrId, subq1.c.InvoiceNo==subq2.c.SalInvNo)).join(spid,and_(subq2.c.ProductId==spid.ProductID, subq1.c.SchId==spid.SchemeID))

df = pd.read_sql(rset.statement, rset.session.bind)

结果吓死我了。第一个查询进入无限循环（或需要10个小时以上的查询；而另一个查询则需要26秒！

作为调试此问题的一种方法，我将'subq1'的两个版本都作为独立查询运行，并且两个版本都运行良好-不到3秒即可运行。

关于如何深入了解此问题的任何想法？

Answer 1

In today's world of quantum computing and self driven cars, I would expect multiple joins to be a simple problem. Turns out it is.

My colleague suggested this answer and it worked. I was missing out on group_by. group_by on subquery apparently reduces processing time significantly.

All I had to do was alter subq1 as:

 subq1 = session.query(su.DistCode,dr.RtrId, su.RtrCode, su.InvoiceNo, su.SlabId, 
                          sh.SchId, sslab.PurQty.label('PQ'), 
                      sslab.DiscPer.label('DP'), 
                      sslab.FlatAmt.label('FA')).join(sh).join(dr, and_(dr.DistCode==su.DistCode, dr.RtrCode==su.RtrCode)).join(sslab,and_(su.SlabId==sslab.SlabId, sh.SchId==sslab.SchId)).group_by(su.DistCode,dr.RtrId, su.RtrCode, su.InvoiceNo, su.SlabId, sh.SchId, sslab.DiscPer, sslab.FlatAmt, sslab.PurQty).subquery()

Notice the group_by at the end. Worked like a charm. runs in less than a minute.

SQLALchemy ORM查询需要很长时间才能运行-子查询中的更改非常温和

1 个答案: