在关系中使用映射属性的正确方法是什么,仍然允许在sqlalchemy中进行预先加载?

时间:2014-02-26 23:04:37

标签: python sql sqlalchemy

我在找出创建SQL Expression mapped attribute的正确方法时遇到了一些麻烦,其中涉及两个表格。我想使用我在关系中创建的属性来加入第三个表。我还想避免N + 1查询问题并使用急切加载。我尝试使用column_property()执行此操作,因为我的理解是我无法与@hybrid_property建立关系或使用它来急切加载(如果我是,请纠正我错)

我有一套相当复杂的模型,但我试图将代码最小化到描述这个问题所需的内容。

Network下方,我创建了属性fqdnd_fqdnfqdn是使用选择器创建的,d_fqdn似乎是魔术完成的,因为我不清楚sqlalchemy如何正确地连接network.namedomain.name列。这是我需要帮助的地方。我的直觉是,我在这里做错了,尽管它主要起作用。

所有这一切的重点是在Network上创建一个完全限定的域名属性,我可以使用该属性与DNSRecord建立关系。反过来,这种关系应该允许我使用DNSRecords急切加载joinedload('dns_records')。我在NetworkDNSRecord之间没有任何数据库外键,只有Network.fqdn == DNSRecord.ownerNetwork.fqdn == DNSRecord.target的隐含关系。

尝试使用基于Network.fqdn的选择器创建此隐含关系似乎不起作用。至少我不能让它正常工作。

然而,使用Network.d_fqdn确实允许我创建关系并且几乎按预期工作。我可以使用DNSRecords之类的查询轻松加载session.query(Network).options(joinedload('dns_records')),而无需进行N + 1次查询。

我说使用Network.d_fqdn几乎可行,因为看起来急切加载Network的查询似乎将其结果乘以domain表中的行数。

例如:session.query(PhysicalSite).options(joinedload('networks')).all()当您真正只需要FROM domain, physical_site时,会将FROM physical_site添加到SQL查询中。

是否有人知道使用Network而不是DNSRecord设置fqdnd_fqdn之间关系的正确方法?或者也许完全使用别的东西?任何帮助将不胜感激。

from sqlalchemy import create_engine, select
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy import Table, MetaData, Column, ForeignKey, Integer, String
from sqlalchemy.orm import (backref, remote, foreign, joinedload, relationship,
                            column_property, with_polymorphic, sessionmaker)
engine = create_engine('sqlite:///foo.db', echo=True)

Base = declarative_base()
metadata = Base.metadata

class Domain(Base):
    __tablename__ = 'domain'
    id = Column(Integer, primary_key=True)
    name = Column(String(256), nullable=False, unique=True)
    networks = relationship('Network',
                            cascade="all,delete,delete-orphan",
                            foreign_keys="[Network.domain_id]")
    def __str__(self):
        return name
    def __repr__(self):
        return "Domain({})".format(self.name)

# This association table is needed for many to many relationship with
# Domains and PhysicalSites
domain_physical_site_association_table = Table(
    'domain_physical_site', metadata,
    Column('domain_id', Integer, ForeignKey('domain.id')),
    Column('physical_site_id', Integer, ForeignKey('physical_site.id')))

class PhysicalSite(Base):
    __tablename__ = 'physical_site'
    id = Column(Integer, primary_key=True)
    name = Column(String(256), nullable=False)
    code = Column(String(256), nullable=False)
    domains = relationship(
        'Domain',
        secondary=domain_physical_site_association_table,
        cascade="all,delete",
        backref="physical_sites")
    def __str__(self):
        return "{}:{}".format(self.name, self.code)
    def __repr__(self):
        return "PhysicalSite({})".format(self.code)

# This association table is needed for many to many relationship with
# Networks and PhysicalSites
network_physical_site_association_table = Table(
        'network_physical_site', metadata,
        Column('network_id', Integer, ForeignKey('network.id')),
        Column('physical_site_id', Integer, ForeignKey('physical_site.id')))

class Network(Base):
    __tablename__ = 'network'
    id = Column(Integer, primary_key=True)
    name = Column(String, nullable=False)
    domain_id = Column(Integer, ForeignKey('domain.id'))

    domain = relationship('Domain', foreign_keys="[Network.domain_id]")
    physical_sites = relationship('PhysicalSite',
            secondary=network_physical_site_association_table,
            cascade="all,delete",
            backref="networks")
    # foo-net.example.com.
    fqdn = column_property(
        (select([name + '.' + Domain.name + '.'])
         .where(Domain.id == domain_id)
         .correlate_except(Domain))
    )
    # This works, but causes an additional from clause on the 'domain' table
    # which effectively multiplies all my results by the length of the
    # domain table
    d_fqdn = column_property(name + "." + Domain.name + ".")
    dns_records = relationship('DNSRecord',
        primaryjoin="or_(\
            remote(DNSRecord.target) == foreign(Network.d_fqdn),\
            remote(DNSRecord.owner) == foreign(Network.d_fqdn))",
        uselist=True)
    def __str__(self):
        return self.fqdn
    def __repr__(self):
        return "Network({})".format(self.fqdn)

class DNSRecord(Base):
    __tablename__ = 'dns_record'
    rr_type = Column(String(256), nullable=False, index=True)
    __mapper_args__ = {'polymorphic_on': rr_type, 'with_polymorphic': '*'}
    id = Column(Integer, primary_key=True)
    rr_class = Column(String(256), nullable=False, default='IN')
    owner = Column(String(256), nullable=False, index=True)
    target = Column(String(256), nullable=True, index=True)
    def __str__(self):
        return "{} IN {} {}".format(self.owner, self.rr_type, self.target)
    def __repr__(self):
        return "DNSRecord({})".format(self.rr_type)

class DNSRecordA(DNSRecord):
    __mapper_args__ = {'polymorphic_identity': 'A'}
    def __str__(self):
        return "{} IN A {}".format(self.owner, self.target)

class DNSRecordPTR(DNSRecord):
    __mapper_args__ = {'polymorphic_identity': 'PTR'}
    def __str__(self):
        return "{} IN PTR {}".format(self.owner, self.target)

Session = sessionmaker()
Session.configure(bind=engine)
session = Session()

另外我会注意到我正在使用sqlalchemy 0.9.3和python 2.7.5

1 个答案:

答案 0 :(得分:2)

所以从关系设计的角度来看,这是一种糟糕的做事方式。让我们只关注网络/域/ dnsrecord。一个非常简单的方法是DNSRecord有一个简单的网络外键;在这种情况下,dnsrecord.owner_network_id和dnsrecord.target_network_id。 dnsrecord.owner和dnsrecord.target的字符串形式只是“self.network.name +”。“self.network.domain.name +”。“”。如果您从网络导航到dns_records,那些网络和域对象已经存在于身份地图中,因此“self.network”和“self.network.domain”等访问者是免费的。

也就是说,关系设计非常关乎只存储一次特定信息。

因此需要注意这个设计很糟糕且不必要,为了使它能够正常工作,我们将引用relationship to non primary mapper几乎可以做任何事情,这里是:< / p>

from sqlalchemy import create_engine, select, and_, or_
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy import Table, MetaData, Column, ForeignKey, Integer, String
from sqlalchemy.orm import mapper, relationship, Session, joinedload, foreign

Base = declarative_base()
metadata = Base.metadata

class Domain(Base):
    __tablename__ = 'domain'
    id = Column(Integer, primary_key=True)
    name = Column(String)
    networks = relationship('Network', backref="domain")

class Network(Base):
    __tablename__ = 'network'
    id = Column(Integer, primary_key=True)
    name = Column(String, nullable=False)
    domain_id = Column(Integer, ForeignKey('domain.id'))

    @property
    def fqdn(self):
        return self.name + "." + self.domain.name + "."

class DNSRecord(Base):
    __tablename__ = 'dns_record'
    id = Column(Integer, primary_key=True)
    owner = Column(String)
    target = Column(String)


fqdn_network = select([DNSRecord, Domain.id.label('domain_id'), Domain.name.label('domain_name')]).alias()

d_fqdn = Network.name + "." + fqdn_network.c.domain_name + "."

dns_alt = mapper(DNSRecord, fqdn_network, non_primary=True)
Network.dns_records = relationship(dns_alt, primaryjoin=
                            and_(
                                Network.domain_id == foreign(fqdn_network.c.domain_id),
                                or_(
                                    fqdn_network.c.target == d_fqdn,
                                    fqdn_network.c.owner == d_fqdn
                                )
                            ),
                            viewonly=True
                        )

engine = create_engine('sqlite:///', echo='debug')
Base.metadata.create_all(engine)

session = Session(engine)

session.add_all([
    DNSRecord(owner="apple.foo.com."),
    DNSRecord(target="peach.foo.com."),
    DNSRecord(owner="banana.foo.com."),
    DNSRecord(target="banana.foo.com."),
    DNSRecord(owner="pear.bar.com."),
    DNSRecord(owner="peach.bar.com."),

    Domain(name="foo.com", networks=[
            Network(name="apple"),
            Network(name="peach"),
            Network(name="banana"),
    ]),
    Domain(name="bar.com", networks=[
            Network(name="pear"),
            Network(name="peach"),
    ])
])
session.commit()

for network in session.query(Network).options(joinedload(Network.dns_records)):
    for dns in network.dns_records:
        print dns.owner, dns.target, network.fqdn
        assert dns.owner == network.fqdn or dns.target == network.fqdn

这方面一个特别令人讨厌的方面是你必须在子查询中发生笛卡尔积:

SELECT network.id AS network_id, network.name AS network_name, network.domain_id AS network_domain_id, anon_1.id AS anon_1_id, anon_1.owner AS anon_1_owner, anon_1.target AS anon_1_target, anon_1.domain_id AS anon_1_domain_id, anon_1.domain_name AS anon_1_domain_name 

FROM network 
 LEFT OUTER JOIN (
        SELECT dns_record.id AS id, dns_record.owner AS owner, dns_record.target AS target, 
         domain.id AS domain_id, domain.name AS domain_name 
         FROM dns_record, domain) AS anon_1 
    ON network.domain_id = anon_1.domain_id AND (anon_1.target = network.name || ? || anon_1.domain_name || ? OR anon_1.owner = network.name || ? || anon_1.domain_name || ?)

那是因为我们不能直接将dns_record加入域,除非我们加入类似DNSRecord.target / DNS.Record.owner的字符串拆分或子串的东西,这也不会很好。为了弄清楚如何加入dns_record和domain,我们必须引入“网络”方面的东西。