Question

我试图在读取xlsx文件后将数据插入到我的postgres表中。在将xlsx工作表数据插入表中之前，我需要对其进行验证。

我正在研究熊猫数据框df = pd.read_excel('/Users/ankitg3-mac/Downloads/medical_plans/%s' % filename)

我正在使用sqlalchemy作为我的ORM工具。

我的模型班：

    class MedicalPlan(Base):
        __tablename__ = "medical_plans"

        id = Column(Integer, nullable=False , primary_key=True)
        issuer_id = Column(Integer, ForeignKey('issuers.id'), nullable=False)
        service_area_id = Column(Integer)
        name = Column(String)
        on_exchange = Column(Boolean)
        off_exchange = Column(Boolean)
        starting_percentage_fpl = Column(REAL, nullable=False , default=0)
        ending_percentage_fpl = Column(REAL, nullable=False, default=0)
        metal_level_name = Column(String)
        network_type = Column(String)
        type = Column(String)
        is_age_29_plan = Column(Boolean)
        original_medicare = Column(Boolean)
        default_bhp = Column(Boolean, default=False)
        sort_rank_override = Column(Integer)
        recommended = Column(Boolean, default=False)
        comparable_individual_plan_id_trash = Column(Integer)
        group_or_individual_plan_type = Column(String)
        hios_plan_identifier = Column(String)

我正在使用词典列表进行批量插入。

conn.execute(MedicalPlan.__table__.insert(), medicalPlan_dict)

我的medicalPlan_dict如下所示：

[{u'default_bhp': False, u'price_period': u'Monthly', u'plan_description': '', u'sbc_download_url': '', u'price_note': '', u'starting_percentage_fpl': 0, u'is_uhc_plan': False, 'issuer_id': 440, u'part_b_deductible': '', u'promotional_label': '', u'metal_level_name': u'Silver', u'network_url': '', u'group_or_individual_plan_type': u'Group', u'treatment_cost_calculator_url': '', u'hios_plan_identifier': u'99844RI1800001', u'original_medicare': False, u'part_d_prescription_coverage': '', u'recommended': False, u'off_exchange': True, u'is_age_29_plan': False, u'type': u'MetalPlan', u'ending_percentage_fpl': 0, u'plan_detail_footer': '', u'formulary_url': '', u'plan_detail_items': '', u'highlight_6': '', u'highlight_4': '', u'highlight_5': '', u'hsa_eligible': False, u'highlight_3': u'PCP 20% coinsurance', u'highlight_1': u'Silver', u'name': u'WI 80 INDEMNITY 18 OPTION 1 SILVER RX $10/45/90/25%', u'network_description': '', u'plan_detail_header': '', 'service_area_id': 1, u'data_sourced_from': u'uhc', u'plan_year': 2018, u'highlight_2': u'Indemnity', u'on_exchange': False, u'network_type': u'Indemnity'}, {u'default_bhp': False, u'price_period': u'Monthly', u'plan_description': '', u'sbc_download_url': '', u'price_note': '', u'starting_percentage_fpl': 0, u'is_uhc_plan': False, 'issuer_id': 484, u'part_b_deductible': '', u'promotional_label': '', u'metal_level_name': u'Silver', u'network_url': '', u'group_or_individual_plan_type': u'Group', u'treatment_cost_calculator_url': '', u'hios_plan_identifier': u'99806CAAUSJ-TMP1', u'original_medicare': False, u'part_d_prescription_coverage': '', u'recommended': False, u'off_exchange': True, u'is_age_29_plan': False, u'type': u'MetalPlan', u'ending_percentage_fpl': 0, u'plan_detail_footer': '', u'formulary_url': '', u'plan_detail_items': '', u'highlight_6': '', u'highlight_4': '', u'highlight_5': '', u'hsa_eligible': False, u'highlight_3': u'PCP 20% coinsurance', u'highlight_1': u'Silver', u'name': u'WI 80 INDEMNITY 18 OPTION 1 SILVER RX $10/45/90/25%', u'network_description': '', u'plan_detail_header': '', 'service_area_id': 1, u'data_sourced_from': u'uhc', u'plan_year': 2018, u'highlight_2': u'Indemnity', u'on_exchange': False, u'network_type': u'Indemnity'}, {u'default_bhp': False, u'price_period': u'Monthly', u'plan_description': '', u'sbc_download_url': '', u'price_note': '', u'starting_percentage_fpl': 0, u'is_uhc_plan': False, 'issuer_id': 440, u'part_b_deductible': '', u'promotional_label': '', u'metal_level_name': u'Silver', u'network_url': '', u'group_or_individual_plan_type': u'Group', u'treatment_cost_calculator_url': '', u'hios_plan_identifier': u'99844RI1800002', u'original_medicare': False, u'part_d_prescription_coverage': '', u'recommended': False, u'off_exchange': True, u'is_age_29_plan': False, u'type': u'MetalPlan', u'ending_percentage_fpl': 0, u'plan_detail_footer': '', u'formulary_url': '', u'plan_detail_items': '', u'highlight_6': '', u'highlight_4': '', u'highlight_5': '', u'hsa_eligible': False, u'highlight_3': u'PCP 20% coinsurance', u'highlight_1': u'Silver', u'name': u'WI 80 INDEMNITY 18 OPTION 1 SILVER RX $10/45/90/25%', u'network_description': '', u'plan_detail_header': '', 'service_area_id': 1, u'data_sourced_from': u'uhc', u'plan_year': 2018, u'highlight_2': u'Indemnity', u'on_exchange': False, u'network_type': u'Indemnity'}]

我需要先验证数据，然后再将其插入表中。我阅读了有关sqlalchemy验证的信息，并尝试了以下类似的操作，并假设它将在模型级别进行验证。

@validates('hios_plan_identifier')
    def validate_hios_plan_identifier(self, key, hios_plan_identifier):
        assert '/\A(\d{5}[A-Z]{2}[a-zA-Z0-9]{3,7}-TMP|\d{5}[A-Z]{2}\d{3,7}(\-?\d{2})*)\z/,' in hios_plan_identifier
        return hios_plan_identifier

我需要对每个变量进行少量验证。仅应插入通过的行。

我不确定如何在模型级别进行验证。我该如何实现。

Answer 1

两个选项：

在该列上添加CheckConstraint并在其中添加您的正则表达式，请参见 https://www.postgresql.org/docs/9.3/functions-matching.html。无论您以何种方式将数据放入数据库中，该方法都有效。
使用已证明的here基于事件的验证，创建要插入的对象列表，并将session.add_all()用于批量插入。

在sqlalchemy中提供模型类级别的验证

1 个答案: