Azure数据工厂接收器目标筛选器

Question

如何使用Azure数据工厂在目标表的更新接收器操作中添加筛选器。

基本上，我正在尝试实现以下查询，并且在目标接收器中需要过滤器end_date ='9999-12-31。

update testdb.test_scd2
set end_date = SourceStream.end_date
where cust_id = SourceStream.cust_id
and end_date = '9999-12-31

' ADF脚本-根据Mark的评论进行更新。但是它正在更新目标表中的所有数据。只需要用end_date更新一行-'9999-12-31'

source(output(
        cust_id as string,
        end_date_new as date
    ),
    allowSchemaDrift: true,
    validateSchema: false,
    ignoreNoFilesFound: false,
    wildcardPaths:['walgreen/source/test_upd.dat']) ~> FFWCustomerUpd
source(output(
        cust_id as string,
        eff_date as date,
        end_date as date,
        first_name as string,
        last_name as string,
        status as string
    ),
    allowSchemaDrift: true,
    validateSchema: false,
    ignoreNoFilesFound: false,
    isolationLevel: 'READ_UNCOMMITTED',
    format: 'table') ~> source1
Exists1 alterRow(updateIf(1==1)) ~> AlterTeradataConnectorupd
source1 filter(end_date==toDate('9999-12-31')) ~> Filter1
FFWCustomerUpd, Filter1 exists(FFWCustomerUpd@cust_id == source1@cust_id,
    negate:false,
    broadcast: 'auto')~> Exists1
AlterTeradataConnectorupd sink(input(
        cust_id as string,
        eff_date as date,
        end_date as date,
        first_name as string,
        last_name as string,
        status as string
    ),
    allowSchemaDrift: true,
    validateSchema: false,
    deletable:false,
    insertable:false,
    updateable:true,
    upsertable:false,
    keys:['cust_id'],
    format: 'table',
    mapColumn(
        cust_id,
        end_date = end_date_new
    ),
    skipDuplicateMapInputs: true,
    skipDuplicateMapOutputs: true) ~> TeradataConnectorupd

Answer 1

在“更改”中的“更改行”中设置策略：end_date == 9999-12-31

在接收器中，将您的关键列设置为cust_id

Answer 2

这是伪代码概述，其逻辑流程如下所示：

1. 新传入的源+目标表作为源
2. 在底部信息流中，仅过滤您需要替换的日期
3. 仅过滤顶部日期中具有该日期的ID
4. 将Update的Alter Row设置为true（），以便所有行都将更新。在这一点上，我们有信心他们都是9999-12-31
5. 仅在设置“允许更新”的情况下写入水槽