Question

数据工厂管道使用来自源数据仓库的复制活动->暂存blob存储->接收器数据仓库。

从源到Blob的复制按预期进行（行被复制）。从分段到接收器的复制失败-已复制0行

禁用Polybase并使用批量插入。

{
    "name": "PI_TEST",
    "properties": {
        "activities": [
            {
                "name": "MaterializedEventIdFilter_Copy",
                "type": "Copy",
                "dependsOn": [],
                "policy": {
                    "timeout": "7.00:00:00",
                    "retry": 0,
                    "retryIntervalInSeconds": 30,
                    "secureOutput": false,
                    "secureInput": false
                },
                "userProperties": [
                    {
                        "name": "Destination",
                        "value": "[formigration].[MaterializedEventIdFilter]"
                    }
                ],
                "typeProperties": {
                    "source": {
                        "type": "SqlDWSource",
                        "sqlReaderStoredProcedureName": "[formigration].[proc_GetStgMaterializedEventIdFilter]"
                    },
                    "sink": {
                        "type": "SqlDWSink",
                        "allowPolyBase": true,
                        "writeBatchSize": 100000,
                        "polyBaseSettings": {
                            "rejectValue": 0,
                            "rejectType": "value",
                            "useTypeDefault": false
                        }
                    },
                    "enableStaging": true,
                    "stagingSettings": {
                        "linkedServiceName": {
                            "referenceName": "riskstoreprd",
                            "type": "LinkedServiceReference"
                        },
                        "enableCompression": true
                    }
                },
                "inputs": [
                    {
                        "referenceName": "ioPrePrdMaterializedEventIdFilter",
                        "type": "DatasetReference"
                    }
                ],
                "outputs": [
                    {
                        "referenceName": "CloudPrdMaterializedEventIdFilter",
                        "type": "DatasetReference"
                    }
                ]
            },
            {
                "name": "MaterialisedEvent",
                "type": "SqlServerStoredProcedure",
                "dependsOn": [
                    {
                        "activity": "MaterializedEventIdFilter_Copy",
                        "dependencyConditions": [
                            "Succeeded"
                        ]
                    }
                ],
                "policy": {
                    "timeout": "7.00:00:00",
                    "retry": 2,
                    "retryIntervalInSeconds": 30,
                    "secureOutput": false,
                    "secureInput": false
                },
                "userProperties": [],
                "typeProperties": {
                    "storedProcedureName": "[formigration].[proc_SetStgMaterializedEventIdFilter]"
                },
                "linkedServiceName": {
                    "referenceName": "cloud_prd",
                    "type": "LinkedServiceReference"
                }
            }
        ],
        "annotations": []
    },
    "type": "Microsoft.DataFactory/factories/pipelines"
}

我希望blob中的数据能够进入接收器，但不会复制任何行。

编辑1：检查数据仓库（接收器）是否建立了连接，我可以在一秒钟内观察到从Blob存储创建的外部表等，但没有数据被复制到其中。

INSERT INTO [formigration].[MaterializedEventIdFilter] SELECT * FROM [ADFCopyGeneratedExternalTable_307e2c7f-a56f-4b75-86fb-10ab0cb94548]

Answer 1

在polybase中，外部表只是对Blob存储文件夹/文件的引用，它们没有任何行。如果您想将数据实际复制到仓库中，请创建一个常规表并将其用作复制活动中的接收器！

希望这对您有帮助！

数据工厂Polybase-零行复制到接收器

1 个答案: