我正在尝试将示例数据从一个SQL Server DB复制到另一个。 由于某种原因,管道一直在等待源数据。 当我查看源数据集时,没有创建切片。
以下是我的JSONS:
目的地表:
{
"name": "DestTable1",
"properties": {
"structure": [
{
"name": "C1",
"type": "Int16"
},
{
"name": "C2",
"type": "Int16"
},
{
"name": "C3",
"type": "String"
},
{
"name": "C4",
"type": "String"
}
],
"published": false,
"type": "SqlServerTable",
"linkedServiceName": "SqlServer2",
"typeProperties": {
"tableName": "OferTarget1"
},
"availability": {
"frequency": "Hour",
"interval": 1
}
}
}
来源表:
{
"name": "SourceTable1",
"properties": {
"structure": [
{
"name": "C1",
"type": "Int16"
},
{
"name": "C2",
"type": "Int16"
},
{
"name": "C3",
"type": "String"
},
{
"name": "C4",
"type": "String"
}
],
"published": false,
"type": "SqlServerTable",
"linkedServiceName": "SqlServer",
"typeProperties": {
"tableName": "OferSource1"
},
"availability": {
"frequency": "Hour",
"interval": 1
},
"external": true,
"policy": { }
}
}
管道:
{
"name": "CopyTablePipeline",
"properties": {
"description": "Copy data from source table to target table",
"activities": [
{
"type": "Copy",
"typeProperties": {
"source": {
"type": "SqlSource",
"sqlReaderQuery": "select c1,c2,c3,c4 from OferSource1"
},
"sink": {
"type": "SqlSink",
"writeBatchSize": 1000,
"writeBatchTimeout": "60.00:00:00"
}
},
"inputs": [
{
"name": "SourceTable1"
}
],
"outputs": [
{
"name": "DestTable1"
}
],
"policy": {
"timeout": "01:00:00",
"concurrency": 1
},
"scheduler": {
"frequency": "Hour",
"interval": 1
},
"name": "CopySqlToSql",
"description": "Demo Copy"
}
],
"start": "2017-10-22T09:55:00Z",
"end": "2017-10-22T13:55:00Z",
"isPaused": true,
"hubName": "wer-dev-datafactoryv1_hub",
"pipelineMode": "Scheduled"
}
}
我可以在监视器视图中看到该进程,但是管道卡住并等待源数据到达。
我做错了什么?
答案 0 :(得分:0)
最初的时间表可能有点棘手。时间片可能等待触发器的原因很少
SqlServerTable
,则属性external
应为true。我亲自陷入了复制粘贴JSON文件的陷阱,我花了一段时间来理解这一点。这里有更多文献:https://github.com/aelij/azure-content/blob/master/articles/data-factory/data-factory-sqlserver-connector.md 设置“external”:“true”并指定externalData策略信息Azure数据工厂服务,这是一个在数据工厂外部而不是由数据工厂中的活动生成的表。
确保DateTime.Now位于start
的{{1}}和end
会计之间。有关如何安排活动的更多信息,请参阅本文https://blogs.msdn.microsoft.com/ukdataplatform/2016/05/03/demystifying-activity-scheduling-with-azure-data-factory/
暂停:管道可以暂停,在这种情况下,时间片将出现在监视器中,并显示消息"等待管道恢复"。您可以创建管道JSON并生成delay
,也可以通过右键单击并点击恢复来恢复管道。
检查下一次迭代计划的好方法是使用Monitor选项