azure-data-factory等待来源

时间:2017-10-23 14:29:56

标签: azure-data-factory

我正在尝试将示例数据从一个SQL Server DB复制到另一个。 由于某种原因,管道一直在等待源数据。 当我查看源数据集时,没有创建切片。

以下是我的JSONS:

目的地表:

{
  "name": "DestTable1",
    "properties": {
    "structure": [
      {
        "name": "C1",
        "type": "Int16"
      },
      {
        "name": "C2",
        "type": "Int16"
      },
      {
        "name": "C3",
        "type": "String"
      },
      {
        "name": "C4",
        "type": "String"
      }
    ],
      "published": false,
        "type": "SqlServerTable",
          "linkedServiceName": "SqlServer2",
            "typeProperties": {
      "tableName": "OferTarget1"
    },
    "availability": {
      "frequency": "Hour",
        "interval": 1
    }
  }
}

来源表:

{
  "name": "SourceTable1",
    "properties": {
    "structure": [
      {
        "name": "C1",
        "type": "Int16"
      },
      {
        "name": "C2",
        "type": "Int16"
      },
      {
        "name": "C3",
        "type": "String"
      },
      {
        "name": "C4",
        "type": "String"
      }
    ],
      "published": false,
        "type": "SqlServerTable",
          "linkedServiceName": "SqlServer",
            "typeProperties": {
      "tableName": "OferSource1"
    },
    "availability": {
      "frequency": "Hour",
        "interval": 1
    },
    "external": true,
      "policy": { }
  }
}

管道:

{
  "name": "CopyTablePipeline",
    "properties": {
    "description": "Copy data from source table to target table",
      "activities": [
        {
          "type": "Copy",
          "typeProperties": {
            "source": {
              "type": "SqlSource",
              "sqlReaderQuery": "select c1,c2,c3,c4 from OferSource1"
            },
            "sink": {
              "type": "SqlSink",
              "writeBatchSize": 1000,
              "writeBatchTimeout": "60.00:00:00"
            }
          },
          "inputs": [
            {
              "name": "SourceTable1"
            }
          ],
          "outputs": [
            {
              "name": "DestTable1"
            }
          ],
          "policy": {
            "timeout": "01:00:00",
            "concurrency": 1
          },
          "scheduler": {
            "frequency": "Hour",
            "interval": 1
          },
          "name": "CopySqlToSql",
          "description": "Demo Copy"
        }
      ],
        "start": "2017-10-22T09:55:00Z",
          "end": "2017-10-22T13:55:00Z",
            "isPaused": true,
              "hubName": "wer-dev-datafactoryv1_hub",
                "pipelineMode": "Scheduled"
  }
}    

我可以在监视器视图中看到该进程,但是管道卡住并等待源数据到达。

我做错了什么?

1 个答案:

答案 0 :(得分:0)

最初的时间表可能有点棘手。时间片可能等待触发器的原因很少

活动水平

  1. 来源属性
  2.   

    设置“external”:“true”并指定externalData策略信息Azure数据工厂服务,这是一个在数据工厂外部而不是由数据工厂中的活动生成的表。

    1. 并发(在您的情况下不太可能):如果活动的多个切片在特定时间窗口内有效,则也可以保持活动。例如,对于每月活动,您的开始/结束日期是01-01-2014到01-01-2015,如果并发设置为4,则4个月将并行运行,而其余的则停留在消息“#34;等待并发"
    2. 管道级别

      1. 确保DateTime.Now位于start的{​​{1}}和end会计之间。有关如何安排活动的更多信息,请参阅本文https://blogs.msdn.microsoft.com/ukdataplatform/2016/05/03/demystifying-activity-scheduling-with-azure-data-factory/

      2. 暂停:管道可以暂停,在这种情况下,时间片将出现在监视器中,并显示消息"等待管道恢复"。您可以创建管道JSON并生成delay,也可以通过右键单击并点击恢复来恢复管道。

      3. 检查下一次迭代计划的好方法是使用Monitor选项

        enter image description here