Question

我正在尝试将表从数据库A迁移到同一Azure SQL Server中的数据库B。我成功使用Azure Data Factory将数据从Azure Blob存储迁移到Azure数据库，但是我似乎不明白如何修改我的ADF代码以跨数据库迁移数据。

我已经学会了借助This StackOverflow Link来迁移数据，但是我正在寻找使用“复制活动”迁移表的方法。

我的Azure Blob到Azure SQL的管道如下。请提出修改建议，这些修改将导致数据从一个Azure SQL DB迁移到另一个。这是完整的数据工厂代码。我正在寻找一种解决方法或至少一些指导我的资源。预先感谢。

Azure SQL链接服务

{
"name": "AzureSqlLinkedService",
"properties": {
    "description": "",
    "hubName": "dalete_hub",
    "type": "AzureSqlDatabase",
    "typeProperties": {
        "connectionString": "Data Source=tcp:server.database.windows.net,1433;Initial Catalog=DB;Integrated Security=False;User ID=login@server.database.windows.net;Password=**********;Connect Timeout=30;Encrypt=True"
    }
}

}

Azure存储链接服务

{
"name": "AzureStorageLinkedService",
"properties": {
    "description": "",
    "hubName": "dalete_hub",
    "type": "AzureStorage",
    "typeProperties": {
        "connectionString": "DefaultEndpointsProtocol=https;AccountName=newstorageaccount;AccountKey=**********"
    }
}

}

输入数据集

   {
    "name": "InputDataset",
    "properties": {
        "structure": [
            {
                "name": "Region",
                "type": "String"
            },
            {
                "name": "Sales",
"type": "String"
            }
        ],
        "published": false,
        "type": "AzureBlob",
        "linkedServiceName": "AzureStorageLinkedService",
        "typeProperties": {
            "fileName": "data.txt",
            "folderPath": "adfpoc/",
            "format": {"type": "TextFormat",
                "columnDelimiter": ","
            }
        },
        "availability": {
            "frequency": "Hour",
            "interval": 1
        },
        "external": true,
        "policy": {}
    }
}

输出数据集

    {
    "name": "OutputDataset",
    "properties": {
        "structure": [
            {
                "name": "Region",
                "type": "String"
            },
            {
                "name": "Sales",
                "type": "String"
            }
        ],
        "published": false,
        "type": "AzureSqlTable",
        "linkedServiceName": "AzureSqlLinkedService",
        "typeProperties": {
            "tableName": "data"
        },
        "availability": {
            "frequency": "Hour",
            "interval": 1
        }
    }
}

ADFPipeline

  {
    "name": "ADFTutorialPipeline",
    "properties": {
        "description": "Copy data from a blob to Azure SQL table",
        "activities": [
            {
                "type": "Copy",
                "typeProperties": {
                    "source": {
                        "type": "BlobSource" },
                    "sink": {
                        "type": "SqlSink",
                        "writeBatchSize": 10000,
                        "writeBatchTimeout": "60.00:00:00"
                    }
                },
                "inputs": [
                    {
                        "name": "InputDataset"}
                ],
                "outputs": [
                    {
                        "name": "OutputDataset"
                    }
                ],
                "policy": {
                    "timeout": "01:00:00",
                    "concurrency": 1,
                    "executionPriorityOrder": "NewestFirst"
                },
                "scheduler": {
                    "frequency": "Hour",
                    "interval": 1
                },
                "name": "CopyFromBlobToSQL"
            }
        ],
        "start": "2019-03-11T00:00:00Z",
        "end": "2019-03-12T00:00:00Z",
        "isPaused": false,
        "hubName": "dalete_hub",
        "pipelineMode": "Scheduled"
    }
}

Answer 1

从sql数据库移动到另一个数据库与您已经完成的操作类似，但是这次源将是一个sql表，就像您之前的接收器（或输出）一样。

因此，您应该做的是为新数据库创建一个新的链接服务，然后创建一个数据集作为输入（格式与以前的输出相同，但是更改了链接服务的名称，以便它使用新的链接服务）。

最后，创建管道并正确配置输入和输出数据集。

希望这对您有帮助！

Answer 2

我尝试过与您进行相同的操作，并在Data Factory中成功迁移了表。

例如，我在数据库table1中有一个表dbleon，我想通过复制活动将table1迁移到另一个数据库dbleon1。

我在table1中创建一个新表dbmeon1，该表具有与table1中的dbleon相同的架构。

这是我的ADF代码：

{
    "name": "CopyPipeline_0oh",
    "properties": {
        "activities": [
            {
                "name": "Copy_0oh",
                "type": "Copy",
                "policy": {
                    "timeout": "7.00:00:00",
                    "retry": 0,
                    "retryIntervalInSeconds": 30,
                    "secureOutput": false,
                    "secureInput": false
                },
                "userProperties": [
                    {
                        "name": "Source",
                        "value": "[dbo].[table1]"
                    },
                    {
                        "name": "Destination",
                        "value": "[dbo].[table1]"
                    }
                ],
                "typeProperties": {
                    "source": {
                        "type": "SqlSource"
                    },
                    "sink": {
                        "type": "SqlSink",
                        "writeBatchSize": 10000
                    },
                    "enableStaging": false,
                    "translator": {
                        "type": "TabularTranslator",
                        "columnMappings": {
                            "id": "id",
                            "name": "name"
                        }
                    }
                },
                "inputs": [
                    {
                        "referenceName": "SourceDataset_0oh",
                        "type": "DatasetReference"
                    }
                ],
                "outputs": [
                    {
                        "referenceName": "DestinationDataset_0oh",
                        "type": "DatasetReference"
                    }
                ]
            }
        ]
    },
    "type": "Microsoft.DataFactory/factories/pipelines"
}

源数据集：

{
    "name": "SourceDataset_0oh",
    "properties": {
        "linkedServiceName": {
            "referenceName": "AzureSqlDatabase1",
            "type": "LinkedServiceReference"
        },
        "type": "AzureSqlTable",
        "structure": [
            {
                "name": "id",
                "type": "Int32"
            },
            {
                "name": "name",
                "type": "String"
            }
        ],
        "typeProperties": {
            "tableName": "[dbo].[table1]"
        }
    },
    "type": "Microsoft.DataFactory/factories/datasets"
}

目标数据集：

{
    "name": "DestinationDataset_0oh",
    "properties": {
        "linkedServiceName": {
            "referenceName": "AzureSqlDatabase2",
            "type": "LinkedServiceReference"
        },
        "type": "AzureSqlTable",
        "structure": [
            {
                "name": "id",
                "type": "Int32",
                "precision": 10
            },
            {
                "name": "name",
                "type": "String"
            }
        ],
        "typeProperties": {
            "tableName": "[dbo].[table1]"
        }
    },
    "type": "Microsoft.DataFactory/factories/datasets"
}

希望这会有所帮助。

Azure SQL数据库：使用复制活动将表从一个Azure SQL数据库传输/迁移到另一个Azure SQL数据库

2 个答案: