Question

我的SQL数据库包含表之间的一对多关系。我写了一个查询，以便它包含带有包含json的字段的行（用于相关表行）。这是查询-

select msg.MessageId as id
      ,msg.CreatedDate as [CreatedDate]
      ,
      (select [RecipientTypeId] as [RecipientTypeId]
      ,[RecipientId] as [RecipientId]
      ,[mr.CreatedDate] as [CreatedDate]
      ,[IsRead] as [IsRead]
      ,[ReadDate] as [ReadDate]
       from [dbo].[MsgRecipients] mr
       where msg.messageid = mr.messageid  FOR JSON PATH, INCLUDE_NULL_VALUES) as Recipients
       ,
       (select 
       [Data] as [Data]
      ,[Value] as [Value]
      ,[mc.SomeId] as [SomeId]
       from [dbo].[MessageContent] mc
       where msg.messageid = mc.messageid  FOR JSON PATH, WITHOUT_ARRAY_WRAPPER, INCLUDE_NULL_VALUES) as MessageContent
       from [dbo].[Messages] msg

以下是查询的结果-

在这里，我得到两个字段，其中“收件人”包含对象数组，“消息”包含对象集。

我在ADF中写了这个-

{
    "name": "CopyPipeline_SQL_to_Cosmos",
    "properties": {
        "description": "CopyPipeline_SQL_to_Cosmos",
        "activities": [
            {
                "name": "Copy_lbp",
                "type": "Copy",
                "policy": {
                    "timeout": "7.00:00:00",
                    "retry": 0,
                    "retryIntervalInSeconds": 30,
                    "secureOutput": false,
                    "secureInput": false
                },
                "userProperties": [
                    {
                        "name": "Destination",
                        "value": "Messages1"
                    }
                ],
                "typeProperties": {
                    "source": {
                        "type": "AzureSqlSource",
                        "sqlReaderQuery": {
                            "value": "select msg.MessageId as id, msg.CreatedDate as [CreatedDate], (select [RecipientTypeId] as [RecipientTypeId], [RecipientId] as [RecipientId], [mr.CreatedDate] as [CreatedDate], [ReadDate] as [ReadDate] from [dbo].[MsgRecipients] mr where msg.messageid = mr.messageid  FOR JSON PATH, INCLUDE_NULL_VALUES) as Recipients, (select  [Data] as [Data], [Value] as [Value], [mc.SomeId] as [SomeId] from [dbo].[MessageContent] mc where msg.messageid = mc.messageid  FOR JSON PATH, WITHOUT_ARRAY_WRAPPER, INCLUDE_NULL_VALUES) as MessageContent from [dbo].[Messages] msg  where CreatedDate >= '@{formatDateTime(pipeline().parameters.windowStart, 'yyyy-MM-dd HH:mm' )}' AND CreatedDate < '@{formatDateTime(pipeline().parameters.windowEnd, 'yyyy-MM-dd HH:mm' )}'",
                            "type": "Expression"
                        }
                    },
                    "sink": {
                        "type": "DocumentDbCollectionSink",
                        "nestingSeparator": "",
                        "writeBatchSize": 10000,
                        "writeBehavior": "upsert"
                    },
                    "enableStaging": false
                },
                "inputs": [
                    {
                        "referenceName": "SourceDataset_lbp",
                        "type": "DatasetReference"
                    }
                ],
                "outputs": [
                    {
                        "referenceName": "DestinationDataset_lbp",
                        "type": "DatasetReference"
                    }
                ]
            },
            {
                "name": "Custom1",
                "type": "Custom",
                "policy": {
                    "timeout": "7.00:00:00",
                    "retry": 0,
                    "retryIntervalInSeconds": 30,
                    "secureOutput": false,
                    "secureInput": false
                }
            }
        ],
        "parameters": {
            "windowStart": {
                "type": "String"
            },
            "windowEnd": {
                "type": "String"
            }
        }
    },
    "type": "Microsoft.DataFactory/factories/pipelines"
}

我正在获取这样的记录->

在上图中，您可以注意到在收件人和MessageContent中已添加了整个json。

我期望如此-

Answer 1

我没有可用的数据，因此无法测试，但我认为您将必须使用“模式映射”。

请在此处阅读有关结构的信息 https://docs.microsoft.com/en-us/azure/data-factory/copy-activity-schema-and-type-mapping#alternative-schema-mapping

ADF（Azure数据工厂）将数据从SQL Server复制到Cosmos DB，其中字段包含json对象

1 个答案: