将Amazon S3用作AWS DMS的源-源端点外部表定义

时间:2020-05-18 08:11:08

标签: amazon-web-services amazon-s3 dms

在将CSV文件从AWS S3迁移到AWS RDS时,我需要一些帮助。

有人可以帮助我在源端点中定义json外部表定义,以将2个或多个表包含到同一数据库中吗?

目前,我能够将一个CSV文件迁移到RDS中。

但是我需要将S3存储桶的所有CSV文件都迁移到AWS RDS(为此使用AWS DMS服务)。

这是我定义的架构

{ “ TableCount”:“ 2”, “表格”:[ { “ TableName”:“ Active_Data”, “ TablePath”:“ Data / Active_Data /”, “ TableOwner”:“数据”, “ TableColumns”:[ { “ ColumnName”:“ LOCATIONID”, “ ColumnType”:“ STRING”, “ ColumnNullable”:“ false”, “ ColumnLength”:“ 10” }, { “ ColumnName”:“ id”, “ ColumnType”:“ STRING”, “ ColumnLength”:“ 20” }, { “ ColumnName”:“ class_code”, “ ColumnType”:“ STRING”, “ ColumnLength”:“ 30” }, { “ ColumnName”:“ org_status”, “ ColumnType”:“ STRING”, “ ColumnLength”:“ 10” }, { “ ColumnName”:“名称”, “ ColumnType”:“ STRING”, “ ColumnLength”:“ 100” }, { “ ColumnName”:“ m_name”, “ ColumnType”:“ STRING”, “ ColumnNullable”:“ true”, “ ColumnLength”:“ 100”

            },
            {
                "ColumnName": "m_id",
                "ColumnType": "STRING",
                "ColumnNullable": "true",
                "ColumnLength": "20"
            },
            {
                "ColumnName": "SPN",
                "ColumnType": "STRING",
                "ColumnNullable": "true",
                "ColumnLength": "100"
            },
            {
                "ColumnName": "SPN_ID",
                "ColumnType": "STRING",
                "ColumnNullable": "true",
                "ColumnLength": "20"
            },
            {
                "ColumnName": "created_time",
                "ColumnType": "DATETIME",
                "ColumnLength": "20"
            }
        ],
        "TableColumnsTotal": "10"
    },
    {
        "TableName": "Resource_Data",
        "TablePath": "Data/Resource_Data/",
        "TableOwner": "Data",
        "TableColumns": [
            {
                "ColumnName": "LOCATIONID",
                "ColumnType": "STRING",
                "ColumnNullable": "false",
                "ColumnLength": "10"
            },
            {
                "ColumnName": "C_name",
                "ColumnType": "STRING",
                "ColumnLength": "100"
            },
            {
                "ColumnName": "C_id",
                "ColumnType": "STRING",
                "ColumnLength":"10"
            },
            {
                "ColumnName": "m_id",
                "ColumnType": "STRING",
                "ColumnLength": "10"
            },
            {
                "ColumnName": "m_name",
                "ColumnType": "STRING",
                "ColumnLength": "100"
            },
            {
                "ColumnName": "SPN",
                "ColumnType": "STRING",
                "ColumnLength": "100"                   
            },
            {
                "ColumnName": "# Managed Res",
                "ColumnType": "INT8",
                "ColumnLength": "10"                    
            }               
        ],
        "TableColumnsTotal": "7"
    }
]

}

S3的文件夹结构如下:

BucketName / Data / Active_Data / Active_Data.csv BucketName / Data / Resource_Data / Resource_Data.csv

定义端点和复制实例后,我开始了任务 它正在创建两个数据库Resource_Data和Customer_Data,但仅加载表(仅Resource_Data)

我希望将两个表都加载到同一数据库中。

您能告诉我我哪里出问题了吗?

0 个答案:

没有答案