我有一个带有txt文件的azure blob。某些列具有空值，因此当它们保存到数据库表中时，它们为NULL。我可以使用直接SQL和SSIS ETL包。
行示例：
1002,100，黄油，用盐BUTTER鞭打，W盐，Y，0,6.38 ,,,
最后三个假设为null。
当我尝试使用ADF时，我收到此错误：
复制活动遇到用户错误： 错误码= UserErrorInvalidDataValue，'类型= Microsoft.DataTransfer.Common.Shared.HybridDeliveryException，消息=列 'CarbohydratesFactor'包含无效值''。无法转换' ' 输入 '十进制'，源= Microsoft.DataTransfer.Common， '' 类型= System.FormatException，消息=输入 string的格式不正确。，Source = mscorlib，'。
FoodDescriptionsAzureBlob：
{
"name": "FoodDescriptionsAzureBlob",
"properties": {
"structure": [
{
"name": "NutrientDatabankNumber",
"type": "Int32"
},
{
"name": "FoodGroupCode",
"type": "Int32"
},
{
"name": "LongDescription",
"type": "String"
},
{
"name": "ShortDescription",
"type": "String"
},
{
"name": "CommonName",
"type": "String"
},
{
"name": "ManufacturerName",
"type": "String"
},
{
"name": "Survey",
"type": "String"
},
{
"name": "ReferenceDescription",
"type": "String"
},
{
"name": "RefusePercentage",
"type": "Int32"
},
{
"name": "ScientificName",
"type": "String"
},
{
"name": "NitrogenFactor",
"type": "Decimal"
},
{
"name": "ProteinFactor",
"type": "Decimal"
},
{
"name": "FatFactor",
"type": "Decimal"
},
{
"name": "CarbohydratesFactor",
"type": "Decimal"
}
],
"published": false,
"type": "AzureBlob",
"linkedServiceName": "AzureStorageLinkedService",
"typeProperties": {
"fileName": "FOOD_DES.txt",
"folderPath": "gym-nutrition-data/NutrientData/",
"format": {
"type": "TextFormat",
"rowDelimiter": "\n",
"columnDelimiter": "^",
"nullValue": "",
"quoteChar": "~"
}
},
"availability": {
"frequency": "Minute",
"interval": 15
},
"external": true,
"policy": {}
}
}
FoodDescriptionsSQLAzure：
{
"name": "FoodDescriptionsSQLAzure",
"properties": {
"structure": [
{
"name": "NutrientDatabankNumber",
"type": "Int32"
},
{
"name": "FoodGroupCode",
"type": "Int32"
},
{
"name": "LongDescription",
"type": "String"
},
{
"name": "ShortDescription",
"type": "String"
},
{
"name": "CommonName",
"type": "String"
},
{
"name": "ManufacturerName",
"type": "String"
},
{
"name": "Survey",
"type": "String"
},
{
"name": "ReferenceDescription",
"type": "String"
},
{
"name": "RefusePercentage",
"type": "Int32"
},
{
"name": "ScientificName",
"type": "String"
},
{
"name": "NitrogenFactor",
"type": "Decimal"
},
{
"name": "ProteinFactor",
"type": "Decimal"
},
{
"name": "FatFactor",
"type": "Decimal"
},
{
"name": "CarbohydratesFactor",
"type": "Decimal"
}
],
"published": false,
"type": "AzureSqlTable",
"linkedServiceName": "AzureSqlLinkedService",
"typeProperties": {
"tableName": "FoodDescriptions"
},
"availability": {
"frequency": "Minute",
"interval": 15
}
}
}
管道
{
"name": "NutrientDataBlobToAzureSqlPipeline",
"properties": {
"description": "Copy nutrient data from Azure BLOB to Azure SQL",
"activities": [
{
"type": "Copy",
"typeProperties": {
"source": {
"type": "BlobSource",
"treatEmptyAsNull": true
},
"sink": {
"type": "SqlSink",
"writeBatchSize": 10000,
"writeBatchTimeout": "60.00:00:00"
}
},
"inputs": [
{
"name": "FoodGroupDescriptionsAzureBlob"
}
],
"outputs": [
{
"name": "FoodGroupDescriptionsSQLAzure"
}
],
"policy": {
"timeout": "01:00:00",
"concurrency": 1,
"executionPriorityOrder": "NewestFirst"
},
"scheduler": {
"frequency": "Minute",
"interval": 15
},
"name": "FoodGroupDescriptions",
"description": "#1 Bulk Import FoodGroupDescriptions"
},
{
"type": "Copy",
"typeProperties": {
"source": {
"type": "BlobSource",
"treatEmptyAsNull": true
},
"sink": {
"type": "SqlSink",
"writeBatchSize": 10000,
"writeBatchTimeout": "60.00:00:00"
}
},
"inputs": [
{
"name": "FoodDescriptionsAzureBlob"
}
],
"outputs": [
{
"name": "FoodDescriptionsSQLAzure"
}
],
"policy": {
"timeout": "01:00:00",
"concurrency": 1,
"executionPriorityOrder": "NewestFirst"
},
"scheduler": {
"frequency": "Minute",
"interval": 15
},
"name": "FoodDescriptions",
"description": "#2 Bulk Import FoodDescriptions"
}
],
"start": "2015-07-14T00:00:00Z",
"end": "2015-07-14T00:00:00Z",
"isPaused": false,
"hubName": "gymappdatafactory_hub",
"pipelineMode": "Scheduled"
}
}
我尝试设置“treatEmptyAsNull”：管道中是真的，没有运气。
答案 0 :(得分：2)
我必须从blob数据集中删除"rowDelimiter": "\n",。
答案 1 :(得分：0)
源文本文件主要有行分隔符＆＃34; \ r \ n＆＃34;。使用行分隔符设置＆＃34; \ n＆＃34;，最后一列将具有数据值＆＃34; \ r＆＃34;这不是一个空字符串，不会被视为空值。如果没有行分隔符设置，ADF Copy默认使用行分隔符＆＃34; \ r \ n＆＃34;，最后一列将具有空字符串，可以视为空。