Question

我正在使用LOAD DATA LOCAL INFILE将csv加载到我的数据库中。我的数据库中的列数与我的csv文件中的列数相同。我的问题是前1000个左右的行正确导入到sql中，然后接下来的几百行被几列关闭，然后它又恢复正常。我唯一的猜测可能是因为空格。每个字段都包含在＆＃34;＆＃34;我在我的代码中指定了手动更新csv会很痛苦，因为超过700,000行并且我从外部服务器下载并且每天都会更新它。我真的想弄清问题的原因和解决方案。

这是我的加载数据代码（没有数据库连接的东西）：

$fieldseparator = ","; 
$lineseparator = "\r\n";
$doubleQuotes = "\"";
$csvfile = "Doba_Power_Export_Product_Apparel_shoes_and_jewelry.csv";

if(!file_exists($csvfile)) {
die("File not found. Make sure you specified the correct path.");
}

try {
$pdo = new PDO("mysql:host=$databasehost;dbname=$databasename", 
    $databaseusername, $databasepassword,
    array(
        PDO::MYSQL_ATTR_LOCAL_INFILE => true,
        PDO::ATTR_ERRMODE => PDO::ERRMODE_EXCEPTION
    )
);
} catch (PDOException $e) {
die("database connection failed: ".$e->getMessage());
}

$affectedRows = $pdo->exec("
LOAD DATA LOCAL INFILE ".$pdo->quote($csvfile)." INTO TABLE `$databasetable`
  FIELDS TERMINATED BY ".$pdo->quote($fieldseparator)."
  OPTIONALLY ENCLOSED BY ".$pdo->quote($doubleQuotes)."
  LINES TERMINATED BY ".$pdo->quote($lineseparator)."
  IGNORE 1 LINES ". $affectedRows);

echo "Loaded a total of $affectedRows records from this csv file.\n";

我从另一个Stackoverflow线程获取此代码并对其进行了多次修改以到达我所处的位置。我已经搜索了Stackoverflow和许多其他在线板寻找答案，并且无法找到一个完全适合我的方法。我现在的代码集已经从一个接一个的尝试发展到了这一点。我意识到我可能正在寻找错误的地方，或者搜索错误的短语来找到答案。我希望能指出正确的方向。

我不知道这是否有帮助，但这里有几行来自我的csv：

"6","4.00","B & F","17154","GFCAP2","Giovanni Navarre&reg; Solid Genuine Leather Baseball Cap","","Spanning the worlds of sports and fashion, this soft solid leather, all American baseball cap affords attention and comfort to the lady or gentleman wearing it. Strap with hook and loop fastener easily adjusts to any size.","new","","","Giovanni Navarre","100","CN","2016-01-06 14:05:42.0","21548","GFCAP2","","024409080722","Giovanni Navarre&reg; Solid Genuine Leather Baseball Cap","0.24","0","0","0.24","9.16","","0.00","5.39","","5.24","","19.95","1732","in-stock","2016-08-25","1","0","2016-06-28 14:05:39.0","","Apparel, shoes & jewelry||Apparel||Men||Hat","Case UPC:=024409003301||Web Description:=Spanning the worlds of sports and fashion, this soft solid leather, all American baseball cap affords attention and comfort to the lady or gentleman wearing it. Strap with hook and loop fastener easily adjusts to any size.","http://d1k0ppjronk6up.cloudfront.net/products/6/_Store_Support_images_GFCAP2_GFCAP2_800.jpg","800","800","","Apparel","0"

"6","4.00","B & F","17306","GFUM48","All-Weather&trade; 48&quot; Auto-Open Umbrella","","For sporting events, outdoor concerts or just a plain rainy day, this umbrella will please even the most discriminating owner. Measures 48&quot; across the top.","new","","","All Weather","60","CN","2016-01-09 14:05:46.0","21700","GFUM48","","024409081279","All-Weather&trade; 48&quot; Auto-Open Umbrella","0.80","0","0","0.80","10.46","","0.00","4.96","","4.82","","18.95","567","in-stock","2016-10-20","0","0","2016-06-24 14:06:01.0","","Apparel, shoes & jewelry||Accessories||Cold-weather accessories","Case UPC:=024409006296||Web Description:=For sporting events, outdoor concerts or just a plain rainy day, this umbrella will please even the most discriminating owner. Measures 48&quot; across the top.","http://d1k0ppjronk6up.cloudfront.net/products/6/_Store_Support_images_GFUM48_GFUM48_800.jpg","800","800","","Apparel","0"

更新

此行是正确插入的行：

"6","4.00","B & F","17154","GFCAP2","Giovanni Navarre&reg; Solid Genuine Leather Baseball Cap","","Spanning the worlds of sports and fashion, this soft solid leather, all American baseball cap affords attention and comfort to the lady or gentleman wearing it. Strap with hook and loop fastener easily adjusts to any size.","new","","","Giovanni Navarre","100","CN","2016-01-06 14:05:42.0","21548","GFCAP2","","024409080722","Giovanni Navarre&reg; Solid Genuine Leather Baseball Cap","0.24","0","0","0.24","9.16","","0.00","5.39","","5.24","","19.95","1732","in-stock","2016-08-25","1","0","2016-06-28 14:05:39.0","","Apparel, shoes & jewelry||Apparel||Men||Hat","Case UPC:=024409003301||Web Description:=Spanning the worlds of sports and fashion, this soft solid leather, all American baseball cap affords attention and comfort to the lady or gentleman wearing it. Strap with hook and loop fastener easily adjusts to any size.","http://d1k0ppjronk6up.cloudfront.net/products/6/_Store_Support_images_GFCAP2_GFCAP2_800.jpg","800","800","","Apparel","0"

虽然这个插入不正确：

"93","0.00","Alphabroder","4188327","M999","12.7 oz. Fleece Blanket","","100% polyester fleece; One side is anti-pill; 60""""W x 50""""H; Finished with a matching whipstitch (Charcoal has a Black whipstitch);","new","cozy and durable fleece throw; each throw is finished with a matching whipstitch (except charcoal which has a black whipstitch); 60\ x 50\""""""""""""""""""""","Harriton","Harriton","0","","2015-10-30 10:40:51.0","4804844","M999~B03446430","B03446430","","CHARCOAL - OS","1.29","0","0","1.29","11.55","","0.00","7.39","","7.18","21.66","21.66","5618","in-stock","","0","0","2016-06-29 01:21:23.0","2015-10-29 12:13:27.0","Apparel, shoes & jewelry||Accessories||Bags & backpacks","color:=CHARCOAL||manufacturer:=Harriton||Popularity:=380||size:=OS","","","","","Apparel","0"

这些是我的字段：

"supplier_id","drop_ship_fee","supplier_name","product_id","product_sku","title","warranty","description","condition","details","manufacturer","brand_name","case_pack_quantity","country_of_origin","product_last_update","item_id","item_sku","mpn","upc","item_name","item_weight","ship_alone","ship_freight","ship_weight","ship_cost","max_ship_single_box","map","price","custom_price","prepay_price","street_price","msrp","qty_avail","stock","est_avail","pending_order_quantity","qty_on_order","item_last_update","item_discontinued_date","categories","attributes","image_file","image_width","image_height","additional_images","folder_paths","is_customized"

Answer 1

错误解析的行在details字段中有反斜杠。默认情况下，MYSQL将这些视为转义前缀，但看起来您的数据供应商希望它们是文字的。你需要覆盖它。

$affectedRows = $pdo->exec("
LOAD DATA LOCAL INFILE ".$pdo->quote($csvfile)." INTO TABLE `$databasetable`
  FIELDS TERMINATED BY ".$pdo->quote($fieldseparator)."
  OPTIONALLY ENCLOSED BY ".$pdo->quote($doubleQuotes)."
  ESCAPED BY ''
  LINES TERMINATED BY ".$pdo->quote($lineseparator)."
  IGNORE 1 LINES ". $affectedRows);

LOAD DATA LOCAL INFILE跳过列

1 个答案: