使用bash脚本修改CSV标头

时间:2013-07-22 19:36:59

标签: bash csv sed

我使用bash脚本检索了一堆CSV文件& wget&然后创造&导入到mysql

我遇到的问题是CSV文件格式如此示例

name,affiliate_feed_currency,price,msrp,special_price,affiliate_feed_freight,brand,short_description,description,Product_URL,Product_Image_URL,Product_ThumbNail_URL,Product_Category,Product_Season,Product_Gender,affiliate_feed_availability,Product_Sizes,Product_Colours,Tracking_Code,affiliate_link
"Custom Labeled Beer","AUD$","59.99","59.99","","9","Brewtopia Custom Labelled Beer","","Looking for the most unique head turning way of branding you company event? Or just looking for a gift to impress the 'man-who-has-everything'? For around the price of an imported beer we'll deliver to you our award winning Premium Lager professionally custom branded with your company customer name message or logo","http://www.brewtopia.com.au/design-a-label/custom-beer-wine-labels-bottled-water","http://www.brewtopia.com.au/images/products/medium/0_1195186629.jpg","http://www.brewtopia.com.au/images/products/thumbs/0_1195186629.jpg","Gifts","","","","","","<img src='http://www.is1.clixgalore.com/Impression.asp?BID=34359&AfID=264058&AdID=4817' width='0' height='0' border='0'><A href='http://www.clixGalore.com/PSale.aspx?BID=34359&AfID=264058&AdID=4817&AffDirectURL=http://www.brewtopia.com.au/design-a-label/custom-beer-wine-labels-bottled-water'>Custom Labeled Beer</A>","http://www.clixGalore.com/PSale.aspx?BID=34359&AfID=264058&AdID=4817&AffDirectURL=http://www.brewtopia.com.au/design-a-label/custom-beer-wine-labels-bottled-water"
"Fathers Day Beer","AUD$","59.99","59.99","","9","Fathers Day Beer","Brewtopia Fathers Day Beer","Want to get on dad's good side for Fathers Day this year? Suck up to dad by buying him his OWN custom labelled case of beer. For around the price of an imported beerwe'll deliver to you our award winning Premium Lager professionally custom branded with dad's face and a message. You'll be in his good books for ages!","http://www.brewtopia.com.au/design-a-label/custom-beer-wine-labels-bottled-water","http://www.brewtopia.com.au/images/Dadsday/dads-day.jpg","","Gifts","","","","","","<img src='http://www.is1.clixgalore.com/Impression.asp?BID=34359&AfID=264058&AdID=4817' width='0' height='0' border='0'><A href='http://www.clixGalore.com/PSale.aspx?BID=34359&AfID=264058&AdID=4817&AffDirectURL=http://www.brewtopia.com.au/design-a-label/custom-beer-wine-labels-bottled-water'>Fathers Day Beer</A>","http://www.clixGalore.com/PSale.aspx?BID=34359&AfID=264058&AdID=4817&AffDirectURL=http://www.brewtopia.com.au/design-a-label/custom-beer-wine-labels-bottled-water"
"Valentines Day Beer","AUD$","59.99","59.99","","9","Valentines Day Beer","Brewtopias Valentines Day Beer","Want to get your bloke to commit to you this Valentines Day? Try this... Design him his OWN beer! For around the price of an imported beer we'll deliver to you our award winning Premium Lager professionally custom branded with your man's face and a sexy message on the label. You'll have him popping the question in no time!","http://www.brewtopia.com.au/design-a-label/custom-beer-wine-labels-bottled-water","http://www.brewtopia.com.au/images/valentines/valmed.jpg","http://www.brewtopia.com.au/images/valentines/valthumb.jpg","Gifts","","","","","","<img src='http://www.is1.clixgalore.com/Impression.asp?BID=34359&AfID=264058&AdID=4817' width='0' height='0' border='0'><A href='http://www.clixGalore.com/PSale.aspx?BID=34359&AfID=264058&AdID=4817&AffDirectURL=http://www.brewtopia.com.au/design-a-label/custom-beer-wine-labels-bottled-water'>Valentines Day Beer</A>","http://www.clixGalore.com/PSale.aspx?BID=34359&AfID=264058&AdID=4817&AffDirectURL=http://www.brewtopia.com.au/design-a-label/custom-beer-wine-labels-bottled-water"
"Personalized Beer","AUD$","59.99","59.99","","9","Custom Personalized Beer","Brewtopia Personalized Beer","Looking for the most unique head turning way of branding you company event? Or just looking for a gift to impress the 'man-who-has-everything'? For around the price of an imported beer we'll deliver to you our award winning Premium Lager professionally custom branded with your company customer name message or logo","http://www.brewtopia.com.au/design-a-label/custom-beer-wine-labels-bottled-water","http://www.brewtopia.com.au/images/products/medium/0_1195186629.jpg","http://www.brewtopia.com.au/images/products/thumbs/0_1195186629.jpg","Personalized Beer","","","","","","<img src='http://www.is1.clixgalore.com/Impression.asp?BID=34359&AfID=264058&AdID=4817' width='0' height='0' border='0'><A href='http://www.clixGalore.com/PSale.aspx?BID=34359&AfID=264058&AdID=4817&AffDirectURL=http://www.brewtopia.com.au/design-a-label/custom-beer-wine-labels-bottled-water'>Personalized Beer</A>","http://www.clixGalore.com/PSale.aspx?BID=34359&AfID=264058&AdID=4817&AffDirectURL=http://www.brewtopia.com.au/design-a-label/custom-beer-wine-labels-bottled-water"
"Personalised Beer","AUD$","59.99","59.99","","9","Custom Personalised Beer","Brewtopia Personalised Beer","Looking for the most unique head turning way of branding you company event? Or just looking for a gift to impress the 'man-who-has-everything'? For around the price of an imported beer we'll deliver to you our award winning Premium Lager professionally custom personalised with your company customer name message or logo","http://www.brewtopia.com.au/design-a-label/custom-beer-wine-labels-bottled-water","http://www.brewtopia.com.au/images/products/medium/0_1195186629.jpg","http://www.brewtopia.com.au/images/products/thumbs/0_1195186629.jpg","Personalised Beer","","","","","","<img src='http://www.is1.clixgalore.com/Impression.asp?BID=34359&AfID=264058&AdID=4817' width='0' height='0' border='0'><A href='http://www.clixGalore.com/PSale.aspx?BID=34359&AfID=264058&AdID=4817&AffDirectURL=http://www.brewtopia.com.au/design-a-label/custom-beer-wine-labels-bottled-water'>Personalised Beer</A>","http://www.clixGalore.com/PSale.aspx?BID=34359&AfID=264058&AdID=4817&AffDirectURL=http://www.brewtopia.com.au/design-a-label/custom-beer-wine-labels-bottled-water"
"Personalized Wine","AUD$","12.99","12.99","","9","Custom Personalized wine","Brewtopia Personalized Wine","Looking for the most unique head turning way of branding you company event? Or just looking for a gift to impress the 'man-or-woman-who-has-everything'? We have a range to suit your budget - we'll deliver to you our Premium quality wine professionally custom branded with your company customer name message or logo","http://www.brewtopia.com.au/design-a-label/custom-beer-wine-labels-bottled-water","http://www.brewtopia.com.au/images/products/medium/affiliate buttons_wine.jpg","http://www.brewtopia.com.au/images/products/thumbs/affiliate buttons_wine.jpg.jpg","Personalized Wine","","","","","","<img src='http://www.is1.clixgalore.com/Impression.asp?BID=34359&AfID=264058&AdID=4817' width='0' height='0' border='0'><A href='http://www.clixGalore.com/PSale.aspx?BID=34359&AfID=264058&AdID=4817&AffDirectURL=http://www.brewtopia.com.au/design-a-label/custom-beer-wine-labels-bottled-water'>Personalized Wine</A>","http://www.clixGalore.com/PSale.aspx?BID=34359&AfID=264058&AdID=4817&AffDirectURL=http://www.brewtopia.com.au/design-a-label/custom-beer-wine-labels-bottled-water"
"Personalised Wine","AUD$","12.99","12.99","","9","Custom Personalised wine","Brewtopia Personalised Wine","Looking for the most unique head turning way of branding you company event? Or just looking for a gift to impress the 'man-who-has-everything'? We have a range to suit your budget - we'll deliver to you our Premium quality Wine professionally custom branded with your company customer name message or logo","http://www.brewtopia.com.au/design-a-label/custom-beer-wine-labels-bottled-water","http://www.brewtopia.com.au/images/products/medium/affiliate buttons_wine.jpg","http://www.brewtopia.com.au/images/products/thumbs/affiliate buttons_wine.jpg.jpg","Personalised Wine","","","","","","<img src='http://www.is1.clixgalore.com/Impression.asp?BID=34359&AfID=264058&AdID=4817' width='0' height='0' border='0'><A href='http://www.clixGalore.com/PSale.aspx?BID=34359&AfID=264058&AdID=4817&AffDirectURL=http://www.brewtopia.com.au/design-a-label/custom-beer-wine-labels-bottled-water'>Personalised Wine</A>","http://www.clixGalore.com/PSale.aspx?BID=34359&AfID=264058&AdID=4817&AffDirectURL=http://www.brewtopia.com.au/design-a-label/custom-beer-wine-labels-bottled-water"
"Personalised Water","AUD$","32.40","32.40","","9","Custom Personalised Water","Brewtopia Personalised Water","Looking for the most unique head turning way of branding you company event? Or just looking for a gift to keep you hydrated? We have a range to suit your budget - we'll deliver to you our Pure Spring Water professionally personalized custom branded with your company customer name message or logo","http://www.brewtopia.com.au/design-a-label/custom-beer-wine-labels-bottled-water","http://www.brewtopia.com.au/images/products/medium/affiliate buttons_water.jpg","http://www.brewtopia.com.au/images/products/thumbs/affiliate buttons_water.jpg","Personalised Water","","","","","","<img src='http://www.is1.clixgalore.com/Impression.asp?BID=34359&AfID=264058&AdID=4817' width='0' height='0' border='0'><A href='http://www.clixGalore.com/PSale.aspx?BID=34359&AfID=264058&AdID=4817&AffDirectURL=http://www.brewtopia.com.au/design-a-label/custom-beer-wine-labels-bottled-water'>Personalised Water</A>","http://www.clixGalore.com/PSale.aspx?BID=34359&AfID=264058&AdID=4817&AffDirectURL=http://www.brewtopia.com.au/design-a-label/custom-beer-wine-labels-bottled-water"
"Personalized Water","AUD$","32.40","32.40","","9","Custom Personalized Water","Brewtopia Personalized Water","Looking for the most unique head turning way of branding you company event? Or just looking for a gift to keep you hydrated? We have a range to suit your budget - we'll deliver to you our Pure Spring Water professionally personalized custom branded with your company customer name message or logo","http://www.brewtopia.com.au/design-a-label/custom-beer-wine-labels-bottled-water","http://www.brewtopia.com.au/images/products/medium/affiliate buttons_water.jpg","http://www.brewtopia.com.au/images/products/thumbs/affiliate buttons_water.jpg","Personalized Water","","","","","","<img src='http://www.is1.clixgalore.com/Impression.asp?BID=34359&AfID=264058&AdID=4817' width='0' height='0' border='0'><A href='http://www.clixGalore.com/PSale.aspx?BID=34359&AfID=264058&AdID=4817&AffDirectURL=http://www.brewtopia.com.au/design-a-label/custom-beer-wine-labels-bottled-water'>Personalized Water</A>","http://www.clixGalore.com/PSale.aspx?BID=34359&AfID=264058&AdID=4817&AffDirectURL=http://www.brewtopia.com.au/design-a-label/custom-beer-wine-labels-bottled-water"
"Personalized Champagne","AUD$","19.99","19.99","","9","Custom Personalized Champagne","Brewtopia Personalized Champagne","Looking for the most unique head turning way of branding you company event? Or just looking for a gift to impress the 'man-or-woman-who-has-everything'? We have a range to suit your budget - we'll deliver to you our Premium quality Sparkling Wine professionally custom branded with your company customer name message or logo","http://www.brewtopia.com.au/design-a-label/custom-beer-wine-labels-bottled-water","http://www.brewtopia.com.au/images/products/medium/affiliate buttons_champ.jpg","http://www.brewtopia.com.au/images/products/thumbs/affiliate buttons_champ.jpg","Personalized Champagne","","","","","","<img src='http://www.is1.clixgalore.com/Impression.asp?BID=34359&AfID=264058&AdID=4817' width='0' height='0' border='0'><A href='http://www.clixGalore.com/PSale.aspx?BID=34359&AfID=264058&AdID=4817&AffDirectURL=http://www.brewtopia.com.au/design-a-label/custom-beer-wine-labels-bottled-water'>Personalized Champagne</A>","http://www.clixGalore.com/PSale.aspx?BID=34359&AfID=264058&AdID=4817&AffDirectURL=http://www.brewtopia.com.au/design-a-label/custom-beer-wine-labels-bottled-water"
"Personalised Champagne","AUD$","19.99","19.99","","9","Custom Personalised Champagne","Brewtopia Personalised Champagne","Looking for the most unique head turning way of branding you company event? Or just looking for a gift to impress the 'man-or-woman-who-has-everything'? We have a range to suit your budget - we'll deliver to you our Premium quality Sparkling Wine professionally custom branded with your company customer name message or logo","http://www.brewtopia.com.au/design-a-label/custom-beer-wine-labels-bottled-water","http://www.brewtopia.com.au/images/products/medium/affiliate buttons_champ.jpg","http://www.brewtopia.com.au/images/products/thumbs/affiliate buttons_champ.jpg","Personalized Champagne","","","","","","<img src='http://www.is1.clixgalore.com/Impression.asp?BID=34359&AfID=264058&AdID=4817' width='0' height='0' border='0'><A href='http://www.clixGalore.com/PSale.aspx?BID=34359&AfID=264058&AdID=4817&AffDirectURL=http://www.brewtopia.com.au/design-a-label/custom-beer-wine-labels-bottled-water'>Personalised Champagne</A>","http://www.clixGalore.com/PSale.aspx?BID=34359&AfID=264058&AdID=4817&AffDirectURL=http://www.brewtopia.com.au/design-a-label/custom-beer-wine-labels-bottled-water"

实际名称&amp;列数有所不同,所以我要做的是将所有CSV文件中的标题更改为:

"name","affiliate_feed_currency","price","msrp","special_price","affiliate_feed_freight","brand,short_description","description","Product_URL","Product_Image_URL","Product_ThumbNail_URL","Product_Category","Product_Season","Product_Gender","affiliate_feed_availability","Product_Sizes","Product_Colours","Tracking_Code","affiliate_link"

我知道这可以使用简单的sed语句来实现,但我不知道如何编写它:(

4 个答案:

答案 0 :(得分:5)

是的,这是一个单行程序:

sed -i -r '1{s/(^|$)/"/g;s/,/","/g}' file.csv

这意味着:

  • -i - 正确编辑文件

  • -r - 使用extended regular expressions

  • 1{...} - 仅在文件的第一行{...}执行操作

  • s/(^|$)/"/g - 替换(s)与^一致的行的开头($)或结尾(")它们出现(g),然后

  • s/,/","/g - 将s的每个实例替换为,","

答案 1 :(得分:2)

对于你问题中的输入,这一行应该完成这项工作:

sed '1s/\b/"/g' file

您可以添加-i来进行就地更改。

使用您的数据进行测试:

kent$  echo 'column1,column2,column3,column4,column5
"value1","value2","value3","value4","value5"'|sed '1s/\b/"/g'
"column1","column2","column3","column4","column5"
"value1","value2","value3","value4","value5"

修改

添加一个awk解决方案,它适用于带空格的标题。对于sed one-liner,请查看Zero的答案。

awk -v q="\"" 'NR==1{$0=q$0q;gsub(/,/,q"&"q)}7' file

并测试:

kent$  echo 'column1,column2,column3,column4,col                umn5
"value1","value2","value3","value4","value5"'|awk -v q="\"" 'NR==1{$0=q$0q;gsub(/,/,q"&"q)}7'
"column1","column2","column3","column4","col                umn5"
"value1","value2","value3","value4","value5"

答案 2 :(得分:2)

这可能适合你(GNU sed):

sed -r '1s/[^,]+/"&"/g' file

使用双引号包围任何不是逗号的内容。

答案 3 :(得分:0)

使用awk的一种方法(使用嵌入逗号的csv):

awk 'BEGIN{FS=OFS=","}NR==1{for(i=1;i<=NF;i++) $i=q $i q}1' q='"' mycsv > newcsv