将文本前缀和URL列表提取到相应的编号输出行

时间:2019-05-21 21:35:47

标签: curl awk sed grep

我正在尝试从具有URL列表的文本文件(raw.txt)中提取“ product_sku”。然后,我需要将提取的文本作为后缀添加到输出文件(clean.txt)编号为(0-36)的每个URL的末尾

每个网址组都有一个不同的“ product_sku”,因此每行都需要对应于该网址组的“ product_sku”。

我可以使用以下内容提取网址:

cat raw.txt | grep -E -o "WEBSITE/product_images/870-.*" > clean.txt

我正在努力将“ product_sku”添加到列表中。

代码如下所示。

... ...表示为清楚起见已删除的其他网址。

var StockNumber = 'U52684'
var product_id = '972930'
var product_sku = 'NN26ZZA'
        var threesixty_start_frame = 4
    var threesixty_images = WEBSITE/product_images/870-972930/XafJtPxTqNQuPNQpjF5h.JPG
WEBSITE/product_images/870-972930/9YZH6VMT2zH6AntfaAIG.JPG
WEBSITE/product_images/870-972930/SrE9oVwVmJjS58SwZB9q.JPG
WEBSITE/product_images/870-972930/1enLX6dc9bbG7k9SEAfL.JPG

... ...

var StockNumber = 'U75102'
var product_id = '768442'
var product_sku = 'LT65YMP'
        var threesixty_start_frame = 4
    var threesixty_images = WEBSITE/product_images/870-768442/fg7G7CX2Z0oK8mCVlJN3.JPG
WEBSITE/product_images/870-768442/PGGJ5RaWoYx7VStbsBjL.JPG
WEBSITE/product_images/870-768442/SEEbiNEWA5MXsIUwuaZH.JPG
WEBSITE/product_images/870-768442/TrEPAFaEhxhMxQNDD8qh.JPG

... ...

var StockNumber = 'U79649'
var product_id = '957033'
var product_sku = 'LS16ZBC'
        var threesixty_start_frame = 4
    var threesixty_images = WEBSITE/product_images/870-957033/K5iXEYmG2a4QncRBOrvL.JPG
WEBSITE/product_images/870-957033/fbaj3T5dKtmH0HTX11q5.JPG
WEBSITE/product_images/870-957033/WvBUOrjCMWQGe4gwNhrF.JPG
WEBSITE/product_images/870-957033/ixtB4SbtrFZTIVotvxSd.JPG

... ...

理想的输出如下:

WEBSITE/product_images/870-972930/XafJtPxTqNQuPNQpjF5h.JPG NN26ZZA_01.JPG
WEBSITE/product_images/870-972930/9YZH6VMT2zH6AntfaAIG.JPG NN26ZZA_02.JPG
WEBSITE/product_images/870-972930/SrE9oVwVmJjS58SwZB9q.JPG NN26ZZA_03.JPG
WEBSITE/product_images/870-972930/1enLX6dc9bbG7k9SEAfL.JPG NN26ZZA_04.JPG
...
...
WEBSITE/product_images/870-768442/fg7G7CX2Z0oK8mCVlJN3.JPG LT65YMP_01.JPG
WEBSITE/product_images/870-768442/PGGJ5RaWoYx7VStbsBjL.JPG LT65YMP_02.JPG
WEBSITE/product_images/870-768442/SEEbiNEWA5MXsIUwuaZH.JPG LT65YMP_03.JPG
WEBSITE/product_images/870-768442/TrEPAFaEhxhMxQNDD8qh.JPG LT65YMP_04.JPG
...
...

等等

2 个答案:

答案 0 :(得分:2)

您可以使用awk捕获上一行中看到的product_sku,然后根据需要重新使用它。例如,此处product_sku用于所有具有WEBSITE的行。请注意,每次看到product_sku时,c的计数器也会重置。这样,每个product_sku的编号都会重新开始。

awk  '/product_sku/{p_sku=$NF; c=1;next} /WEBSITE/{url=gensub(/.*(WEBSITE.*)/,"\\1","g");print url,p_sku "_"c".jpeg";c++}' input
WEBSITE/product_images/870-972930/XafJtPxTqNQuPNQpjF5h.JPG NN26ZZA_1.jpeg
WEBSITE/product_images/870-972930/9YZH6VMT2zH6AntfaAIG.JPG NN26ZZA_2.jpeg
WEBSITE/product_images/870-972930/SrE9oVwVmJjS58SwZB9q.JPG NN26ZZA_3.jpeg
WEBSITE/product_images/870-972930/1enLX6dc9bbG7k9SEAfL.JPG NN26ZZA_4.jpeg
WEBSITE/product_images/870-768442/fg7G7CX2Z0oK8mCVlJN3.JPG LT65YMP_1.jpeg
WEBSITE/product_images/870-768442/PGGJ5RaWoYx7VStbsBjL.JPG LT65YMP_2.jpeg
WEBSITE/product_images/870-768442/SEEbiNEWA5MXsIUwuaZH.JPG LT65YMP_3.jpeg
WEBSITE/product_images/870-768442/TrEPAFaEhxhMxQNDD8qh.JPG LT65YMP_4.jpeg
WEBSITE/product_images/870-957033/K5iXEYmG2a4QncRBOrvL.JPG LS16ZBC_1.jpeg
WEBSITE/product_images/870-957033/fbaj3T5dKtmH0HTX11q5.JPG LS16ZBC_2.jpeg
WEBSITE/product_images/870-957033/WvBUOrjCMWQGe4gwNhrF.JPG LS16ZBC_3.jpeg
WEBSITE/product_images/870-957033/ixtB4SbtrFZTIVotvxSd.JPG LS16ZBC_4.jpeg

注意:此解决方案需要gawk才能运行。

答案 1 :(得分:0)

$ cat tst.awk
$2 == "product_sku" {
    sku = $4
    gsub(/\047/,"",sku)
    cnt = 0
}
s = index($0,"WEBSITE/product_images/870-") {
    printf "%s %s_%02d.JPG\n", substr($0,s), sku, ++cnt
}

$ awk -f tst.awk file
WEBSITE/product_images/870-972930/XafJtPxTqNQuPNQpjF5h.JPG NN26ZZA_01.JPG
WEBSITE/product_images/870-972930/9YZH6VMT2zH6AntfaAIG.JPG NN26ZZA_02.JPG
WEBSITE/product_images/870-972930/SrE9oVwVmJjS58SwZB9q.JPG NN26ZZA_03.JPG
WEBSITE/product_images/870-972930/1enLX6dc9bbG7k9SEAfL.JPG NN26ZZA_04.JPG
WEBSITE/product_images/870-768442/fg7G7CX2Z0oK8mCVlJN3.JPG LT65YMP_01.JPG
WEBSITE/product_images/870-768442/PGGJ5RaWoYx7VStbsBjL.JPG LT65YMP_02.JPG
WEBSITE/product_images/870-768442/SEEbiNEWA5MXsIUwuaZH.JPG LT65YMP_03.JPG
WEBSITE/product_images/870-768442/TrEPAFaEhxhMxQNDD8qh.JPG LT65YMP_04.JPG
WEBSITE/product_images/870-957033/K5iXEYmG2a4QncRBOrvL.JPG LS16ZBC_01.JPG
WEBSITE/product_images/870-957033/fbaj3T5dKtmH0HTX11q5.JPG LS16ZBC_02.JPG
WEBSITE/product_images/870-957033/WvBUOrjCMWQGe4gwNhrF.JPG LS16ZBC_03.JPG
WEBSITE/product_images/870-957033/ixtB4SbtrFZTIVotvxSd.JPG LS16ZBC_04.JPG