如何通过shell脚本发送电子邮件给distcp失败?

时间:2019-08-20 20:32:14

标签: bash hadoop

我在FOR循环中使用distcp命令将数据复制到Azure Blob存储上的多个容器。有时,特定的副本会失败。我希望在此类故障发生时获得电子邮件警报。如何获取Discp的故障日志并将其邮寄给我?

cd /var/lib/hadoop-hdfs/azcopy_linux_amd64_10.2.1/
hdfs dfs -ls hdfs://nameservice1:8020/user/securonix/snypr/resources-enriched/ | awk 'NR > 1 { print $8 }' | awk -F/ '{print "resourcegroupidpq" $8}' > parq.txt
for i in $(cat parq.txt)
   do
      ./azcopy make "https://parquet.blob.core.windows.net/$i?sv=2018-03-28&ss=bfqt&srt=sco&sp=rwdlacup&se=2919-06-29T12:00:04Z&st=2019-06-29T04:00:04Z&spr=htt"
   done

hdfs dfs -ls hdfs://nameservice1:8020/user/securonix/snypr/resources-enriched/ | awk 'NR > 1 { print $8 }' | awk -F/ '{print "resourcegroupidpq=" $8}' | awk -F= '{print $2}' > parq1.txt
for j in $(cat parq1.txt)
   do
      hadoop distcp -m 250 -update hdfs://nameservice1:8020/user/securonix/snypr/resources-enriched/$j/resourcegroupidpq=$j/typepq=enriched/ wasb://resourcegroupidpq$j@parquet.blob.core.windows.net/typepq=enriched/
   done

0 个答案:

没有答案