我有s3个存储桶对象,如下所示:
my_bucket/inside/another/2018-10-09/hello.txt
my_bucket/inside/another/2018-10-09/hello.json
my_bucket/inside/another/2018-10-08/hello.java
my_bucket/inside/another/2018-10-08/hello.json
my_bucket/inside/another/2018-10-10/hello.json
我正在尝试加载本地目录,例如如下所示:
new/2018-10-09/hello.json
new/2018-10-08/hello.json
new/2018-10-10/hello.json
熟悉提取文件的命令,但是我想在写入本地文件时修改目录结构:
aws s3api list-objects --bucket "my_bucket" --output text --query "Contents[?(LastModified>='2018-10-07' && contains(Key,'.json'))].{Key: Key}" | xargs -I {} aws s3 cp my_bucket/{} new/{}
我正在寻找一种用单行方式替换目录结构而不用在调用s3 copy命令时修改值的方式。例如也许像〜
aws s3api list-objects --bucket "my_bucket" --output text --query "Contents[?(LastModified>='2018-10-07' && contains(Key,'.json'))].{Key: Key}" | xargs -I {} aws s3 cp my_bucket/{} new/`echo {} | sed 's/inside\/another\//my_directory/g'`
尽管这更是一个UNIX查询,但我愿意接受一些建议,如果有可用的选项,这些建议将允许我通过s3api处理。
P.S。出于测试目的,还可以考虑以下内容:
where list.txt contains:
my_bucket/inside/another/2018-10-09/hello.txt
my_bucket/inside/another/2018-10-09/hello.json
my_bucket/inside/another/2018-10-08/hello.java
my_bucket/inside/another/2018-10-08/hello.json
my_bucket/inside/another/2018-10-10/hello.json
命令:
cat list.txt | xargs -I {} aws s3 cp s3://my_bucket/{} new/`echo {} | sed 's/inside\/another\//my_directory/g'`