使用PowerShell脚本将Amazon S3存储桶内容复制到本地文件夹

时间:2016-11-17 12:54:31

标签: xml powershell amazon-s3 copy bucket

我找到了一个PowerShell脚本,可将所有文件从Amazon S3存储桶复制到我们的现场服务器上的本地文件夹中:

# Your account access key - must have read access to your S3 Bucket
$accessKey = "MyAccessKey"
# Your account secret access key
$secretKey = "MySecretKey"
# The region associated with your bucket e.g. eu-west-1, us-east-1 etc. (see http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-regions-availability-zones.html#concepts-regions)
$region = "eu-west-1"
# The name of your S3 Bucket
$bucket = "MyBucket/"
# The folder in your bucket to copy, including trailing slash. Leave blank to copy the entire bucket
$keyPrefix = ""
# The local file path where files should be copied
$localPath = "C:\test\"   

$objects = Get-S3Object -BucketName $bucket -KeyPrefix $keyPrefix -AccessKey $accessKey -SecretKey $secretKey -Region $region

foreach($object in $objects) {
$localFileName = $object.Key -replace $keyPrefix, ''
if ($localFileName -ne '') {
    $localFilePath = Join-Path $localPath $localFileName
    Copy-S3Object -BucketName $bucket -Key $object.Key -LocalFile $localFilePath -AccessKey $accessKey -SecretKey $secretKey -Region $region
}
}

如果我自己运行第一部分(直到并包括Get-S3Object行),然后显示 $ objects 变量,它会一个接一个地显示文件的详细信息以下PowerShell中的信息

ETag : "38763873d83763c3876"
BucketName : MyBucketName
Key : 3287653_32876_to_38763_3876_client.xml
LastModified : 14/10/2016 11:26:51
Owner : Amazon.S3.Model.Owner
Size : 485
StorageClass : Standard

所以它成功地从桶中获取文件...... 但是,如果我运行整个脚本,每次循环迭代都会出现错误,其中包含:

Copy-S3Object : The specified key does not exist.
At line:5 char:9
+         Copy-S3Object -BucketName $bucket -Key $object.Key -LocalFile ...
+         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo          : InvalidOperation: (Amazon.PowerShe...yS3ObjectCmdlet:CopyS3ObjectCmdlet) [Copy-S3Object]
, InvalidOperationException
+ FullyQualifiedErrorId : Amazon.S3.AmazonS3Exception,Amazon.PowerShell.Cmdlets.S3.CopyS3ObjectCmdlet

因此我怀疑它与循环中的命令或变量有关...

foreach($object in $objects) {
$localFileName = $object.Key -replace $keyPrefix, ''
if ($localFileName -ne '') {
    $localFilePath = Join-Path $localPath $localFileName
    Copy-S3Object -BucketName $bucket -Key $object.Key -LocalFile $localFilePath -AccessKey $accessKey -SecretKey $secretKey -Region $region
}
}

......但是还没弄清楚是什么。

有人对我做错了有什么好主意吗?

2 个答案:

答案 0 :(得分:0)

您可以在Powershell中使用S3的AWSCLI命令,它们应该已经安装在AWS Powershell Cmdlet附带。

这也是一个更容易理解的命令:

aws s3 cp s3://bucketname C:\folder --recursive

答案 1 :(得分:0)

问题是,您尝试在不是对象的事物上使用Copy-S3Object。你认为S3中的文件夹实际上不是文件夹,只是对象名称的前缀。

我已成功使用如下构造: Get-S3Object -BucketName $bucket -KeyPrefix $key | Copy-S3Object -LocalFolder $localPath