使用boto从S3 bitbucket递归下载特定文件夹及其所有文件

时间:2017-03-13 21:08:27

标签: python amazon-web-services amazon-s3 boto boto3

我知道如何使用aws cli

递归下载文件
aws s3 cp {s3Directory} {localDirectory} --recursive

但我想使用Python Boto,下面是我的代码。我想从桶'news-live'下载文件夹及其所有文件('Daily / 2017/02/15 /')。当我执行以下命令时,所有文件夹都已创建,但没有下载文件。

def main():
DOWNLOAD_LOCATION_PATH = "/Users/test/"
BUCKET_NAME = 'news-live'
conn  = boto.connect_s3(AWS_ACCESS_KEY_ID, AWS_ACCESS_SECRET_KEY)
bucket = conn.get_bucket(BUCKET_NAME)

#goto through the list of files
bucket_list = bucket.list('Daily/2017/02/15/')

for l in bucket_list:
    key_string = str(l.key)
    print("bucket key", l.key)
    s3_path = DOWNLOAD_LOCATION_PATH + key_string
    try:
        print ("Current File is ", s3_path)
        l.get_contents_to_filename(s3_path)
    except (OSError,S3ResponseError) as e:
        pass
        # check if the file has been downloaded locally  
        if not os.path.exists(s3_path):
            try:
                os.makedirs(s3_path)
            except OSError as exc:
                # let guard againts race conditions
                import errno
                if exc.errno != errno.EEXIST:
                    raise

我不知道如何开展这项工作。

0 个答案:

没有答案