Question

在接下来的页面

https://googlecloudplatform.github.io/google-cloud-python/latest/storage/blobs.html

所有API调用都可以用于Python＆amp; Google云端存储。甚至在github上的“官方”样本

https://github.com/GoogleCloudPlatform/python-docs-samples/blob/master/storage/cloud-client/snippets.py

没有相关的例子。

最后，使用与下载文件相同的方法下载目录会产生错误

Error:  [Errno 21] Is a directory:

Answer 1

您只需首先列出目录中的所有文件，然后逐个下载：

bucket_name = 'your-bucket-name'
prefix = 'your-bucket-directory/'
dl_dir = 'your-local-directory/'

storage_client = storage.Client()
bucket = storage_client.get_bucket(bucket_name=bucket_name)
blobs = bucket.list_blobs(prefix=prefix)  # Get list of files
for blob in blobs:
    filename = blob.name.replace('/', '_') 
    blob.download_to_filename(dl_dir + filename)  # Download

blob.name包含整个目录结构+文件名，因此如果您想要与存储桶中的文件名相同，则可能需要先将其解压缩（而不是将/替换为_ }）

Answer 2

让我们说，我们想从以下存储路径下载FINALFOLDER： gs://TEST_BUCKET_NAME/FOLDER1/FOLDER2/FINALFOLDER 下载后，最终路径如下： D:\\my_blob_data\FINALFOLDER

from os import makedirs
from os.path import join, isdir, isfile, basename
from google.cloud import storage

# if your environment was authenticated, the default config will be picked up
storage_client = storage.Client() # comment this line if you want to use service account

# uncomment the line below if you have a service account json
# storage_client = storage.Client.from_service_account_json('creds/sa.json')

bucket_name = 'TEST_BUCKET_NAME'
prefix = 'FOLDER2'
dst_path = 'D:\\my_blob_data'

if isdir(dstPath) == False:
    makedirs(dstPath)

bucket = storage_client.bucket(bucket_name=bucket_name)
blobs = bucket.list_blobs(prefix=prefix)  # Get list of files
for blob in blobs:
    blob_name = blob.name 
    dst_file_name = blob_name.replace('FOLDER1/FOLDER2', dst_path) #.replace('FOLDER1/FOLDER2', 'D:\\my_blob_data') 
    # extract the final directory and create it in the destination path if it does not exist
    dst_dir = dst_file_name.replace('/' + basename(dst_file_name), '')
    if isdir(dst_dir) == False:
        makedirs(dst_dir)
    # download the blob object
    blob.download_to_filename(dst_file_name)

Answer 3

如果您想保持相同的目录结构而不重命名，并且还要创建嵌套文件夹。我有针对python 3.5+的基于@ksbg答案的解决方案：

from pathlib import Path
bucket_name = 'your-bucket-name'
prefix = 'your-bucket-directory/'
dl_dir = 'your-local-directory/'

storage_client = storage.Client()
bucket = storage_client.get_bucket(bucket_name=bucket_name)
blobs = bucket.list_blobs(prefix=prefix)  # Get list of files
for blob in blobs:
    if blob.name.endswith("/"):
        continue
    file_split = blob.name.split("/")
    directory = "/".join(file_split[0:-1])
    Path(directory).mkdir(parents=True, exist_ok=True)
    blob.download_to_filename(blop.name)

Answer 4

引用此链接-https://medium.com/@sandeepsinh/multiple-file-download-form-google-cloud-storage-using-python-and-gcs-api-1dbcab23c44

1-添加您的凭据Json 2-列出存储桶项目 3-下载

import logging
import os
from google.cloud import storage
global table_id
global bucket_name
logging.basicConfig(format=’%(levelname)s:%(message)s’, level=logging.DEBUG) 
bucket_name = ‘mybucket’
table_id = ‘shakespeare’
storage_client = storage.Client.from_service_account_json(‘/google-cloud/keyfile/service_account.json’)
# The “folder” where the files you want to download are
folder=’/google-cloud/download/{}’.format(table_id)
delimiter=’/’
bucket=storage_client.get_bucket(bucket_name)
blobs=bucket.list_blobs(prefix=table_id, delimiter=delimiter) #List all objects that satisfy the filter.
# Download the file to a destination 
def download_to_local():
 logging.info(‘File download Started…. Wait for the job to complete.’)
 # Create this folder locally if not exists
 if not os.path.exists(folder):
 os.makedirs(folder)
 # Iterating through for loop one by one using API call
 for blob in blobs:
 logging.info(‘Blobs: {}’.format(blob.name))
 destination_uri = ‘{}/{}’.format(folder, blob.name) 
 blob.download_to_filename(destination_uri)
 logging.info(‘Exported {} to {}’.format(
 blob.name, destination_uri))
if __name__ == ‘__main__’:
 download_to_local()

Python - 从Google云端存储下载整个目录

4 个答案: