Question

我正在尝试从Google云存储文件夹下载多个文件。我可以下载单个文件，但无法下载多个文件。我从this link处获得了此参考，但似乎没有用。代码如下：

# [download multiple files]
bucket_name = 'bigquery-hive-load'
# The "folder" where the files you want to download are
folder="/projects/bigquery/download/shakespeare/"

# Create this folder locally
if not os.path.exists(folder):
    os.makedirs(folder)

# Retrieve all blobs with a prefix matching the folder
    bucket=storage_client.get_bucket(bucket_name)
    print(bucket)
    blobs=list(bucket.list_blobs(prefix=folder))
    print(blobs)
    for blob in blobs:
        if(not blob.name.endswith("/")):
            blob.download_to_filename(blob.name)

# [End download to multiple files]

是否可以下载与模式（名称）或其他内容匹配的多个文件。由于我是从bigquery导出文件，因此文件名将如下所示：

shakespeare-000000000000.csv.gz
shakespeare-000000000001.csv.gz
shakespeare-000000000002.csv.gz
shakespeare-000000000003.csv.gz

参考：下载单个文件的工作代码：

# [download to single files]

edgenode_destination_uri = '/projects/bigquery/download/shakespeare-000000000000.csv.gz'
bucket_name = 'bigquery-hive-load'
gcs_bucket = storage_client.get_bucket(bucket_name)
blob = gcs_bucket.blob("shakespeare.csv.gz")
blob.download_to_filename(edgenode_destination_uri)
logging.info('Downloded {} to {}'.format(
    gcs_bucket, edgenode_destination_uri))

# [end download to single files]

Answer 1

经过一番尝试，我解决了这个问题，也无法阻止自己在这里发布。

bucket_name = 'mybucket'
folder='/projects/bigquery/download/shakespeare/'
delimiter='/'
file = 'shakespeare'

# Retrieve all blobs with a prefix matching the file.
bucket=storage_client.get_bucket(bucket_name)
# List blobs iterate in folder 
blobs=bucket.list_blobs(prefix=file, delimiter=delimiter) # Excluding folder inside bucket
for blob in blobs:
   print(blob.name)
   destination_uri = '{}/{}'.format(folder, blob.name) 
   blob.download_to_filename(destination_uri)

Answer 2

看来您在python代码中可能只是缩进了错误的级别。以if开头的块在上述# [download multiple files] bucket_name = 'bigquery-hive-load' # The "folder" where the files you want to download are folder="/projects/bigquery/download/shakespeare/" # Create this folder locally if not os.path.exists(folder): os.makedirs(folder) # Retrieve all blobs with a prefix matching the folder bucket=storage_client.get_bucket(bucket_name) print(bucket) blobs=list(bucket.list_blobs(prefix=folder)) print(blobs) for blob in blobs: if(not blob.name.endswith("/")): blob.download_to_filename(blob.name) # [End download to multiple files]的范围内，因此如果文件夹已经存在，则永远不会执行。

尝试一下：

                      **Changing the port number on Windows**

使用Python从Google云存储下载多个文件

2 个答案: