我是GCP的新手,有使用Python的经验。 我尝试为场景编写一个Cloud Function,以解压缩GCS中的文件并将其复制到另一个存储桶。
from google.cloud import storage
import tarfile
client = storage.Client()
def untar_lookupfiles(data, context):
# Get the file that has been uploaded to GCS
bucket = client.get_bucket(data['Source_bucketName'])
#copy the tarfiles to another bucket
bucket = client.get_bucket('Target_bucketName')
blob = bucket.blob('gs://path/to/file.name')
blob.upload_from_filename('/path/to/source.file')
# Untar the files
print('Untaring Files: {}'.format(data['name']))
untar = tarfile.open("marfiles.tar.gz", "r:gz") # filename is hard coded should be replaced with data['name']
untar.extractall(path=dir)
但是该代码似乎缺少某些内容,有人可以帮助我使用该代码。我没有使用nodejs编写代码的经验。感谢您的帮助。
答案 0 :(得分:2)
这是一个功能,它将解压缩放置在一个存储桶中的文件并将内容存储在另一个存储桶中:
在requirements.txt
中:
google-cloud-storage
在main.py
中:
import io
import os
import tarfile
from google.cloud import storage
client = storage.Client()
input_bucket = client.get_bucket('INPUT-BUCKET-NAME')
output_bucket = client.get_bucket('OUTPUT-BUCKET-NAME')
def untar(data, context):
# Get the contents of the uploaded file
input_blob = input_bucket.get_blob(data['name']).download_as_string()
# Turn the upload file into a tar file
tar = tarfile.open(fileobj=io.BytesIO(input_blob))
# Iterate over all files in the tar file
for member in tar.getnames():
# Extract the individual file
file_object = tar.extractfile(member)
# Check if it's a file or directory (which should be skipped)
if file_object:
# Create a new blob instance in the output bucket
output_blob = output_bucket.blob(os.path.join(data['name'], member))
# Write the contents of the file to the output blob
output_blob.upload_from_string(file_object.read())
要部署:
$ gcloud beta functions deploy test \
--runtime python37 \
--project PROJECT_NAME \
--trigger-resource INPUT_BUCKET_NAME \
--trigger-event google.storage.object.finalize