如何正确上传和获取blobs google cloud python客户端库多线程?

时间:2017-06-06 07:17:31

标签: python google-cloud-datastore google-cloud-storage

我正在尝试从服务器下载mp3文件,然后使用以下代码上传到Google存储空间

from google.cloud import storage
bucket = storage_client.get_bucket('bucketname')
blob = bucket.blob(uploadKey)
blob.upload_from_filename(flacFileName)

但需要注意的是,下载和上传是多线程的,在某些线程中我得到以下异常

Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
    self.run()
  File "/usr/lib/python2.7/threading.py", line 754, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/home/tamim/upworkProjects/speech_recong/upload_to_storage.py", line 51, in downloadMp3AndUploadToBucket
    blob.upload_from_filename(flacFileName)
  File "/home/tamim/.local/lib/python2.7/site-packages/google/cloud/storage/blob.py", line 719, in upload_from_filename
    file_obj, content_type=content_type, client=client)
  File "/home/tamim/.local/lib/python2.7/site-packages/google/cloud/storage/blob.py", line 673, in upload_from_file
    connection.http, request, retries=num_retries)
  File "/home/tamim/.local/lib/python2.7/site-packages/google/cloud/streaming/http_wrapper.py", line 384, in make_api_request
    redirections=redirections)
  File "/home/tamim/.local/lib/python2.7/site-packages/google/cloud/streaming/http_wrapper.py", line 347, in _make_api_request_no_retry
    redirections=redirections, connection_type=connection_type)
  File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
    self.gen.next()
  File "/home/tamim/.local/lib/python2.7/site-packages/google/cloud/streaming/http_wrapper.py", line 107, in _httplib2_debug_level
    http.connections[connection_key].set_debuglevel(old_level)
KeyError: 'https:accounts.google.com'

我也得到[SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:2178)例外。但是这些例外是不稳定的,如果我不使用多个线程,我也没有这些问题。那么,如何正确地将文件上传到google存储多线程?

我查看了此链接https://github.com/GoogleCloudPlatform/google-cloud-python/issues/1214和此处https://developers.google.com/api-client-library/python/guide/thread_safety,但我无法找到如何使用google-cloud-storage客户端库进行操作的答案。

0 个答案:

没有答案