使用swift python API无法上传大小超过2147483647字节的对象

时间:2017-07-24 18:14:25

标签: python ibm-cloud ibm-cloud-infrastructure openstack-swift

连接到对象存储

      user1_connection = swiftclient.client.Connection(user=sl_user,key=user_key,authurl=endpoint_url) 

将对象上传到容器

   with open(object_path, 'r') as temp:
     user2_connection.put_object(container_name, object_name, contents= temp.read(),content_type=content_type)

上传会因对象大小为3GB而引发错误。

错误:OverflowError:字符串长于2147483647字节

更新

出于测试目的,我试图在多部分中上传17mb的文件。

代码

     file_size = os.path.getsize(object_path)
                    block_size = 5242880
                    chunk_size = block_size
                    chunks = math.ceil(file_size / float(chunk_size))
                    chunks = int (chunks)
                    print("Reading in file")
                    file = open(object_path, 'rb')
                    for i in range(0, chunks):
                              data = file.read(chunk_size)
                              print("Uploading part {} of {}".format(i + 1, chunks))
                              chunk_name = "chunk-{0:0>5}".format(i)
                              user2_connection.put_object(container_name, object_name+'/'+chunk_name,
                                            contents= data,
                                            content_type=content_type)
                    headers = {
                    "X-Object-Manifest": container_name+"/"+object_name+"/",
                    }
                    user2_connection.post_object(container_name,object_name,headers=headers)

快速列表YUGOTEST1 --lh

       0 2017-07-25 04:14:29 application/octet-stream programs/ProductionObjectStorage20150828.log
    5.0M 2017-07-25 04:14:24 application/octet-stream programs/ProductionObjectStorage20150828.log/chunk-00000
    5.0M 2017-07-25 04:14:25 application/octet-stream programs/ProductionObjectStorage20150828.log/chunk-00001
    5.0M 2017-07-25 04:14:27 application/octet-stream programs/ProductionObjectStorage20150828.log/chunk-00002
    1.5M 2017-07-25 04:14:28 application/octet-stream programs/ProductionObjectStorage20150828.log/chunk-00003
     16M

swift stat YUGOTEST1 programs / ProductionObjectStorage20150828.log

           Account: AUTH_abcd7
         Container: YUGOTEST1
            Object: programs/ProductionObjectStorage20150828.log
      Content Type: application/octet-stream
    Content Length: 17348182
     Last Modified: Tue, 25 Jul 2017 04:14:30 GMT
              ETag: "afbd3bc65dbcd"
          Manifest: YUGOTEST1/programs/ProductionObjectStorage20150828.log/
     Accept-Ranges: bytes
       X-Timestamp: 1500956069.09158
        X-Trans-Id: tx0bcd

快速下载YUGOTEST1程序/ ProductionObjectStorage20150828.log

    Error downloading object 'YUGOTEST1/programs/ProductionObjectStorage20150828.log': [Errno 21] Is a directory: u'programs/ProductionObjectStorage20150828.log'

0 个答案:

没有答案