Question

我尝试使用boto3 for python将.zip文件上传到S3但我的目录中的.zip文件未正确上传。代码下载给定用户的所有电子邮件，将它们压缩到同一目录中并将其上载到S3存储桶。问题是上传的文件不是我打算上传的文件。而是只出现一个18kb的文件。

以下是代码：

import sys
import imaplib
import getpass
import email
import shutil
import boto3
import os

username = input("Enter user's first name: ")
surname = input("Enter user's surname: ")
email_address = username + "." + surname + "@gmail.com"
password = getpass.getpass()
directory = username + surname + '/'
def download_emails(server):
    result, data = server.uid('search', None, "ALL")    #search all email and return their uids
    if result == 'OK':
        for num in data[0].split():
            result, data = server.uid('fetch', num, '(RFC822)')    #RFC is a standard for the format of ARPA Internet text messages
            if result == 'OK':
                email_message = email.message_from_bytes(data[0][1])    #raw email text including headers
                file_name = email_message['Subject']       #use dates and file names(can be changed)
                if not os.path.exists(directory):
                    os.makedirs(directory)      #create a dir for user's emails
                try:
                    email_file = open(directory + file_name+'.eml', 'wb')   #open a file for each email and insert the data.
                    email_file.write(data[0][1])
                    email_file.close()
                except:
                    pass

#function to zip all the emails
def archive(zipname, directory):
    return shutil.make_archive(zipname, 'zip', root_dir=directory, base_dir=None)

#function to upload zipped emails to AWS bucket
def upload_to_s3(file_name):
    s3 = boto3.resource('s3',
                aws_access_key_id=accessKey,
                aws_secret_access_key=secretKey,
                aws_session_token=secretToken,
                )

    s3.Bucket('user-backups').put_object(Key=username.title() + " " +
                                surname.title() + "/" + file_name, Body=file_name)
    print("Uploaded")


def main():
    server = imaplib.IMAP4_SSL("imap.gmail.com", 993)   #connect to gmail's imap server
    server.login(email_address, password)   #enter creds
    result, data = server.select('"[Gmail]/All Mail"')  #get all emails(inbox, outbox etc)
    if result == 'OK':
        print("Downloading")
        download_emails(server)
        server.close()
    else:
        print("ERROR: Unable to open mailbox ", result)
    server.logout()
    archive(username + surname, directory)
    upload_to_s3(username + surname + ".zip")
    #os.remove(email_address + ".zip")
    #shutil.rmtree(email_address)
    print("Done")
if __name__ == "__main__":
    main()

Answer 1

put_object函数接受 Body ，它是字节对象或文件对象。您当前刚刚传递了纯文件名（字符串）。

来自文档：

正文（字节或可搜索文件类对象） - 对象数据。

所以修复应该是传递文件对象。请咨询this以了解如何操作。

Answer 2

只需使用s3.client.upload_file。

upload_file（文件名，Bucket，Key，ExtraArgs =无，回调=无，配置=无）

def upload_to_s3(file_name):
  s3 = boto3.client('s3')
  Key = username.title() + " " + surname.title() + "/" + file_name
  try: 
     s3.meta.client.upload_file('/path/to/file', 'user-backups', Key)
  except Exception as e:
     print(e)

Answer 3

有多种上传方式。签出这个boto3 document，这里复制了以下方法：

The managed upload methods are exposed in both the client and resource interfaces of boto3:

S3.Client method to upload a file by name: S3.Client.upload_file()
S3.Client method to upload a readable file-like object: S3.Client.upload_fileobj()
S3.Bucket method to upload a file by name: S3.Bucket.upload_file()
S3.Bucket method to upload a readable file-like object: S3.Bucket.upload_fileobj()
S3.Object method to upload a file by name: S3.Object.upload_file()
S3.Object method to upload a readable file-like object: S3.Object.upload_fileobj()

我使用s3.client.upload_file使它工作。

upload_file（文件名，存储桶，密钥，ExtraArgs = None，Callback = None，   Config = None）。
  将文件上传到S3对象。

import boto3
s3Resource = boto3.resource('s3')

try: 
    s3Resource.meta.client.upload_file('/path/to/file', 'bucketName', 'keyName')
except Exception as err:
    print(err)

您可以使用ExtraArgs参数设置ACL，元数据，内容编码等。

例如：

import boto3
s3Resource = boto3.resource('s3')

try: 
    s3Resource.meta.client.upload_file(
        '/path/to/file', 
        'bucketName',
        'keyName', 
        ExtraArgs={'ACL': 'public-read'})
except Exception as err:
    print(err)

此boto3 doc上列出了所有有效的额外参数。我将它们复制到下面进行记录。

ALLOWED_UPLOAD_ARGS = ['ACL', 'CacheControl', 'ContentDisposition',   
'ContentEncoding', 'ContentLanguage', 'ContentType', 'Expires', 
'GrantFullControl',   'GrantRead', 'GrantReadACP', 'GrantWriteACP', 'Metadata', 
'RequestPayer',   'ServerSideEncryption', 'StorageClass', 
'SSECustomerAlgorithm', 'SSECustomerKey',    
'SSECustomerKeyMD5', 'SSEKMSKeyId', 'WebsiteRedirectLocation']

您可以查看this article以获得更多信息。

Answer 4

以上答案均无效！
以下代码对我有用..

 import os 
 def upload_file_zip(s3_folder,local_file_path):
   s3_client = boto3.client('s3')
   s3_path = os.path.join(s3_folder, os.path.basename(local_file_path))
   with open(local_file_path,mode='rb') as data:
      s3_client.upload_fileobj(data, BUCKET_NAME, s3_path)

Boto3没有上传zip文件到S3 python

4 个答案: