无法使用python / boto连接到Amazon S3 - 代理连接问题

时间:2015-07-30 19:07:08

标签: python amazon-s3 proxy boto

我有一个应该连接到S3的脚本,找到一个特定的存储桶,然后为该存储桶中的每个新密钥获取get_contents_to_filename

conn = S3Connection(ACCESS_KEY_ID, SECRET_ACCESS_KEY, debug=1,calling_format=ProtocolIndependentOrdinaryCallingFormat())
bucket = Bucket(conn, 'bucket_name_here')

这是获取存储桶中密钥的循环

        # Loop through each key for each hour of the day
        for idx, key in enumerate(bucket.list(PATH)):

            logger2.info('Key: %s' % str(key))

            # Create new filename for output - appid_yyyy_mm_dd_hh

            file_name = app_id + dt.strftime("%Y_%m_%d") + "_" + str(idx) + '.log.gz'
            current_file = dateutil.parser.parse(key.last_modified[:19])

            # print current time and last observer time
            logger3.info('Current File Datetime:'+ ' ' + str(current_file) + '       ' +
                             'Last Modified Datetime:' + ' ' + str(max_file_datetime))

            if current_file <= max_file_datetime:
                    logger.info('skipping file: ' + str(key.name))
            else:
                try:
                    logger.info('downloading file' + str(key.name))
                    key.get_contents_to_filename(outlocation + file_name,cb=mycb,num_cb=5)
                except Exception, e:
                    logger.error('Unable to download Key contents to file: %s' %str(e))

我从我公司的代理服务器后面运行此代码 - 目前使用cntlm尝试绕过它但由于连接问题我在get_contents_to_filename上一直收到错误

这是我得到的错误:

socket.error:[Errno 10060]连接尝试失败,因为连接方没有正确响应 由于连接的主机未能响应,因此时间点或建立的连接失败

当我打开调试时,这是输出:

     2015-07-30 19:13:52,427 foo [DEBUG]:Using access key provided by 

client.
2015-07-30 19:13:52,427 foo [DEBUG]:Using secret key provided by client.
2015-07-30 19:13:52,489 foo [DEBUG]:path=/------string------/
2015-07-30 19:13:52,489 foo [DEBUG]:auth_path=/------string------/
2015-07-30 19:13:52,489 foo [DEBUG]:path=/------string------/?prefix-------------------------
2015-07-30 19:13:52,489 foo [DEBUG]:auth_path=/------string------/?prefix-------------------------
2015-07-30 19:13:52,489 foo [DEBUG]:Method: GET
2015-07-30 19:13:52,489 foo [DEBUG]:Path: /------string------/?prefix-------------------------
2015-07-30 19:13:52,489 foo [DEBUG]:Data:
2015-07-30 19:13:52,489 foo [DEBUG]:Headers: {}
2015-07-30 19:13:52,489 foo [DEBUG]:Host: s3.amazonaws.com
2015-07-30 19:13:52,489 foo [DEBUG]:Port: 443
2015-07-30 19:13:52,489 foo [DEBUG]:Params: {}
2015-07-30 19:13:52,489 foo [DEBUG]:establishing HTTPS connection: host=s3.amazonaws.com, kwargs={'port': 443, 'timeout'
: 70}
2015-07-30 19:13:52,941 foo [DEBUG]:Token: None
2015-07-30 19:13:52,941 foo [DEBUG]:StringToSign:
GET


Thu, 30 Jul 2015 18:13:52 GMT
/exports.localytics/
2015-07-30 19:13:52,941 foo [DEBUG]:Signature:
AWS ---------------string---------------=
2015-07-30 19:13:52,941 foo [DEBUG]:Final headers: {'Date': 'Thu, 30 Jul 2015 18:13:52 GMT', 'Content-Length': '0', 'Aut
horization': u'-----string_here----------', 'User-Agent': 'Boto/2.38.0 Python/2.7.10 Windows/
8'}
2015-07-30 19:14:13,999 foo [DEBUG]:encountered error exception, reconnecting
2015-07-30 19:14:13,999 foo [DEBUG]:establishing HTTPS connection: host=s3.amazonaws.com, kwargs={'port': 443, 'timeout'
: 70}
2015-07-30 19:14:14,171 foo [DEBUG]:Token: None
2015-07-30 19:14:14,171 foo [DEBUG]:StringToSign:
GET

调试输出的最后一部分在连接重试时重复几次,然后我得到socket.error。我对boto / s3的所有内容都很陌生,并且对python很新,所以对任何不良做法表示歉意。

是代理问题还是其他问题?

0 个答案:

没有答案