通过smart_open＆amp; amp;访问S3逐行读取文件内容

Question

我使用smart_open从S3获取数据。但是，当我逐行阅读文件时，它的投掷错误。我正在迭代桶和获取密钥（文件名）。现在我需要逐行阅读文件内容。

for key, content in smart_open.s3_iter_bucket(bucket = bucket, prefix = prefix):

            print key
            with smart_open.smart_open(bucket+key) as fin:
                for line in fin:
                    print line

抛出Bad Request错误。 Pl建议

Answer 1

我相信您的存储桶名称和键是独立的字符串，而传递给smart_open的参数是格式化的URI，必须包含S3前缀。所以...我会尝试：

for key, content in smart_open.s3_iter_bucket(bucket = bucket, prefix = prefix):

        print key
        s3_uri = 's3://' + bucket + '/' + key
        with smart_open.smart_open(s3_uri) as fin:
            for line in fin:
                print line