Question

我通过多个线程下载文件，每个线程都是按流的。

def download_file(self, dest:str, filename:str):

    r = requests.get(self.url, stream=True, headers={
        "Accept": "application/xml",
        "Accept-Charset": "utf-8",
        "Accept-Language": "pt-BR,pt;q=0.8,en-US;q=0.6,en;q=0.4",
        "Authorization": self.auth
    })
    with open(f"{dest}/{filename}", 'wb') as f:
        total_length = r.headers.get("Content-length")
        if not total_length:
            f.write(r.content)
        else:
            for chunk in progress.bar(r.iter_content(chunk_size=1024), expected_size=(int(total_length)/1024) + 1):
                if chunk:  # filter out keep-alive new chunks
                    f.write(chunk)
                    f.flush() 
        f.close()
    return True

此后，我将这些下载的文件上传到S3存储桶。

def process_path_for_S3(local_file_path):
    if not parsed_args.ignore_s3:
        s3_repository = parsed_args.s3
        logging.info(f"Upando o arquivo '{local_file_path['filename']}' para o S3...")

        s3_key = f"{s3_repository.split('/')[1]}/{s3_repository.split('/')[2]}/{local_file_path['filename']}"
        for i in range(0, parsed_args.tries):
            try:
                _s3.s3.meta.client.upload_file(Bucket=s3_repository.split('/')[0], Key=s3_key, Filename=local_file_path["local_path"])
            except Exception as e:
                logging.info(">>> ERRO <<<")
                logging.info(e)
                logging.info(">>> NÃO FOI POSSÍVEL REALIZAR O UPLOAD DO ARQUIVO PARA O S3 <<<")
                logging.info("Tentando novamente...")
                sleep(5)
                continue
            break
        logging.info(f">>> ARQUIVO UPADO COM SUCESSO PARA O S3 <<<")
    else:
        logging.info(">>> IGNORANDO PROCESSO DE UPLOAD NO S3 <<<")

logging.info(f">>> PROCESSANDO UPLOAD PARA O S3 EM {parsed_args.s3_threads} THREADS <<<")
s3_multiprocessor._map(process_path_for_S3, local_file_paths) # Multithreading code.

问题是，当我尝试打开文件以将其上传到S3时，出现权限错误。

[Errno 13] Permission denied: 'C:/Users/User/AppData/Roaming/...'

这不可能是由于多线程（保存一个线程并试图同时打开另一个线程）引起的，因为我已经尝试禁用多线程，并且遇到了相同的错误。我能想到的唯一原因是“按流下载”。我的同事建议我检查文件中文件的结尾，但是如果我不能打开，该怎么办？

下载文件后如何避免权限被拒绝？

0 个答案: