Question

如何在两个不同的函数中引用相同的临时目录。我需要访问在move_source_to_dest中解压缩的文件，作为pd.read_csv语句中函数df_to_csv的输入。我尝试了一些变化，但没有任何结果。请帮忙。

def move_source_to_dest(key, src_session):

    with tempfile.TemporaryDirectory() as tempdir:

        try:
            print("downloading {}/{}".format(s3_src_bucket, key))
            src_session.client('s3').download_file(Bucket=s3_src_bucket, Key=key,
                                               Filename=os.path.join(tempdir, os.path.basename(key)))

            #Command to decompress the files
            command = "bzip2 -dk " + os.path.join(tempdir, os.path.basename(key))
            subprocess.call(command,shell = True)


        except Exception as e:
            print("exception handling {}/{}".format(s3_src_bucket, key))
            raise e
def df_to_csv(key, src_session):
    with tempfile.TemporaryDirectory() as tempdir:
        try:
            #Reading all the columns names from the file "ambs_ambi_ColumnsNames.txt"
            with open('./shakenbake_ds/ambs_ambi_ColumnsNames.txt') as f:
                clist= f.read().splitlines()

                #file = open('ambs_ambi_ColumnsNames.txt','r')
                #clist=file.readlines()

            Filename=os.path.join(tempdir, os.path.basename(key[:-4]))
            Fileout=os.path.join(tempdir, os.path.basename(key[:-4])) + "-out.csv" 

            with open('./shakenbake_ds/ambs_ambi_OutColumnsNames.txt') as o:
                outcols= o.read().splitlines()
                #file = open('ambs_ambi_OutColumnsNames.txt','r')
                #outcols=file.readlines()
                #global Filename
            c=0
            for chunk in pd.read_csv(Filename, sep="\x01", names=clist ,iterator=True, chunksize=300000):

Answer 1

将临时目录作为参数传递给两个函数：

with tempfile.TemporaryDirectory() as tempdir:
    move_source_to_dest(key, src_session, tempdir)
    df_to_csv(key, src_session, tempdir)

Answer 2

在函数之间共享状态的一种常用方法是将该状态作为参数传递给这些函数。

例如，

node.default["java"]["install_flavor"] = "openjdk"
node.default["java"]["jdk_version"]    = "8"

include_recipe "java"

将with tempfile.TemporaryDirectory() as tempdir: move_source_to_dest(tempdir, ...) df_to_csv(tempdir, ...)和move_source_to_dest重新定义为：

df_to_csv

如何在两个不同的函数中引用相同的临时目录

2 个答案: