Python读取文件utf-8使用了大量内存

时间:2017-05-04 12:44:45

标签: python encoding utf-8

当我添加utf-8编码时,它使用大约3000mb的内存(并冻结我的电脑) 下面的例子很好,

pool = ThreadPoolExecutor(max_workers=10)
with open("data.txt") as f:
        for line in f:
            pool.submit(thread, line)
pool.shutdown(wait=True)

但是当我添加utf-8编码时,它基本上会崩溃我的电脑

pool = ThreadPoolExecutor(max_workers=10)
with open("data.txt", encoding='utf-8') as f:
        for line in f:
            pool.submit(thread, line)
pool.shutdown(wait=True)

0 个答案:

没有答案