with gzip.open(sys.argv[1] + ".json.gz", mode="w") as outfile:
s = json.dumps(data, indent=2, ensure_ascii=False)
print "JSON.DUMP", round(time.time() - start_time_2), "s"
s = s.encode('utf-8')
print "Encode.utf8", round(time.time() - start_time_2), "s"
outfile.write(s)
解析文件完成32.0 s
JSON.DUMP 64.0 s
Encode.utf8 83.0 s
写600s
s
大约800 MB。
写入文件后压缩为100Mb。 有没有办法让写入+压缩更快,因为它是瓶颈。