我遇到了pool.apply_async问题。
我正在运行多个插入查询,并将它们作为带有save()函数的列表传递给我,以在其中构建插入查询。 由于使用pool.apply_async,每次运行此脚本时,我都会得到不同数量的记录。
PS:我为此使用python2.7。
rs = []
i = 0
## Fetch 1000 rows from the cursor at a time an append it to an array.
## Once all the rows are appended to the array save the values and exit.
pool_size = 30
pool = Pool(pool_size)
while True:
rows = cursor.fetchmany(1000)
i += len(rows)
#print "fetching 1000"
if (len(rows) == 0 ):
break
for re in rows:
rs.append(re)
if (len(rs) > 3000):
print "\n \nInside the len(rs) loop"
pool.apply_async(saveRows.save (rs, columns, host, port, schema, load_dt_time, startsql, saveTo, table, jobId, stage))
rs = []
pool.close()
pool.join()