我每10秒就有大约300多个表在mysql中计算。每个表对应每个统计表
示例:
def pool_func(pool, table_name):
for name in table_name:
pool.add(gevent.spawn(BaseStat(table_name=name).run))
pool.join()
BaseStat.run :(我使用crontab进行部署,每分钟运行一次)
self.frequency = 10
def run(self):
try:
if self.table_name is None:
raise Exception("You should set BaseStat table_name")
self.stat_table_name = self.table_name + self.table_stat_suffix
start = 0
self._run_stat()
while start < 60:
start += self.frequency
gevent.sleep(self.frequency)
self.start_time, self.end_time = self._get_time(self.start_time, self.end_time)
self._run_stat()
except Exception as e:
logger.get("stat-log").error("[%s] : %s" % (self.table_name, util.error_msg()))
当表的数量很小时,它可以正常工作但现在至少延迟2分钟。总运行时间pool_func
约578.056058168秒