线程池:
BlockingQueue<Runnable> workQueue = new LinkedBlockingDeque<>(5000);
private ThreadPoolExecutor taskExecutor = new ThreadPoolExecutor(1, 5, 30, TimeUnit.SECONDS, workQueue);
任务插入:
我正在读取数据库中的n条记录,将其拆分为块并将块开始 - 否和结束 - 否为工人。
do {
try {
taskExecutor.execute(new EmailWorker(lowerOffset, upperOffset));
logger.trace("Added work: seq:"+lowerOffset+" to seq no:"+upperOffset+" to executor pool");
lowerOffset = lowerOffset + (batchSize-1) + 1;
upperOffset = lowerOffset + (batchSize-1);
Thread.sleep(500);
} catch(RejectedExecutionException ree) {
logger.trace("Too many work items. Pausing for a while, let Email workers catch up."+ree.getMessage());
try {
Thread.sleep(10000);
} catch (InterruptedException ex) {
}
} catch (CriticalException ex) {
throw ex;
} catch (InterruptedException ex) {
logger.error("", ex);
} catch (Exception ex) {
logger.error("",ex);
}
} while(upperOffset <= maxSeqNo);
在worker中,我在run方法中有以下结构。由于我在开始时尝试阻止,因此无论如何都应该捕获其中发生的任何错误。
public void run() {
logger.trace("email worker work started");
ResultSet rs = null;
try {
//
} catch (SQLException | ClassNotFoundException sq) {
kafkaLogger.error(sq.getClass() + " :: " + sq.getMessage());
logger.error(sq.getMessage());
} catch (Exception sq) {
kafkaLogger.error(sq.getClass() + " :: " + sq.getMessage());
logger.error(sq.getMessage());
}
}
这是ThreadPool的输出
[2018-01-15 19:03:46] :: TRACE :: EmailSupervizor:156 - [Running, pool size = 2, active threads = 1, queued tasks = 1, completed tasks = 0]
[2018-01-15 19:03:51] :: TRACE :: EmailSupervizor:157 - [Running, pool size = 1, active threads = 1, queued tasks = 2, completed tasks = 0]
[2018-01-15 19:03:56] :: TRACE :: EmailSupervizor:160 - [Running, pool size = 1, active threads = 1, queued tasks = 3, completed tasks = 0]
[2018-01-15 19:03:61] :: TRACE :: EmailSupervizor:156 - [Running, pool size = 2, active threads = 1, queued tasks = 4, completed tasks = 0]