我有一个Connector类,它建立连接并将任务委托给两个子任务 - JobManager和DataRetriever。我使用JobManager的观察者模式作为Observable。这会向Connector类提交一个条目对。
典型的连接器类如下:
class Connector implements Observable, Closeable
{
....
private void submitandMonitor(List<Callable<String>> bulkTasks, List<Callable<String>> soapTasks)
throws InterruptedException
{
// Bulk job submission
bulkExecutor = Executors.newFixedThreadPool(NBULKTHREADS,
new ThreadFactoryBuilder().setNameFormat("BulkDownloader-%d").build());
bulkCompletionService = new ExecutorCompletionService<String>(bulkExecutor);
bulkTasks.forEach(task -> bulkCompletionService.submit(task));
// Status poll thread configuration
statusPollExec = Executors.newScheduledThreadPool(0,
new ThreadFactoryBuilder().setNameFormat("StatusPoller").build());
statusPollExec.scheduleAtFixedRate(statusPoller, 15, 15, TimeUnit.MINUTES);
// Wait until all the bulk jobs are completed
shutdownLatch.await();
bulkExecutor.shutdown();
}
@Override
public void close() throws SQLException, ClientProtocolException, IOException,
RetriesExhaustedException
{
try
{
if (bulkExecutor != null)
{
if (!bulkExecutor.isShutdown())
bulkExecutor.shutdown();
bulkExecutor.awaitTermination(15, TimeUnit.SECONDS);
logger.debug("Bulk executor shutdown completed");
}
}
catch (InterruptedException e)
{
logger.warn("Auto shutdown duration exceeded, manually terminating Bulk executor!");
bulkExecutor.shutdownNow();
logger.warn("Manual shutdown for Bulk executor completed");
}
....// Same set of try catches for executors
}
}
职位经理包括:
class JobManager
{
// Method that does not bother about thread shutdown
private void submitJobs()
{
// Had sObjects.parallelStream() but changed to an iterative loop suspecting not responding to shutdown - Probably the offending method
for(Entry<SalesforceObject, Boolean> item : sObjects.entrySet())
{
SalesforceObject sObject = item.getKey();
Boolean queryAll = item.getValue();
try
{
// Method to submit the values for bulk requests. No loop
submitBulkJob(sObject, queryAll);
// Add to jobDetailMap <,> when job successful; used in monitorJobs()
}
catch (Exception e)
{
// Set the params and send info to observer
}
}
}
private void monitorJobs() throws InterruptedException
{
while (jobdetailMap.size() > 0)
{
for (Iterator<Entry<SalesforceObject, JobDetail>> iterator = jobdetailMap.entrySet().iterator(); iterator
.hasNext();)
{
Entry<SalesforceObject, JobDetail> entry = iterator.next();
SalesforceObject sObject = entry.getKey();
String sObjname = sObject.getsObjname();
// Check for status and send info to observer
}
Thread.sleep(Constants.sleep5000);
}
}
@Override
public String call() throws Exception
{
submitJobs();
monitorJobs();
setsObjectstatus(null);
return this.getClass().getSimpleName();
}
}
submitJobs()
是在迭代任务列表和提交时提交的。 monitorJobs()
迭代并检查任务列表的状态,并且在完成之前不会终止。
由于close已经关闭并终止了这一点,我注意到作业管理器仍然通知连接器,我经常以TaskRejectedExecution结束。这是否意味着shutDownNow()
不会终止实例?跟进:出现的另一个问题是,如果我使用并行流在作业管理器中提交作业,并且如果线程终止,我应该如何处理结束并行流?