ThreadPoolExecutor动态任务执行,等到所有任务完成

时间:2020-01-28 07:17:26

标签: java threadpool threadpoolexecutor

我也有ThreadPoolExecutor

ThreadPoolExecutor executor = new ThreadPoolExecutor(0, Integer.MAX_VALUE, 60L, TimeUnit.SECONDS, new SynchronousQueue<>());

任务执行如下

executor.execute(task)

现在,每个任务可能也可以对同一执行者执行更多任务,而那些新任务可以提交更多任务

问题是我希望主线程等待所有任务执行完毕,然后再调用shutdown

以下方法是否一定行得通? (即封锁/等待主要 直到所有任务都完成)

while (executor.getCompletedTaskCount() < executor.getTaskCount()) {
      try {
        Thread.sleep(100);
      } catch (InterruptedException e) {
        LOGGER.error("Exception in atomic Count wait thread sleep", e);
        break;
      }
    }
}

这最终会打破循环吗?只是通过初步测试,我发现它即使在线程异常的情况下也能正常工作

PS 我无法使用闩锁,因为我事先不知道任务数量 也没有接受的答案here

3 个答案:

答案 0 :(得分:2)

您可能应该保留提交的期货。

Deque<Future<?>> futures = new ConcurrentLinkedDeque<>();

然后每次提交任务时。

futures.add(executor.submit( runnable, "Doesn't Really Matter, but Can be Useful"));

然后在主线程中等待。

while(futures.size()>0){
    futures.pop().get();
}

这将为您保证.get在任务完成之前不会完成,并且如果另一个任务添加了更多任务,那么期货将在原始任务完成之前反映出更改。

答案 1 :(得分:1)

在我看来,获取任务的实际计数是不确定的,因为在提交任务时将调用execute方法,并且可能发生以下3种情况之一。 1.任务开始执行(添加到Workers中) 2.任务入队(添加到WorkQueue中) 3.由于工作人员队列容量,工作人员容量和资源耗尽而拒绝了任务

 /**
     * Executes the given task sometime in the future.  The task
     * may execute in a new thread or in an existing pooled thread.
     *
     * If the task cannot be submitted for execution, either because this
     * executor has been shutdown or because its capacity has been reached,
     * the task is handled by the current {@code RejectedExecutionHandler}.
     *
     * @param command the task to execute
     * @throws RejectedExecutionException at discretion of
     *         {@code RejectedExecutionHandler}, if the task
     *         cannot be accepted for execution
     * @throws NullPointerException if {@code command} is null
     */
    public void execute(Runnable command) {
        if (command == null)
            throw new NullPointerException();
        /*
         * Proceed in 3 steps:
         *
         * 1. If fewer than corePoolSize threads are running, try to
         * start a new thread with the given command as its first
         * task.  The call to addWorker atomically checks runState and
         * workerCount, and so prevents false alarms that would add
         * threads when it shouldn't, by returning false.
         *
         * 2. If a task can be successfully queued, then we still need
         * to double-check whether we should have added a thread
         * (because existing ones died since last checking) or that
         * the pool shut down since entry into this method. So we
         * recheck state and if necessary roll back the enqueuing if
         * stopped, or start a new thread if there are none.
         *
         * 3. If we cannot queue task, then we try to add a new
         * thread.  If it fails, we know we are shut down or saturated
         * and so reject the task.
         */
        int c = ctl.get();
        if (workerCountOf(c) < corePoolSize) {
            if (addWorker(command, true))
                return;
            c = ctl.get();
        }
        if (isRunning(c) && workQueue.offer(command)) {
            int recheck = ctl.get();
            if (! isRunning(recheck) && remove(command))
                reject(command);
            else if (workerCountOf(recheck) == 0)
                addWorker(null, false);
        }
        else if (!addWorker(command, false))
            reject(command);
    }

getTaskCount()和getCompletedTaskCount()方法由mainLock保护,因此我们知道内部线程是否仍在向执行者提交任务将由主执行中的时间检查(while (executor.getCompletedTaskCount() < executor.getTaskCount())来完成。这种情况可能会暂时导致误报,最终导致错误的结果。

/**
     * Returns the approximate total number of tasks that have ever been
     * scheduled for execution. Because the states of tasks and
     * threads may change dynamically during computation, the returned
     * value is only an approximation.
     *
     * @return the number of tasks
     */
    public long getTaskCount() {
        final ReentrantLock mainLock = this.mainLock;
        mainLock.lock();
        try {
            long n = completedTaskCount;
            for (Worker w : workers) {
                n += w.completedTasks;
                if (w.isLocked())
                    ++n;
            }
            return n + workQueue.size();
        } finally {
            mainLock.unlock();
        }
    }
    /**
     * Returns the approximate total number of tasks that have
     * completed execution. Because the states of tasks and threads
     * may change dynamically during computation, the returned value
     * is only an approximation, but one that does not ever decrease
     * across successive calls.
     *
     * @return the number of tasks
     */
    public long getCompletedTaskCount() {
        final ReentrantLock mainLock = this.mainLock;
        mainLock.lock();
        try {
            long n = completedTaskCount;
            for (Worker w : workers)
                n += w.completedTasks;
            return n;
        } finally {
            mainLock.unlock();
        }
    }

**此处使用的代码段来自JDK 1.8 222

答案 2 :(得分:0)

根据Java(8)文档,用于获取完整计数和已提交计数(即executor.getCompletedTaskCount() & executor.getTaskCount())的方法并不总是能提供100%的准确计数,因此该方法可能并不总是有效。

public long getTaskCount()

返回已完成的大约个任务总数 计划执行。因为任务和线程的状态可能 在计算过程中动态变化,返回的值只是 近似值

public long getCompletedTaskCount()

返回已完成任务的大约总数 执行。因为任务和线程的状态可能会改变 动态地在计算过程中,返回的值只是一个 近似值,但是在连续的过程中永远不会减少 呼叫