如何使用ExecutorService等待所有线程完成？

Question

我需要一次执行4个任务，如下所示：

ExecutorService taskExecutor = Executors.newFixedThreadPool(4);
while(...) {
    taskExecutor.execute(new MyTask());
}
//...wait for completion somehow

一旦所有这些内容完成，我该如何收到通知？现在我想不出比设置一些全局任务计数器更好的事情，并在每个任务结束时减少它，然后在无限循环中监视这个计数器变为0;或获得一个Futures列表，并在无限循环监视器isDone中为所有这些。什么是更好的解决方案，不涉及无限循环？

感谢。

Answer 1

基本上ExecutorService shutdown()拨打awaitTermination()然后{{3}}：

ExecutorService taskExecutor = Executors.newFixedThreadPool(4);
while(...) {
  taskExecutor.execute(new MyTask());
}
taskExecutor.shutdown();
try {
  taskExecutor.awaitTermination(Long.MAX_VALUE, TimeUnit.NANOSECONDS);
} catch (InterruptedException e) {
  ...
}

Answer 2

使用CountDownLatch：

CountDownLatch latch = new CountDownLatch(totalNumberOfTasks);
ExecutorService taskExecutor = Executors.newFixedThreadPool(4);
while(...) {
  taskExecutor.execute(new MyTask());
}

try {
  latch.await();
} catch (InterruptedException E) {
   // handle
}

并在你的任务中（包含在try / finally中）

latch.countDown();

Answer 3

ExecutorService.invokeAll()为你做到了。

ExecutorService taskExecutor = Executors.newFixedThreadPool(4);
List<Callable<?>> tasks; // your tasks
// invokeAll() returns when all tasks are complete
List<Future<?>> futures = taskExecutor.invokeAll(tasks);

Answer 4

你也可以使用期货清单：

List<Future> futures = new ArrayList<Future>();
// now add to it:
futures.add(executorInstance.submit(new Callable<Void>() {
  public Void call() throws IOException {
     // do something
    return null;
  }
}));

然后当你想加入所有这些时，它基本上相当于加入每个，（带来额外的好处，它将子线程的异常重新引发到main）：

for(Future f: this.futures) { f.get(); }

基本上诀窍是一次调用每个Future上的.get（），而不是无限循环调用isDone（）on（all或each）。因此，一旦最后一个线程完成，您就可以保证“继续”前进并经过此块。需要注意的是，由于.get（）调用会重新引发异常，如果其中一个线程死亡，您可能会在其他线程完成之前从此引发[为避免这种情况，您可以添加{{1}围绕接听电话]。另一个警告是它保留了对所有线程的引用，所以如果它们有线程局部变量，它们将不会被收集，直到你通过这个块之后（尽管你可能能够绕过它，如果它成为一个问题，通过删除未来离开ArrayList）。如果您想知道哪个“最先完成”，您可以使用https://stackoverflow.com/a/31885029/32453

之类的东西

Answer 5

在Java8中，您可以使用CompletableFuture：

执行此操作

ExecutorService es = Executors.newFixedThreadPool(4);
List<Runnable> tasks = getTasks();
CompletableFuture<?>[] futures = tasks.stream()
                               .map(task -> CompletableFuture.runAsync(task, es))
                               .toArray(CompletableFuture[]::new);
CompletableFuture.allOf(futures).join();    
es.shutdown();

Answer 6

只是我的两分钱。 为了克服CountDownLatch预先知道任务数量的要求，您可以使用简单的Semaphore以旧方式进行。

ExecutorService taskExecutor = Executors.newFixedThreadPool(4);
int numberOfTasks=0;
Semaphore s=new Semaphore(0);
while(...) {
    taskExecutor.execute(new MyTask());
    numberOfTasks++;
}

try {
    s.aquire(numberOfTasks);
...

在您的任务中，只需像s.release()

那样致电latch.countDown();

Answer 7

Java 5及更高版本中的CyclicBarrier类是为此类设计的。

Answer 8

游戏有点晚，但为了完成...

而不是“等待”完成所有任务，你可以考虑好莱坞原则，“不要打电话给我，我会打电话给你” - 当我完成时。 我认为结果代码更优雅......

Guava提供了一些有趣的工具来实现这一目标。

示例::

将ExecutorService包装到ListeningExecutorService ::

中

ListeningExecutorService service = MoreExecutors.listeningDecorator(Executors.newFixedThreadPool(10));

提交一系列可执行的callables ::

for (Callable<Integer> callable : callables) {
  ListenableFuture<Integer> lf = service.submit(callable);
  // listenableFutures is a collection
  listenableFutures.add(lf)
});

现在是必不可少的部分：

ListenableFuture<List<Integer>> lf = Futures.successfulAsList(listenableFutures);

附加一个回调给ListenableFuture，你可以用来在所有期货完成时收到通知::

        Futures.addCallback(lf, new FutureCallback<List<Integer>>() {
        @Override
        public void onSuccess(List<Integer> result) {
            log.info("@@ finished processing {} elements", Iterables.size(result));
            // do something with all the results
        }

        @Override
        public void onFailure(Throwable t) {
            log.info("@@ failed because of :: {}", t);
        }
    });

这也提供了一个优点，即您可以在处理完成后在一个地方收集所有结果......

更多信息here

Answer 9

遵循以下方法之一。

1. 迭代submit ExecutorService上get()返回的所有Future任务，并根据Future Kiran对象上的阻止调用invokeAll()检查状态1}}
2. 在ExecutorService
上使用shutdown, awaitTermination, shutdownNow
3. CountDownLatch
4. ForkJoinPool或Executors.html#newWorkStealingPool
5. 按正确顺序使用ThreadPoolExecutor的node.name API

相关的SE问题：

How is CountDownLatch used in Java Multithreading?

How to properly shutdown java ExecutorService

Answer 10

这是两个选项，请稍稍混淆哪个是最好的选择。

选项1：

ExecutorService es = Executors.newFixedThreadPool(4);
List<Runnable> tasks = getTasks();
CompletableFuture<?>[] futures = tasks.stream()
                               .map(task -> CompletableFuture.runAsync(task, es))
                               .toArray(CompletableFuture[]::new);
CompletableFuture.allOf(futures).join();    
es.shutdown();

选项2：

ExecutorService es = Executors.newFixedThreadPool(4);
List< Future<?>> futures = new ArrayList<>();
for(Runnable task : taskList) {
    futures.add(es.submit(task));
}

for(Future<?> future : futures) {
    try {
        future.get();
    }catch(Exception e){
        // do logging and nothing else
    }
}
es.shutdown();

在这里放置future.get（）;尝试捕获是个好主意吧？

Answer 11

您可以将任务包装在另一个可运行的任务中，它将发送通知：

taskExecutor.execute(new Runnable() {
  public void run() {
    taskStartedNotification();
    new MyTask().run();
    taskFinishedNotification();
  }
});

Answer 12

我刚刚编写了一个解决您问题的示例程序。没有给出简洁的实现，所以我将添加一个。虽然您可以使用executor.shutdown()和executor.awaitTermination()，但这不是最佳做法，因为不同线程所花费的时间是不可预测的。

ExecutorService es = Executors.newCachedThreadPool();
    List<Callable<Integer>> tasks = new ArrayList<>();

    for (int j = 1; j <= 10; j++) {
        tasks.add(new Callable<Integer>() {

            @Override
            public Integer call() throws Exception {
                int sum = 0;
                System.out.println("Starting Thread "
                        + Thread.currentThread().getId());

                for (int i = 0; i < 1000000; i++) {
                    sum += i;
                }

                System.out.println("Stopping Thread "
                        + Thread.currentThread().getId());
                return sum;
            }

        });
    }

    try {
        List<Future<Integer>> futures = es.invokeAll(tasks);
        int flag = 0;

        for (Future<Integer> f : futures) {
            Integer res = f.get();
            System.out.println("Sum: " + res);
            if (!f.isDone()) 
                flag = 1;
        }

        if (flag == 0)
            System.out.println("SUCCESS");
        else
            System.out.println("FAILED");

    } catch (InterruptedException | ExecutionException e) {
        e.printStackTrace();
    }

Answer 13

Just to provide more alternatives here different to use latch/barriers. You can also get the partial results until all of them finish using CompletionService.

From Java Concurrency in practice: "If you have a batch of computations to submit to an Executor and you want to retrieve their results as they become available, you could retain the Future associated with each task and repeatedly poll for completion by calling get with a timeout of zero. This is possible, but tedious. Fortunately there is a better way: a completion service."

Here the implementation

public class TaskSubmiter {
    private final ExecutorService executor;
    TaskSubmiter(ExecutorService executor) { this.executor = executor; }
    void doSomethingLarge(AnySourceClass source) {
        final List<InterestedResult> info = doPartialAsyncProcess(source);
        CompletionService<PartialResult> completionService = new ExecutorCompletionService<PartialResult>(executor);
        for (final InterestedResult interestedResultItem : info)
            completionService.submit(new Callable<PartialResult>() {
                public PartialResult call() {
                    return InterestedResult.doAnOperationToGetPartialResult();
                }
        });

    try {
        for (int t = 0, n = info.size(); t < n; t++) {
            Future<PartialResult> f = completionService.take();
            PartialResult PartialResult = f.get();
            processThisSegment(PartialResult);
            }
        } 
        catch (InterruptedException e) {
            Thread.currentThread().interrupt();
        } 
        catch (ExecutionException e) {
            throw somethinghrowable(e.getCause());
        }
    }
}

Answer 14

您可以使用此代码：

public class MyTask implements Runnable {

    private CountDownLatch countDownLatch;

    public MyTask(CountDownLatch countDownLatch {
         this.countDownLatch = countDownLatch;
    }

    @Override
    public void run() {
         try {
             //Do somethings
             //
             this.countDownLatch.countDown();//important
         } catch (InterruptedException ex) {
              Thread.currentThread().interrupt();
         }
     }
}

CountDownLatch countDownLatch = new CountDownLatch(NUMBER_OF_TASKS);
ExecutorService taskExecutor = Executors.newFixedThreadPool(4);
for (int i = 0; i < NUMBER_OF_TASKS; i++){
     taskExecutor.execute(new MyTask(countDownLatch));
}
countDownLatch.await();
System.out.println("Finish tasks");

Answer 15

这是我的解决方案，基于“AdamSkywalker”提示，它可以正常工作

package frss.main;

import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;

public class TestHilos {

    void procesar() {
        ExecutorService es = Executors.newFixedThreadPool(4);
        List<Runnable> tasks = getTasks();
        CompletableFuture<?>[] futures = tasks.stream().map(task -> CompletableFuture.runAsync(task, es)).toArray(CompletableFuture[]::new);
        CompletableFuture.allOf(futures).join();
        es.shutdown();

        System.out.println("FIN DEL PROCESO DE HILOS");
    }

    private List<Runnable> getTasks() {
        List<Runnable> tasks = new ArrayList<Runnable>();

        Hilo01 task1 = new Hilo01();
        tasks.add(task1);

        Hilo02 task2 = new Hilo02();
        tasks.add(task2);
        return tasks;
    }

    private class Hilo01 extends Thread {

        @Override
        public void run() {
            System.out.println("HILO 1");
        }

    }

    private class Hilo02 extends Thread {

        @Override
        public void run() {
            try {
                sleep(2000);
            }
            catch (InterruptedException e) {
                e.printStackTrace();
            }
            System.out.println("HILO 2");
        }

    }


    public static void main(String[] args) {
        TestHilos test = new TestHilos();
        test.procesar();
    }
}

Answer 16

您应该使用executorService.shutdown()和executorService.awaitTermination方法。

示例如下：

public class ScheduledThreadPoolExample {

    public static void main(String[] args) throws InterruptedException {
        ScheduledExecutorService executorService = Executors.newScheduledThreadPool(5);
        executorService.scheduleAtFixedRate(() -> System.out.println("process task."),
                0, 1, TimeUnit.SECONDS);

        TimeUnit.SECONDS.sleep(10);
        executorService.shutdown();
        executorService.awaitTermination(1, TimeUnit.DAYS);
    }

}

Answer 17

Java 8 - 我们可以使用流API来处理流。请参阅下面的代码段

final List<Runnable> tasks = ...; //or any other functional interface
tasks.stream().parallel().forEach(Runnable::run) // Uses default pool

//alternatively to specify parallelism 
new ForkJoinPool(15).submit(
          () -> tasks.stream().parallel().forEach(Runnable::run) 
    ).get();

Answer 18

所以我在这里发布链接问题的答案，因为有人想要一种更简单的方法来做到这一点

ExecutorService executor = Executors.newFixedThreadPool(10);
CompletableFuture[] futures = new CompletableFuture[10];
int i = 0;
while (...) {
    futures[i++] =  CompletableFuture.runAsync(runner, executor);
}

CompletableFuture.allOf(futures).join(); // THis will wait until all future ready.

Answer 19

您可以使用自己的ExecutorCompletionService子类来包装taskExecutor，以及您自己的BlockingQueue实现，以便在每个任务完成时获得通知，并执行您所需的任何回调或其他操作已完成任务的数量达到了预期目标。

Answer 20

我创建了以下工作示例。这个想法是要有一种方法来处理具有多个线程的任务池（我以队列为例）（通过编程方式由numberOfTasks / threshold确定），并等到所有线程完成后再继续进行其他处理。 / p>

using System;
using System.Diagnostics;
using System.Linq;
using ExtensionMethods;

namespace ConsoleApp1
{

    class Program
    {

        static void Main(string[] args)
        {
            // test data...
            string s = string.Join(";", Enumerable.Range(65, 26).Select(c => (char)c));
            s = s.Insert(3, ";;;");

            string o = "";

            Stopwatch sw = new Stopwatch();

            sw.Start();
            for (int i = 1; i <= 1000000; i++) {
                o = s.Split(';', 21);
            }
            sw.Stop();
            Console.WriteLine("Item directly selected: " + sw.ElapsedMilliseconds);

            sw.Restart();
            for (int i = 1; i <= 1000000; i++) {
                o = s.Split(';')[21];
            }
            sw.Stop();
            Console.WriteLine("Item from split array:  " + sw.ElapsedMilliseconds + "\r\n");


            Console.WriteLine(s);
            Console.WriteLine(o);

            Console.ReadLine();

        }
    }
}

希望有帮助！

Answer 21

使用ExecutorService清洁方式

 List<Future<Void>> results = null;
 try {
     List<Callable<Void>> tasks = new ArrayList<>();
     ExecutorService executorService = Executors.newFixedThreadPool(4);
     results = executorService.invokeAll(tasks);
 } catch (InterruptedException ex) {
     ...
 } catch (Exception ex) {
     ...
 }

Answer 22

ExecutorService WORKER_THREAD_POOL 
  = Executors.newFixedThreadPool(10);
CountDownLatch latch = new CountDownLatch(2);
for (int i = 0; i < 2; i++) {
    WORKER_THREAD_POOL.submit(() -> {
        try {
            // doSomething();
            latch.countDown();
        } catch (InterruptedException e) {
            Thread.currentThread().interrupt();
        }
    });
}

// wait for the latch to be decremented by the two remaining threads
latch.await();

如果doSomething()抛出其他异常，则latch.countDown()似乎将不会执行，那么我该怎么办？

Answer 23

如果您依次使用更多线程ExecutionServices，并希望等待每个EXECUTIONSERVICE完成。最好的方法如下：

ExecutorService executer1 = Executors.newFixedThreadPool(THREAD_SIZE1);
for (<loop>) {
   executer1.execute(new Runnable() {
            @Override
            public void run() {
                ...
            }
        });
} 
executer1.shutdown();

try{
   executer1.awaitTermination(Long.MAX_VALUE, TimeUnit.NANOSECONDS);

   ExecutorService executer2 = Executors.newFixedThreadPool(THREAD_SIZE2);
   for (true) {
      executer2.execute(new Runnable() {
            @Override
            public void run() {
                 ...
            }
        });
   } 
   executer2.shutdown();
} catch (Exception e){
 ...
}

Answer 24

使用 Project Loom 在 AutoCloseable 执行器服务上尝试使用资源语法

Project Loom 旨在为 Java 的并发能力添加新功能。

其中一项功能是使 ExecutorService AutoCloseable。这意味着每个 ExecutorService 实现都将提供一个 close 方法。这意味着我们可以使用 try-with-resources 语法自动关闭 ExecutorService 对象。

ExecutorService#close 方法会阻塞，直到所有提交的任务完成。使用 close 代替调用 shutdown 和 awaitTermination。

成为 AutoCloseable 有助于 Project Loom 将 “structured concurrency” 引入 Java。

try (
    ExecutorService executorService = Executors.… ;
) {
    // Submit your `Runnable`/`Callable` tasks to the executor service.
    …
}
// At this point, flow-of-control blocks until all submitted tasks are done/canceled/failed.
// After this point, the executor service will have been automatically shutdown, wia `close` method called by try-with-resources syntax.

有关 Project Loom 的更多信息，请搜索 Ron Pressler 和 Project Loom 团队其他人的演讲和采访。随着 Project Loom 的发展，请关注更新的内容。

Project Loom 技术的实验版本为 available now，基于抢先体验 Java 17。

Answer 25

您可以在 Runner 类上调用 waitTillDone（）：

Runner runner = Runner.runner(4); // create pool with 4 threads in thread pool

while(...) {
    runner.run(new MyTask()); // here you submit your task
}


runner.waitTillDone(); // and this blocks until all tasks are finished (or failed)


runner.shutdown(); // once you done you can shutdown the runner

您可以重复使用此类，并在调用shutdown（）之前多次调用waitTillDone（），并且您的代码极其简单。此外，您无需事先知道 任务数量。

要使用它，只需将此gradle / maven compile 'com.github.matejtymes:javafixes:1.1.1'依赖项添加到您的项目中。

更多细节可以在这里找到：

https://github.com/MatejTymes/JavaFixes

http://matejtymes.blogspot.com/2016/04/executor-that-notifies-you-when-task.html

Answer 26

这可能会有所帮助

Log.i(LOG_TAG, "shutting down executor...");
executor.shutdown();
while (true) {
                try {
                    Log.i(LOG_TAG, "Waiting for executor to terminate...");
                    if (executor.isTerminated())
                        break;
                    if (executor.awaitTermination(5000, TimeUnit.MILLISECONDS)) {
                        break;
                    }
                } catch (InterruptedException ignored) {}
            }

Answer 27

执行程序getActiveCount()中有一个方法 - 它提供活动线程的计数。

在跨越线程后，我们可以检查activeCount()值是否为0。一旦值为零，就意味着当前没有活动线程正在运行，这意味着任务已完成：

while (true) {
    if (executor.getActiveCount() == 0) {
    //ur own piece of code
    break;
    }
}