通过Streams并行执行多个查询

时间:2015-11-11 11:47:48

标签: java parallel-processing java-8 java-stream

我有以下方法:

public String getResult() {

        List<String> serversList = getServerListFromDB();

        List<String> appList = getAppListFromDB();

        List<String> userList = getUserFromDB();

        return getResult(serversList, appList, userList);
    }

这里我按顺序调用三个方法,然后点击DB并获取结果,然后我对从DB命中获得的结果进行后处理。我知道如何通过使用Threads同时调用这三种方法。但我想使用Java 8 Parallel Stream来实现这一目标。有人可以指导我如何通过Parallel Streams实现同样的目标吗?

编辑我只想通过Stream并行调用这些方法。

private void getInformation() {
    method1();
    method2();
    method3();
    method4();
    method5();
}

4 个答案:

答案 0 :(得分:11)

您可以这样使用CompletableFuture

public String getResult() {

    // Create Stream of tasks:
    Stream<Supplier<List<String>>> tasks = Stream.of(
            () -> getServerListFromDB(),
            () -> getAppListFromDB(),
            () -> getUserFromDB());

    List<List<String>> lists = tasks
         // Supply all the tasks for execution and collect CompletableFutures
         .map(CompletableFuture::supplyAsync).collect(Collectors.toList())
         // Join all the CompletableFutures to gather the results
         .stream()
         .map(CompletableFuture::join).collect(Collectors.toList());

    // Use the results. They are guaranteed to be ordered in the same way as the tasks
    return getResult(lists.get(0), lists.get(1), lists.get(2));
}

答案 1 :(得分:2)

foreach是用于side-effects的内容,您可以在foreach上致电parallel stream。例如:

listOfTasks.parallelStream().foreach(list->{
  submitToDb(list);
});

但是,parallelStream使用的公共ForkJoinPool可能对IO-bound任务不利。

考虑使用CompletableFuture并提供适当的ExecutorService。它提供了更大的灵活性(continuation,配置)。例如:

ExecutorService executorService = Executors.newCachedThreadPool();
List<CompletableFuture> allFutures = new ArrayList<>();
for(Query query:queries){
 CompletableFuture<String> query = CompletableFuture.supplyAsync(() -> {
        // submit query to db
        return result;
  }, executorService);
  allFutures.add(query);
}

 CompletableFuture<Void> all = CompletableFuture.allOf(allFutures.toArray(new CompletableFuture[allFutures.size()]));

答案 2 :(得分:2)

如前所述,标准并行流可能不是最适合您的用例。我将使用ExecutorService异步完成每个任务,并在调用getResult方法时“加入”它们:

ExecutorService es = Executors.newFixedThreadPool(3);

Future<List<String>> serversList = es.submit(() -> getServerListFromDB());
Future<List<String>> appList = es.submit(() -> getAppListFromDB());
Future<List<String>> userList = es.submit(() -> getUserFromDB());

return getResult(serversList.get(), appList.get(), userList.get());

答案 3 :(得分:1)

不太清楚你的意思是什么,但是如果你只想在这些列表上并行运行某些进程,你可以这样做:

    List<String> list1 = Arrays.asList("1", "234", "33");

    List<String> list2 = Arrays.asList("a", "b", "cddd");

    List<String> list3 = Arrays.asList("1331", "22", "33");

    List<List<String>> listOfList = Arrays.asList(list1, list2, list3);

    listOfList.parallelStream().forEach(list -> System.out.println(list.stream().max((o1, o2) -> Integer.compare(o1.length(), o2.length()))));

(它会打印每个列表中最冗长的元素)。