如何散布在链式CompletableFuture内部?

时间:2018-10-25 21:36:04

标签: java-8 parallel-processing completable-future

我想链接一个CompletableFuture,以便它在处理过程中呈扇形散开。我的意思是我对列表有一个开放的CompletableFuture,并且我想对列表中的每个项目应用计算。

第一步是调用 m_myApi.getResponse(request,executor),它发出一个异步调用。

该异步调用的结果具有 getCandidates 方法。我想同时解析所有这些候选人。

当前,我的代码以串行方式解析它们

public CompletableFuture<List<DOMAIN_OBJECT>> parseAllCandidates(@Nonnull final REQUEST request, @Nonnull final Executor executor)
{
        CompletableFuture<RESPONSE> candidates = m_myApi.getResponse(request, executor);
        return candidates.thenApplyAsync(response -> response.getCandidates()
                                                   .stream()
                                                   .map(MyParser::ParseCandidates)
                                                   .collect(Collectors.toList()));
}

我想要这样的东西:

public CompletableFuture<List<DOMAIN_OBJECT>> parseAllCandidates(@Nonnull final REQUEST request, @Nonnull final Executor executor)
{
        CompletableFuture<RESPONSE> candidates = m_myApi.getResponse(request, executor);
        return candidates.thenApplyAsync(response -> response.getCandidates()
                                                   .stream()
                                                   .PARSE_IN_PARALLEL_USING_EXECUTOR
}

2 个答案:

答案 0 :(得分:4)

this answer中所述,如果Executor恰好是一个Fork / Join池,则存在(未记录的)功能,即在其工作线程之一中开始并行流将执行并行操作。使用该执行程序进行操作。

当您想支持任意Executor实现时,事情会变得更加复杂。一种解决方案看起来像

public CompletableFuture<List<DOMAIN_OBJECT>> parseAllCandidates(
       @Nonnull final REQUEST request, @Nonnull final Executor executor)
{
    CompletableFuture<RESPONSE> candidates = m_myApi.getResponse(request, executor);
    return candidates.thenComposeAsync(
        response -> {
            List<CompletableFuture<DOMAIN_OBJECT>> list = response.getCandidates()
                .stream()
                .map(CompletableFuture::completedFuture)
                .map(f -> f.thenApplyAsync(MyParser::ParseCandidates, executor))
                .collect(Collectors.toList());
            return CompletableFuture.allOf(list.toArray(new CompletableFuture<?>[0]))
                .thenApplyAsync(x ->
                    list.stream().map(CompletableFuture::join).collect(Collectors.toList()),
                    executor);
        },
        executor);
}

首要的事情是我们必须在开始等待任何异步作业之前提交所有可能的异步作业,以实现执行程序可能支持的最大并行度。因此,第一步,我们必须在List中收集所有期货。

第二步,我们可以遍历列表和join所有期货。如果执行程序是Fork / Join池,并且将来还没有完成,它将检测到该错误并启动补偿线程以重新获得配置的并行度。但是,对于任意执行程序,我们不能假定具有此功能。最值得注意的是,如果执行程序是单线程执行程序,则可能导致死锁。

因此,仅当所有期货都已完成时,该解决方案才使用CompletableFuture.allOf来执行迭代和联接所有期货的操作。因此,该解决方案永远不会阻塞执行者的线程,使其与任何Executor实现兼容。

答案 1 :(得分:2)

已经有一个Mono: Config attempting to parse: '/Library/Frameworks/Mono.framework/Versions/5.12.0/etc/mono/config'. Mono: Config attempting to parse: '/Users/user/.mono/config'. Mono: process_set_name: using [GalaxianSDL.exe] as prog name Mono: mono_w32handle_new: create Process handle 0x7fcb1901d608 Mono: Assembly Loader probing location: '/Library/Frameworks/Mono.framework/Versions/5.12.0/lib/mono/4.5/mscorlib.dll'. Mono: Image addref mscorlib[0x7fcb18408d30] -> /Library/Frameworks/Mono.framework/Versions/5.12.0/lib/mono/4.5/mscorlib.dll[0x7fcb18805400]: 2 Mono: Prepared to set up assembly 'mscorlib' (/Library/Frameworks/Mono.framework/Versions/5.12.0/lib/mono/4.5/mscorlib.dll) Mono: AOT: module /Library/Frameworks/Mono.framework/Versions/5.12.0/lib/mono/4.5/mscorlib.dll.dylib wants to load image 0: mscorlib Mono: Found assembly remapping for mscorlib and was for the same version 4.0.0.0 Mono: Found assembly remapping for mscorlib and was for the same version 4.0.0.0 Mono: Assembly Loader probing location: '/Library/Frameworks/Mono.framework/Versions/5.12.0/lib/mono/4.5/mscorlib.dll'. Mono: AOT: image '/Library/Frameworks/Mono.framework/Versions/5.12.0/lib/mono/4.5/mscorlib.dll.dylib' found. Mono: Assembly Loader loaded assembly from location: '/Library/Frameworks/Mono.framework/Versions/5.12.0/lib/mono/4.5/mscorlib.dll'. Mono: Config attempting to parse: '/Library/Frameworks/Mono.framework/Versions/5.12.0/lib/mono/4.5/mscorlib.dll.config'. Mono: Config attempting to parse: '/Library/Frameworks/Mono.framework/Versions/5.12.0/etc/mono/assemblies/mscorlib/mscorlib.config'. Mono: AOT: FOUND method System.OutOfMemoryException:.ctor (string) [0x10c956690 - 0x10c9566b2 0x10ccdc208] Mono: AOT NOT FOUND: (wrapper runtime-invoke) <Module>:runtime_invoke_void__this___object (object,intptr,intptr,intptr). Mono: AOT: FOUND method System.SystemException:.ctor (string) [0x10c96ec90 - 0x10c96ecb2 0x10ccdcbff] Mono: AOT: FOUND method System.Exception:.cctor () [0x10c93b500 - 0x10c93b527 0x10ccdb68d] Mono: AOT NOT FOUND: (wrapper runtime-invoke) object:runtime_invoke_void (object,intptr,intptr,intptr). Mono: AOT NOT FOUND: (wrapper alloc) object:AllocSmall (intptr,intptr). Mono: AOT: FOUND method System.Exception:.ctor (string) [0x10c939b80 - 0x10c939bbf 0x10ccdb512] Mono: AOT: FOUND method System.Exception:Init () [0x10c939af0 - 0x10c939b54 0x10ccdb506] Mono: AOT: FOUND method System.NullReferenceException:.ctor (string) [0x10c953570 - 0x10c953592 0x10ccdc146] Mono: AOT: FOUND method System.StackOverflowException:.ctor (string) [0x10c963f70 - 0x10c963f92 0x10ccdc912] Mono: Assembly mscorlib[0x7fcb18408d30] added to domain GalaxianSDL.exe, ref_count=1 Mono: Assembly Loader probing location: 'GalaxianSDL.exe'. Mono: Image addref GalaxianSDL[0x7fcb18507c60] -> /Users/user/Documents/GalaxianSDL/GalaxianSDL/bin/Debug/GalaxianSDL.exe[0x7fcb18805c00]: 2 Mono: Prepared to set up assembly 'GalaxianSDL' (/Users/user/Documents/GalaxianSDL/GalaxianSDL/bin/Debug/GalaxianSDL.exe) Mono: Assembly GalaxianSDL[0x7fcb18507c60] added to domain GalaxianSDL.exe, ref_count=1 Mono: AOT: image '/Users/user/Documents/GalaxianSDL/GalaxianSDL/bin/Debug/GalaxianSDL.exe.dylib' not found: dlopen(/Users/user/Documents/GalaxianSDL/GalaxianSDL/bin/Debug/GalaxianSDL.exe.dylib, 9): image not found Mono: AOT: image '/Library/Frameworks/Mono.framework/Versions/5.12.0/lib/mono/aot-cache/amd64/GalaxianSDL.exe.dylib' not found: dlopen(/Library/Frameworks/Mono.framework/Versions/5.12.0/lib/mono/aot-cache/amd64/GalaxianSDL.exe.dylib, 9): image not found Mono: Assembly Loader loaded assembly from location: 'GalaxianSDL.exe'. Mono: Config attempting to parse: '/Users/user/Documents/GalaxianSDL/GalaxianSDL/bin/Debug/GalaxianSDL.exe.config'. Mono: Config attempting to parse: '/Library/Frameworks/Mono.framework/Versions/5.12.0/etc/mono/assemblies/GalaxianSDL/GalaxianSDL.config'. Mono: Assembly Loader probing location: 'GalaxianSDL.exe'. Mono: AOT: image '/Users/user/Documents/GalaxianSDL/GalaxianSDL/bin/Debug/GalaxianSDL.exe.dylib' not found: dlopen(/Users/user/Documents/GalaxianSDL/GalaxianSDL/bin/Debug/GalaxianSDL.exe.dylib, 9): image not found Mono: AOT: image '/Library/Frameworks/Mono.framework/Versions/5.12.0/lib/mono/aot-cache/amd64/GalaxianSDL.exe.dylib' not found: dlopen(/Library/Frameworks/Mono.framework/Versions/5.12.0/lib/mono/aot-cache/amd64/GalaxianSDL.exe.dylib, 9): image not found Mono: Found assembly remapping for mscorlib and was for the same version 4.0.0.0 Mono: Found assembly remapping for mscorlib and was for the same version 4.0.0.0 Mono: Assembly Ref addref GalaxianSDL[0x7fcb18507c60] -> mscorlib[0x7fcb18408d30]: 2 Mono: AOT NOT FOUND: (wrapper runtime-invoke) object:runtime_invoke_void (object,intptr,intptr,intptr). Mono: Config attempting to parse: '/Users/user/Documents/GalaxianSDL/GalaxianSDL/bin/Debug/GalaxianSDL.exe.config'. Mono: Domain GalaxianSDL.exe search path is: Mono: path[0] = '/Users/user/Documents/GalaxianSDL/GalaxianSDL/bin/Debug/' Mono: End of domain GalaxianSDL.exe search path. Mono: Assembly Loader probing location: '/Library/Frameworks/Mono.framework/Versions/5.12.0/lib/mono/gac/Tao.Sdl/1.2.8.1__9c7a200e36c0094e/Tao.Sdl.dll'. Mono: Assembly Loader probing location: '/Library/Frameworks/Mono.framework/Versions/5.12.0/lib/Tao.Sdl.dll'. Mono: Assembly Loader probing location: '/Library/Frameworks/Mono.framework/Versions/5.12.0/lib/mono/4.5//Facades/Tao.Sdl.dll'. Mono: Assembly Loader probing location: '/Library/Frameworks/Mono.framework/Versions/5.12.0/lib/mono/gac/Tao.Sdl/1.2.8.1__9c7a200e36c0094e/Tao.Sdl.exe'. Mono: Assembly Loader probing location: '/Library/Frameworks/Mono.framework/Versions/5.12.0/lib/Tao.Sdl.exe'. Mono: Assembly Loader probing location: '/Library/Frameworks/Mono.framework/Versions/5.12.0/lib/mono/4.5//Facades/Tao.Sdl.exe'. Mono: AOT: FOUND method System.AppDomain:DoAssemblyResolve (string,System.Reflection.Assembly,bool) [0x10c98c3e0 - 0x10c98c71c 0x10ccdd933] Mono: AOT NOT FOUND: (wrapper runtime-invoke) <Module>:runtime_invoke_object__this___object_object_byte (object,intptr,intptr,intptr). Mono: Domain GalaxianSDL.exe search path is: Mono: path[0] = '/Users/user/Documents/GalaxianSDL/GalaxianSDL/bin/Debug/' Mono: End of domain GalaxianSDL.exe search path. Mono: Assembly Loader probing location: '/Library/Frameworks/Mono.framework/Versions/5.12.0/lib/mono/gac/Tao.Sdl/1.2.8.1__9c7a200e36c0094e/Tao.Sdl.dll'. Mono: Assembly Loader probing location: '/Users/user/Documents/GalaxianSDL/GalaxianSDL/bin/Debug/Tao.Sdl.dll'. Mono: Assembly Loader probing location: '/Library/Frameworks/Mono.framework/Versions/5.12.0/lib/Tao.Sdl.dll'. Mono: Assembly Loader probing location: '/Library/Frameworks/Mono.framework/Versions/5.12.0/lib/mono/4.5//Facades/Tao.Sdl.dll'. Mono: Assembly Loader probing location: '/Library/Frameworks/Mono.framework/Versions/5.12.0/lib/mono/gac/Tao.Sdl/1.2.8.1__9c7a200e36c0094e/Tao.Sdl.exe'. Mono: Assembly Loader probing location: '/Users/user/Documents/GalaxianSDL/GalaxianSDL/bin/Debug/Tao.Sdl.exe'. Mono: Assembly Loader probing location: '/Library/Frameworks/Mono.framework/Versions/5.12.0/lib/Tao.Sdl.exe'. Mono: Assembly Loader probing location: '/Library/Frameworks/Mono.framework/Versions/5.12.0/lib/mono/4.5//Facades/Tao.Sdl.exe'. Mono: The following assembly referenced from /Users/user/Documents/GalaxianSDL/GalaxianSDL/bin/Debug/GalaxianSDL.exe could not be loaded: Assembly: Tao.Sdl (assemblyref_index=1) Version: 1.2.8.1 Public Key: 9c7a200e36c0094e The assembly was not found in the Global Assembly Cache, a path listed in the MONO_PATH environment variable, or in the location of the executing assembly (/Users/user/Documents/GalaxianSDL/GalaxianSDL/bin/Debug/). Mono: Failed to load assembly GalaxianSDL[0x7fcb18507c60]. Mono: AOT: FOUND method System.IO.FileNotFoundException:.ctor (string,string) [0x10caa4790 - 0x10caa47e5 0x10cce50bb] Mono: AOT NOT FOUND: (wrapper runtime-invoke) <Module>:runtime_invoke_void__this___object_object (object,intptr,intptr,intptr). Mono: AOT: FOUND method System.IO.IOException:.ctor (string) [0x10caa74a0 - 0x10caa74c2 0x10cce51f3] Unhandled Exception: System.IO.FileNotFoundException: Could not load file or assembly 'Tao.Sdl, Version=1.2.8.1, Culture=neutral, PublicKeyToken=9c7a200e36c0094e' or one of its dependencies. File name: 'Tao.Sdl, Version=1.2.8.1, Culture=neutral, PublicKeyToken=9c7a200e36c0094e' at GalaxianSDL.Main (System.String[] args) [0x00001] in <141a1af9b0a44e5e8272d4a405970f55>:0 [ERROR] FATAL UNHANDLED EXCEPTION: System.IO.FileNotFoundException: Could not load file or assembly 'Tao.Sdl, Version=1.2.8.1, Culture=neutral, PublicKeyToken=9c7a200e36c0094e' or one of its dependencies. File name: 'Tao.Sdl, Version=1.2.8.1, Culture=neutral, PublicKeyToken=9c7a200e36c0094e' at GalaxianSDL.Main (System.String[] args) [0x00001] in <141a1af9b0a44e5e8272d4a405970f55>:0 的版本,它使用一个thenApply作为附加参数。

Executor

如果您在其中传递forkjoin执行程序,则lambda内的parallel streamuse the passed executor instead of the common pool