使用CompletableFuture重试逻辑

Question

我需要在我正在处理的异步框架中提交任务，但我需要捕获异常，并在“中止”之前多次重试相同的任务。

我正在使用的代码是：

int retries = 0;
public CompletableFuture<Result> executeActionAsync() {

    // Execute the action async and get the future
    CompletableFuture<Result> f = executeMycustomActionHere();

    // If the future completes with exception:
    f.exceptionally(ex -> {
        retries++; // Increment the retry count
        if (retries < MAX_RETRIES)
            return executeActionAsync();  // <--- Submit one more time

        // Abort with a null value
        return null;
    });

    // Return the future    
    return f;
}

目前无法编译，因为lambda的返回类型错误：它需要Result，但executeActionAsync返回CompletableFuture<Result>。

如何实现完全异步重试逻辑？

Answer 1

链接后续重试可以是直截了当的：

public CompletableFuture<Result> executeActionAsync() {
    CompletableFuture<Result> f=executeMycustomActionHere();
    for(int i=0; i<MAX_RETRIES; i++) {
        f=f.exceptionally(t -> executeMycustomActionHere().join());
    }
    return f;
}

^{阅读下面的缺点}
这只是按预期重复链接，因为这些后续阶段在非例外情​​况下不会做任何事情。

一个缺点是，如果第一次尝试立即失败，那么当第一个f处理程序被链接时，exceptionally已经异常完成，操作将由调用线程调用，从而删除异步请求的性质完全。通常，join()可能会阻塞一个线程（默认执行程序将启动一个新的补偿线程，但仍然不鼓励它）。不幸的是，没有exceptionallyAsync或exceptionallyCompose方法。

不调用join()的解决方案是

public CompletableFuture<Result> executeActionAsync() {
    CompletableFuture<Result> f=executeMycustomActionHere();
    for(int i=0; i<MAX_RETRIES; i++) {
        f=f.thenApply(CompletableFuture::completedFuture)
           .exceptionally(t -> executeMycustomActionHere())
           .thenCompose(Function.identity());
    }
    return f;
}

演示如何将“compose”与“特殊”处理程序结合起来。

此外，如果所有重试都失败，则仅报告最后一个异常。更好的解决方案应报告第一个异常，随后将重试添加为抑制异常。这种解决方案可以通过链接递归调用来构建，如Gili’s answer暗示的那样，但是，为了将这个想法用于异常处理，我们必须使用上面显示的“compose”和“exceptionally”组合步骤：

public CompletableFuture<Result> executeActionAsync() {
    return executeMycustomActionHere()
        .thenApply(CompletableFuture::completedFuture)
        .exceptionally(t -> retry(t, 0))
        .thenCompose(Function.identity());
}
private CompletableFuture<Result> retry(Throwable first, int retry) {
    if(retry >= MAX_RETRIES) return CompletableFuture.failedFuture(first);
    return executeMycustomActionHere()
        .thenApply(CompletableFuture::completedFuture)
        .exceptionally(t -> { first.addSuppressed(t); return retry(first, retry+1); })
        .thenCompose(Function.identity());
}

CompletableFuture.failedFuture是一种Java 9方法，但如果需要，可以将Java 8兼容的backport添加到代码中：

public static <T> CompletableFuture<T> failedFuture(Throwable t) {
    final CompletableFuture<T> cf = new CompletableFuture<>();
    cf.completeExceptionally(t);
    return cf;
}

Answer 2

我最近使用guava-retrying库解决了类似的问题。

Callable<Result> callable = new Callable<Result>() {
    public Result call() throws Exception {
        return executeMycustomActionHere();
    }
};

Retryer<Boolean> retryer = RetryerBuilder.<Result>newBuilder()
        .retryIfResult(Predicates.<Result>isNull())
        .retryIfExceptionOfType(IOException.class)
        .retryIfRuntimeException()
        .withStopStrategy(StopStrategies.stopAfterAttempt(MAX_RETRIES))
        .build();

CompletableFuture.supplyAsync( () -> {
    try {
        retryer.call(callable);
    } catch (RetryException e) {
        e.printStackTrace();
    } catch (ExecutionException e) {
       e.printStackTrace();
    }
});

Answer 3

我想我成功了。这是我创建的示例类和测试代码：

RetriableTask.java

np.argmin(np.mean(p,axis=2),axis=0)
array([0, 2, 0], dtype=int64)

用法

public class RetriableTask
{
    protected static final int MAX_RETRIES = 10;
    protected int retries = 0;
    protected int n = 0;
    protected CompletableFuture<Integer> future = new CompletableFuture<Integer>();

    public RetriableTask(int number) {
        n = number;
    }

    public CompletableFuture<Integer> executeAsync() {
        // Create a failure within variable timeout
        Duration timeoutInMilliseconds = Duration.ofMillis(1*(int)Math.pow(2, retries));
        CompletableFuture<Integer> timeoutFuture = Utils.failAfter(timeoutInMilliseconds);

        // Create a dummy future and complete only if (n > 5 && retries > 5) so we can test for both completion and timeouts. 
        // In real application this should be a real future
        final CompletableFuture<Integer> taskFuture = new CompletableFuture<>();
        if (n > 5 && retries > 5)
            taskFuture.complete(retries * n);

        // Attach the failure future to the task future, and perform a check on completion
        taskFuture.applyToEither(timeoutFuture, Function.identity())
            .whenCompleteAsync((result, exception) -> {
                if (exception == null) {
                    future.complete(result);
                } else {
                    retries++;
                    if (retries >= MAX_RETRIES) {
                        future.completeExceptionally(exception);
                    } else {
                        executeAsync();
                    }
                }
            });

        // Return the future    
        return future;
    }
}

输出

int size = 10;
System.out.println("generating...");
List<RetriableTask> tasks = new ArrayList<>();
for (int i = 0; i < size; i++) {
    tasks.add(new RetriableTask(i));
}

System.out.println("issuing...");
List<CompletableFuture<Integer>> futures = new ArrayList<>();
for (int i = 0; i < size; i++) {
    futures.add(tasks.get(i).executeAsync());
}

System.out.println("Waiting...");
for (int i = 0; i < size; i++) {
    try {
        CompletableFuture<Integer> future = futures.get(i);
        int result = future.get();
        System.out.println(i + " result is " + result);
    } catch (Exception ex) {
        System.out.println(i + " I got exception!");
    }
}
System.out.println("Done waiting...");

主要想法和一些粘合代码（generating... issuing... Waiting... 0 I got exception! 1 I got exception! 2 I got exception! 3 I got exception! 4 I got exception! 5 I got exception! 6 result is 36 7 result is 42 8 result is 48 9 result is 54 Done waiting... 函数）来自here。

欢迎任何其他建议或改进。

Answer 4

这是一种适用于任何CompletionStage子类的方法，并且不会返回一个虚拟CompletableFuture，它只会等待其他未来更新。

/**
 * Sends a request that may run as many times as necessary.
 *
 * @param request  a supplier initiates an HTTP request
 * @param executor the Executor used to run the request
 * @return the server response
 */
public CompletionStage<Response> asyncRequest(Supplier<CompletionStage<Response>> request, Executor executor)
{
    return retry(request, executor, 0);
}

/**
 * Sends a request that may run as many times as necessary.
 *
 * @param request  a supplier initiates an HTTP request
 * @param executor the Executor used to run the request
 * @param tries    the number of times the operation has been retried
 * @return the server response
 */
private CompletionStage<Response> retry(Supplier<CompletionStage<Response>> request, Executor executor, int tries)
{
    if (tries >= MAX_RETRIES)
        throw new CompletionException(new IOException("Request failed after " + MAX_RETRIES + " tries"));
    return request.get().thenComposeAsync(response ->
    {
        if (response.getStatusInfo().getFamily() != Response.Status.Family.SUCCESSFUL)
            return retry(request, executor, tries + 1);
        return CompletableFuture.completedFuture(response);
    }, executor);
}

Answer 5

也许已经晚了，但希望有人能发现它有用。我最近解决了这个问题，可以在失败时重试剩余的API调用。就我而言，我必须重试500个HTTP状态代码，以下是我的其余客户端代码（我们正在使用Play框架中的WSClient），您可以根据需要将其更改为任何其他客户端。

 int MAX_RETRY = 3;
 CompletableFuture<WSResponse> future = new CompletableFuture<>();

 private CompletionStage<WSResponse> getWS(Object request,String url, int retry, CompletableFuture future) throws JsonProcessingException {
 ws.url(url)
        .post(Json.parse(mapper.writeValueAsString(request)))
        .whenCompleteAsync((wsResponse, exception) -> {
            if(wsResponse.getStatus() == 500 && retry < MAX_RETRY) {
                try {
                    getWS(request, retry+1, future);
                } catch (IOException e) {
                    throw new Exception(e);
                }
            }else {
                future.complete(wsResponse);
            }
        });

     return future;
}

如果状态代码为200或不是500，则此代码将立即返回，而如果HTTP状态为500，它将重试3次。

Answer 6

我建议使用经过验证的库，例如failsafe，而不是实现自己的重试逻辑，该库具有对期货的内置支持（并且似乎比guava-retrying更受欢迎）。对于您的示例，它看起来像：

private static RetryPolicy retryPolicy = new RetryPolicy()
    .withMaxRetries(MAX_RETRIES);

public CompletableFuture<Result> executeActionAsync() {
    return Failsafe.with(retryPolicy)
        .with(executor)
        .withFallback(null)
        .future(this::executeMycustomActionHere);
}

可能您应该避免使用.withFallback(null)，而只让返回的Future的.get()方法抛出结果异常，以便您的方法的调用者可以具体处理它，但这是您必须做出的设计决策制造。

其他需要考虑的事情包括您是应该立即重试还是在两次尝试之间等待一段时间，任何形式的递归退避（在调用可能会中断的Web服务时很有用），以及是否存在特定的例外情况不值得重试的内容（例如，如果该方法的参数无效）。

Answer 7

util类：

public class RetryUtil {

    public static <R> CompletableFuture<R> retry(Supplier<CompletableFuture<R>> supplier, int maxRetries) {
        CompletableFuture<R> f = supplier.get();
        for(int i=0; i<maxRetries; i++) {
            f=f.thenApply(CompletableFuture::completedFuture)
                .exceptionally(t -> {
                    System.out.println("retry for: "+t.getMessage());
                    return supplier.get();
                })
                .thenCompose(Function.identity());
        }
        return f;
    }
}

用法：

public CompletableFuture<String> lucky(){
    return CompletableFuture.supplyAsync(()->{
        double luckNum = Math.random();
        double luckEnough = 0.6;
        if(luckNum < luckEnough){
            throw new RuntimeException("not luck enough: " + luckNum);
        }
        return "I'm lucky: "+luckNum;
    });
}
@Test
public void testRetry(){
    CompletableFuture<String> retry = RetryUtil.retry(this::lucky, 10);
    System.out.println("async check");
    String join = retry.join();
    System.out.println("lucky? "+join);
}

输出

async check
retry for: java.lang.RuntimeException: not luck enough: 0.412296354211683
retry for: java.lang.RuntimeException: not luck enough: 0.4099777199676573
lucky? I'm lucky: 0.8059089479049389

Answer 8

我们需要根据错误情况重试任务。

public static <T> CompletableFuture<T> retryOnCondition(Supplier<CompletableFuture<T>> supplier,
                                             Predicate<Throwable> retryPredicate, int maxAttempts) {
    if (maxAttempts <= 0) {
        throw new IllegalArgumentException("maxAttempts can't be <= 0");
    }
    return retryOnCondition(supplier, retryPredicate, null, maxAttempts);
}

private static <T> CompletableFuture<T> retryOnCondition(
    Supplier<CompletableFuture<T>> supplier, Predicate<Throwable> retryPredicate,
    Throwable lastError, int attemptsLeft) {

    if (attemptsLeft == 0) {
        return CompletableFuture.failedFuture(lastError);
    }

    return supplier.get()
        .thenApply(CompletableFuture::completedFuture)
        .exceptionally(error -> {
            boolean doRetry = retryPredicate.test(error);
            int attempts = doRetry ? attemptsLeft - 1 : 0;
            return retryOnCondition(supplier, retryPredicate, error, attempts);
        })
        .thenCompose(Function.identity());
}

用法：

public static void main(String[] args) {
    retryOnCondition(() -> myTask(), e -> {
        //log exception
        return e instanceof MyException;
    }, 3).join();
}