Question

我使用亚马逊的Java SDK创建了Amazon SQS和SNS logback appender。基本的appender使用同步Java API，但我也通过扩展ch.qos.logback.classic.AsyncAppender类来创建两者的异步版本。

使用异步appender停止logback记录器上下文不能按预期工作。当上下文停止时，所有异步appender都会在退出之前尝试刷新剩余事件。该问题源自ch.qos.logback.core.AsyncAppenderBase#stop方法，该方法中断了工作线程。当Amazon SDK仍处理排队事件并产生com.amazonaws.AbortedException时，将触发中断。在我的测试中，AbortedException发生在SDK处理来自API的响应时，因此实际的消息已经过去，但情况可能并非总是如此。

即使工作者仍应处理剩余的事件队列，是否打算使用logback中断工作线程？如果是这样，我如何处理由中断引起的AbortedException？我可以覆盖整个停止方法并删除中断，但这需要复制粘贴大部分实现。

Answer 1

我终于设法找到了一个解决方案，我认为这个解决方案不是最优的，而且远非简单，但它正在发挥作用。

我的第一次尝试是使用AWS SDK API的异步版本和提供的logback执行程序，因为使用内部执行程序，可以避免中断问题。但这并没有成功，因为工作队列是共享的，在这种情况下，队列必须是特定于appender才能正确停止它。所以我需要在每个appender上使用自己的执行器。

首先，我需要AWS客户端的执行程序。执行程序的问题是提供的线程工厂必须创建守护程序线程，否则如果使用了logback的JVM关闭挂钩，它将无限期地阻塞。

public static ExecutorService newExecutor(Appender<?> appender, int threadPoolSize) {
    final String name = appender.getName();
    return Executors.newFixedThreadPool(threadPoolSize, new ThreadFactory() {

        private final AtomicInteger idx = new AtomicInteger(1);

        @Override
        public Thread newThread(Runnable r) {
            Thread thread = new Thread(r);
            thread.setName(name + "-" + idx.getAndIncrement());
            thread.setDaemon(true);
            return thread;
        }
    });
}

下一个问题是如何使用中断正确停止appender？这需要通过重试处理中断的异常，因为执行程序否则将跳过等待队列刷新。

public static void shutdown(Appender<?> appender, ExecutorService executor, long waitMillis) {
    executor.shutdown();
    boolean completed = awaitTermination(appender, executor, waitMillis);
    if (!completed) {
        appender.addWarn(format("Executor for %s did not shut down in %d milliseconds, " +
                                "logging events might have been discarded",
                                appender.getName(), waitMillis));
    }
}

private static boolean awaitTermination(Appender<?> appender, ExecutorService executor, long waitMillis) {
    long started = System.currentTimeMillis();
    try {
        return executor.awaitTermination(waitMillis, TimeUnit.MILLISECONDS);
    } catch (InterruptedException ie1) {
        // the worker loop is stopped by interrupt, but the remaining queue should still be handled
        long waited = System.currentTimeMillis() - started;
        if (waited < waitMillis) {
            try {
                return executor.awaitTermination(waitMillis - waited, TimeUnit.MILLISECONDS);
            } catch (InterruptedException ie2) {
                appender.addError(format("Shut down of executor for %s was interrupted",
                                         appender.getName()));
            }
        }
        Thread.currentThread().interrupt();
    }
    return false;
}

正常的logback appender预计会以同步方式工作，因此即使没有正确的关闭挂钩也不应该丢失日志记录事件。这是当前异步AWS SDK API调用的问题。我决定使用倒计时锁存器来提供阻塞appender行为。

public class LoggingEventHandler<REQUEST extends AmazonWebServiceRequest, RESULT> implements AsyncHandler<REQUEST, RESULT> {

    private final ContextAware contextAware;
    private final CountDownLatch latch;
    private final String errorMessage;

    public LoggingEventHandler(ContextAware contextAware, CountDownLatch latch, String errorMessage) {
        this.contextAware = contextAware;
        this.latch = latch;
        this.errorMessage = errorMessage;
    }

    @Override
    public void onError(Exception exception) {
        contextAware.addWarn(errorMessage, exception);
        latch.countDown();
    }

    @Override
    public void onSuccess(REQUEST request, RESULT result) {
        latch.countDown();
    }
}

并处理等待锁定。

public static void awaitLatch(Appender<?> appender, CountDownLatch latch, long waitMillis) {
    if (latch.getCount() > 0) {
        try {
            boolean completed = latch.await(waitMillis, TimeUnit.MILLISECONDS);
            if (!completed) {
                appender.addWarn(format("Appender '%s' did not complete sending event in %d milliseconds, " +
                                        "the event might have been lost",
                                        appender.getName(), waitMillis));
            }
        } catch (InterruptedException ex) {
            appender.addWarn(format("Appender '%s' was interrupted, " +
                                    "a logging event might have been lost or shutdown was initiated",
                                    appender.getName()));
            Thread.currentThread().interrupt();
        }
    }
}

然后全部捆绑在一起。以下示例是实际实现的简化版本，仅显示此问题的相关部分。

public class SqsAppender extends UnsynchronizedAppenderBase<ILoggingEvent> {

    private AmazonSQSAsyncClient sqs;

    @Override
    public void start() {
        sqs = new AmazonSQSAsyncClient(
                getCredentials(),
                getClientConfiguration(),
                Executors.newFixedThreadPool(getThreadPoolSize())
        );
        super.start();
    }

    @Override
    public void stop() {
        super.stop();
        if (sqs != null) {
            AppenderExecutors.shutdown(this, sqs.getExecutorService(), getMaxFlushTime());
            sqs.shutdown();
            sqs = null;
        }
    }

    @Override
    protected void append(final ILoggingEvent eventObject) {
        SendMessageRequest request = ...
        CountDownLatch latch = new CountDownLatch(1);
        sqs.sendMessageAsync(request, new LoggingEventHandler<SendMessageRequest, SendMessageResult>(this, latch, "Error"));
        AppenderExecutors.awaitLatch(this, latch, getMaxFlushTime());
    }
}

所有这些都是正确处理以下情况所必需的：

使用异步appender包装时，在logback context stop或shutdown hook上刷新剩余事件队列
使用logback的延迟关闭挂钩时，不要无限期阻止
在未使用异步appender时提供阻止行为
从async appender stop的中断中生存，导致所有AWS SDK流实现中断

以上内容用于开源项目Logback extensions，我是其维护者。

正确的方法来停止自定义logback async appender

1 个答案: