FileWritingMessageHandler(int-file:outbound-channel-adapter)对于单文件存储多个消息的用例很慢

时间:2014-12-05 21:33:30

标签: java spring spring-integration log4j2 spring-el

我在Spring Integration 4.1.0中使用Spring 4.1.2。

我有一个用例,我希望生成一个文件,该文件将包含流向某个频道的每条消息的行。收到的消息都是String类型。这个文件是一个很好用的文件,这意味着没有必要让这个文件的写入在主流的同一个事务中。因此,可以为用例实现异步线控模式。然而,写入该文件的任何消息都必须与它们最初接收的顺序相同(因此要么1个线程需要处理它们,要么需要聚合器等待多个线程完成,然后按原始顺序写入它们)。

我想了解一下处理这个用例的最高性能方法,所以我尝试了一些测试。为了使它更容易,我的测试没有使用异步wire-tap(但在用例中提到了这一点,因为可能有些建议可能涉及批处理/缓冲解决方案)。

总体流程来自"定义整合流程"此链接的一部分:https://spring.io/guides/gs/integration/

我尝试的主要选项是:

  1. 使用int-file:outbound-channel-adapter(创建FileWritingMessageHandler)以及为每条消息附加换行符的转换器(转换器使用SpEL表达式payload + '#{systemProperties['line.separator']}。) spring.expression.compiler.mode=OFF
  2. 使用int-file:outbound-channel-adapter(创建FileWritingMessageHandler)以及为每条消息附加换行符的转换器(转换器使用SpEL表达式payload.toString() + '#{systemProperties['line.separator']}。) spring.expression.compiler.mode=MIXED
    注意:使用payload.toString()而不是payload来解决SpEL问题:https://jira.spring.io/browse/SPR-12514
  3. 使用int:logging-channel-adapter而非int-file:outbound-channel-adapter(省去必须使用带SpEL表达式的变压器)。
    使用RollingRandomAccessFile和同步记录器使用Log4J2进行测试 spring.expression.compiler.mode=OFF
  4. 使用int:logging-channel-adapter而非int-file:outbound-channel-adapter(省去必须使用带SpEL表达式的变压器)。
    使用RollingRandomAccessFile和异步记录器使用Log4J2进行测试。请参阅http://logging.apache.org/log4j/2.0/manual/async.html#Making所有记录器异步 spring.expression.compiler.mode=OFF
  5. 使用int:logging-channel-adapter而非int-file:outbound-channel-adapter(省去必须使用带SpEL表达式的变压器)。
    使用RollingRandomAccessFile和异步记录器使用Log4J2进行测试。请参阅http://logging.apache.org/log4j/2.0/manual/async.html#Making所有记录器异步 spring.expression.compiler.mode=MIXED
  6. 测试用例1和2流程: int-file:outbound-channel-adapter flow

    测试用例3到5流程: int:logging-channel-adapter flow

    输入文件包含XML数据(字符串),其长度在每行1200到1500个字符之间变化(每行是单个消息)。
    在我的测试中,我有203,712条消息 以下是时间安排。由于SpEL编译器在一段时间后启动,因此我显示第一项的时间比最后一项更多。

    |          1              |            2             |              3                 |               4                |              5                 |
    |SpringInt FileAdapter    | SpringInt FileAdapter    | Log4j2 RollingRandomAccessFile | Log4j2 RollingRandomAccessFile | Log4j2 RollingRandomAccessFile |
    |                         |                          | Sync Loggers                   | Async Loggers                  | Async with                     |
    |SpEL-compiler=OFF        | SpEL-compiler=MIXED      | SpEL-compiler=OFF              | SpEL-compiler=OFF              | SpEL-compiler=MIXED            |
    |-------------------------|--------------------------|--------------------------------|--------------------------------|------------------------------- |
    |Cnt=10000 : 0:00:12.670  | Cnt=10000 : 0:00:17.235  | Cnt=10000 : 0:00:08.222        | Cnt=10000 : 0:00:01.847        | Cnt=10000 : 0:00:01.320        |
    |Cnt=20000 : 0:00:24.636  | Cnt=20000 : 0:00:30.208  | Cnt=20000 : 0:00:08.828        | Cnt=20000 : 0:00:02.232        | Cnt=20000 : 0:00:01.839        |
    |Cnt=30000 : 0:00:36.179  | Cnt=30000 : 0:00:44.300  | Cnt=30000 : 0:00:09.426        | Cnt=30000 : 0:00:02.512        | Cnt=30000 : 0:00:02.647        |
    |...                      | ....                     | ...                            | ...                            | ...                            |
    |Cnt=180000 : 0:02:58.935 | Cnt=180000 : 0:04:15.528 | Cnt=180000 : 0:00:17.095       | Cnt=180000 : 0:00:08.546       | Cnt=180000 : 0:00:07.936       |
    |Cnt=200000 : 0:03:16.473 | Cnt=200000 : 0:04:35.582 | Cnt=200000 : 0:00:18.107       | Cnt=200000 : 0:00:09.548       | Cnt=200000 : 0:00:08.660       |
    |Cnt=203712 : 0:03:19.715 | Cnt=203712 : 0:04:39.452 | Cnt=203712 : 0:00:18.284       | Cnt=203712 : 0:00:09.661       | Cnt=203712 : 0:00:08.732       |
    

    拿着一粒盐的时间 - 我没有运行这几十次并取平均值。我也没有提倡log4j2比其他提供的东西更快,比如logback,我只是用它来进行比较。注意:我仅使用文件作为此测试的输入。我指出这一点,因为有人可能建议让Spring Integration将原始文件从fileA复制到fileB。在我们的实际用例中,消息实际上是通过JMS进入的,因此文件到文件的解决方案不是一个真正的选择。
    有趣的观点:

    • Spring Integration FileWritingMessageHandler比任何log4j2产品慢很多 Log4j2-async占用时间FileWritingMessageHandler的4.3%(方案1为199.715秒,方案5为8.732秒)。
      Log4j2-sync花费了4.8%的FileWritingMessageHandler时间(scenario1为199.715秒,scenario4为9.661秒)。
    • FileWritingMessageHandlerspring.expression.compiler.mode=MIXED(场景#2)的Spring Integration spring.expression.compiler.mode=OFF实际上比payload + '#{systemProperties['line.separator']}慢。我认为这是因为在方案#1中我能够使用payload.toString() + '#{systemProperties['line.separator']}而在方案#2中我必须使用logging-channel-adapter
    • 方案3至5与预期相同,相对于其他方案。

    理想情况下,我不想仅仅使用FileWritingMessageHandler将消息写入文件 - 似乎我正在混淆该组件。然而,性能上的提升是显着的,所以不幸的是现在我不能排除使用它 所以我的问题是:

    • 除了编写自己的FileWritingMessageHandler以获得更好的文件写入性能外,我还有哪些其他选择?
    • 我假设如果我在FileWritingMessageHandler之前批处理或聚合,那么写出批处理组的表现可能会更好。我确信我还可以使用任务执行程序和轮询器(我的用例允许这样做)。如果buffersize公开FileWritingMessageHandler属性?
    • ,则应将批处理视为一个选项
    • 是否可以调整StreamWriter,或者可能提供更具特色的版本,这些版本对我的用例更有效(可能从log4j2记录器中获取一些建议/提示)?
    • 文件payload.toString() + '#{systemProperties['line.separator']}会更高效吗?
    • 在这个问题上大声思考:log4j2"包装器应该是什么?纯粹作为文件适配器提供的类(即它必须只记录没有行/类/ etc信息的消息,无论级别如何都必须写入,用户只需传入文件名并可能同步/异步)?
    • SpEL编译器能否更好地优化以处理 <dependencies> <!-- Testing --> <dependency> <groupId>junit</groupId> <artifactId>junit</artifactId> <version>${junit.version}</version> <scope>test</scope> </dependency> <!-- Spring Integration --> <dependency> <groupId>org.springframework.integration</groupId> <artifactId>spring-integration-core</artifactId> <version>${spring.integration.version}</version> </dependency> <dependency> <groupId>org.springframework.integration</groupId> <artifactId>spring-integration-file</artifactId> <version>${spring.integration.version}</version> </dependency> <!-- Logging --> <dependency> <groupId>org.slf4j</groupId> <artifactId>slf4j-api</artifactId> <version>1.7.7</version> </dependency> <!-- Binding for JCL (aka Java Common Logging). --> <!-- Needed since things like the commons libs all use commons-logging which we don't want --> <dependency> <groupId>org.slf4j</groupId> <artifactId>jcl-over-slf4j</artifactId> <version>1.7.7</version> <!-- Making scope be runtime so we'll catch any of our own classes that try to use commons-logging when we compile --> <scope>runtime</scope> </dependency> <!-- Binding for Log4J --> <dependency> <groupId>org.apache.logging.log4j</groupId> <artifactId>log4j-slf4j-impl</artifactId> <!-- As of 9/12/2014 our company Maven repos does not have 2.0.2 --> <version>2.0.1</version> </dependency> <!-- Log4j API and Core implementation required for binding --> <dependency> <groupId>org.apache.logging.log4j</groupId> <artifactId>log4j-api</artifactId> <version>2.0.2</version> </dependency> <dependency> <groupId>org.apache.logging.log4j</groupId> <artifactId>log4j-core</artifactId> <version>2.0.2</version> </dependency> <!-- Async loggers for log4j2 require LMAX disruptor, see http://logging.apache.org/log4j/2.x/manual/async.html --> <dependency> <groupId>com.lmax</groupId> <artifactId>disruptor</artifactId> <version>3.2.1</version> </dependency> <dependency> <groupId>org.apache.commons</groupId> <artifactId>commons-lang3</artifactId> <version>3.1</version> </dependency> </dependencies> 情况,因为如上所述它实际上比不在SpEL本身中调用toString()慢?

    以下是用于测试的代码/配置文件。

    的pom.xml

    package com.xxx;
    
    import java.util.Scanner;
    
    import org.slf4j.Logger;
    import org.slf4j.LoggerFactory;
    import org.springframework.context.support.AbstractApplicationContext;
    import org.springframework.context.support.ClassPathXmlApplicationContext;
    
    /**
     * Starts the Spring Context and will initialize the Spring Integration routes.
     */
    public final class Main {
    
        private static final Logger LOGGER = LoggerFactory.getLogger(Main.class);
    
        private Main() {
        }
    
        /**
         * Load the Spring Integration Application Context
         *
         * @param args - command line arguments
         */
        public static void main(final String... args) {
    
            if (LOGGER.isInfoEnabled()) {
                LOGGER.info("\n=========================================================" + "\n                                                         " + "\n          Welcome to Spring Integration!                 " + "\n                                                         " + "\n    For more information please visit:                   " + "\n    http://www.springsource.org/spring-integration       " + "\n                                                         " + "\n=========================================================");
            }
    
            final AbstractApplicationContext context = new ClassPathXmlApplicationContext("classpath:META-INF/spring/integration/spring-integration-context-usecases.xml");
    
            context.registerShutdownHook();
    
            SpringIntegrationUtils.displayDirectories(context);
    
            final Scanner scanner = new Scanner(System.in);
    
            if (LOGGER.isInfoEnabled()) {
                LOGGER.info("\n=========================================================" + "\n                                                         " + "\n    Please press 'q + Enter' to quit the application.    " + "\n                                                         " + "\n=========================================================");
            }
    
            while (!scanner.hasNext("q")) {
                //Do nothing unless user presses 'q' to quit.
            }
    
            if (LOGGER.isInfoEnabled()) {
                LOGGER.info("Exiting application...bye.");
            }
    
            System.exit(0);
    
        }
    }
    
    
    
    package com.xxx;
    
    import java.io.File;
    import java.io.IOException;
    import java.util.Collections;
    import java.util.Iterator;
    import java.util.List;
    import java.util.Map;
    import java.util.concurrent.atomic.AtomicInteger;
    
    import org.apache.commons.lang3.time.StopWatch;
    import org.slf4j.Logger;
    import org.slf4j.LoggerFactory;
    import org.springframework.integration.IntegrationMessageHeaderAccessor;
    import org.springframework.integration.routingslip.RoutingSlipRouteStrategy;
    import org.springframework.integration.splitter.AbstractMessageSplitter;
    import org.springframework.integration.support.AbstractIntegrationMessageBuilder;
    import org.springframework.integration.transformer.MessageTransformationException;
    import org.springframework.messaging.Message;
    import org.springframework.messaging.MessageChannel;
    import org.springframework.messaging.MessageHeaders;
    import org.springframework.messaging.MessagingException;
    import org.springframework.messaging.core.DestinationResolutionException;
    import org.springframework.util.Assert;
    import org.springframework.util.StringUtils;
    
    /**
     * This class is only needed until a bug is fixed in Spring Integration 4.1.0.
     * See {@link http://stackoverflow.com/questions/27171978/read-csv-file-concurrently-using-spring-integration}
     * Once that is fixed delete this class and use this in the Spring context file.
     * <code>
     * <splitter input-channel="splitChannel" output-channel="executorChannel" expression="T(org.apache.commons.io.FileUtils).lineIterator(payload)"/>
     * </code>
     *
     */
    public class FileSplitter extends AbstractMessageSplitter {
        private static final Logger log = LoggerFactory.getLogger(FileSplitter.class);
    
        int counter = 0;
        StopWatch sw = new StopWatch();
    
        public Object splitMessage(Message<?> message) {
            if (log.isDebugEnabled()) {
                log.debug(message.toString());
            }
            try {
    
                Object payload = message.getPayload();
                Assert.isInstanceOf(File.class, payload, "Expected java.io.File in the message payload");
    
                return org.apache.commons.io.FileUtils.lineIterator((File) payload);
            } catch (IOException e) {
                String msg = "Unable to transform file: " + e.getMessage();
                log.error(msg);
                throw new MessageTransformationException(msg, e);
            }
        }
    
        @Override
        protected void produceOutput(Object result, Message<?> requestMessage) {
    
            Iterator<?> iterator = (Iterator<?>) result;
            sw.start();
            while (iterator.hasNext()) {
                ++counter;
                produceOutputInternal(iterator.next(), requestMessage);
                if (counter % 10000 == 0) {
                    sw.split();
                    System.out.println("Cnt=" + counter + " : " + sw.toSplitString());
                }
            }
            sw.stop();
            System.out.println("completed");
            System.out.println("Cnt=" + counter + " : " + sw.toSplitString());
        }
    
        private Object getOutputChannelFromRoutingSlip(Object reply, Message<?> requestMessage, List<?> routingSlip, AtomicInteger routingSlipIndex) {
            if (routingSlipIndex.get() >= routingSlip.size()) {
                return null;
            }
    
            Object path = routingSlip.get(routingSlipIndex.get());
            Object routingSlipPathValue = null;
    
            if (path instanceof String) {
                routingSlipPathValue = getBeanFactory().getBean((String) path);
            } else if (path instanceof RoutingSlipRouteStrategy) {
                routingSlipPathValue = path;
            } else {
                throw new IllegalArgumentException("The RoutingSlip 'path' can be of " + "String or RoutingSlipRouteStrategy type, but gotten: " + path);
            }
    
            if (routingSlipPathValue instanceof MessageChannel) {
                routingSlipIndex.incrementAndGet();
                return routingSlipPathValue;
            } else {
                Object nextPath = ((RoutingSlipRouteStrategy) routingSlipPathValue).getNextPath(requestMessage, reply);
                if (nextPath != null && (!(nextPath instanceof String) || StringUtils.hasText((String) nextPath))) {
                    return nextPath;
                } else {
                    routingSlipIndex.incrementAndGet();
                    return getOutputChannelFromRoutingSlip(reply, requestMessage, routingSlip, routingSlipIndex);
                }
            }
        }
    
        protected void produceOutputInternal(Object reply, Message<?> requestMessage) {
            MessageHeaders requestHeaders = requestMessage.getHeaders();
    
            Object replyChannel = null;
            if (getOutputChannel() == null) {
                Map<?, ?> routingSlipHeader = requestHeaders.get(IntegrationMessageHeaderAccessor.ROUTING_SLIP, Map.class);
                if (routingSlipHeader != null) {
                    Assert.isTrue(routingSlipHeader.size() == 1, "The RoutingSlip header value must be a SingletonMap");
                    Object key = routingSlipHeader.keySet().iterator().next();
                    Object value = routingSlipHeader.values().iterator().next();
                    Assert.isInstanceOf(List.class, key, "The RoutingSlip key must be List");
                    Assert.isInstanceOf(Integer.class, value, "The RoutingSlip value must be Integer");
                    List<?> routingSlip = (List<?>) key;
                    AtomicInteger routingSlipIndex = new AtomicInteger((Integer) value);
                    replyChannel = getOutputChannelFromRoutingSlip(reply, requestMessage, routingSlip, routingSlipIndex);
                    if (replyChannel != null) {
                        //TODO Migrate to the SF MessageBuilder
                        AbstractIntegrationMessageBuilder<?> builder = null;
                        if (reply instanceof Message) {
                            builder = this.getMessageBuilderFactory().fromMessage((Message<?>) reply);
                        } else if (reply instanceof AbstractIntegrationMessageBuilder) {
                            builder = (AbstractIntegrationMessageBuilder<?>) reply;
                        } else {
                            builder = this.getMessageBuilderFactory().withPayload(reply);
                        }
                        builder.setHeader(IntegrationMessageHeaderAccessor.ROUTING_SLIP, Collections.singletonMap(routingSlip, routingSlipIndex.get()));
                        reply = builder;
                    }
                }
    
                if (replyChannel == null) {
                    replyChannel = requestHeaders.getReplyChannel();
                }
            }
    
            Message<?> replyMessage = createOutputMessage(reply, requestHeaders);
            sendOutput(replyMessage, replyChannel);
        }
    
        private Message<?> createOutputMessage(Object output, MessageHeaders requestHeaders) {
            AbstractIntegrationMessageBuilder<?> builder = null;
            if (output instanceof Message<?>) {
                if (!this.shouldCopyRequestHeaders()) {
                    return (Message<?>) output;
                }
                builder = this.getMessageBuilderFactory().fromMessage((Message<?>) output);
            } else if (output instanceof AbstractIntegrationMessageBuilder) {
                builder = (AbstractIntegrationMessageBuilder<?>) output;
            } else {
                builder = this.getMessageBuilderFactory().withPayload(output);
            }
            if (this.shouldCopyRequestHeaders()) {
                builder.copyHeadersIfAbsent(requestHeaders);
            }
            return builder.build();
        }
    
        private void sendOutput(Object output, Object replyChannel) {
            MessageChannel outputChannel = getOutputChannel();
            if (outputChannel != null) {
                replyChannel = outputChannel;
            }
            if (replyChannel == null) {
                throw new DestinationResolutionException("no output-channel or replyChannel header available");
            }
    
            if (replyChannel instanceof MessageChannel) {
                if (output instanceof Message<?>) {
                    this.messagingTemplate.send((MessageChannel) replyChannel, (Message<?>) output);
                } else {
                    this.messagingTemplate.convertAndSend((MessageChannel) replyChannel, output);
                }
            } else if (replyChannel instanceof String) {
                if (output instanceof Message<?>) {
                    this.messagingTemplate.send((String) replyChannel, (Message<?>) output);
                } else {
                    this.messagingTemplate.convertAndSend((String) replyChannel, output);
                }
            } else {
                throw new MessagingException("replyChannel must be a MessageChannel or String");
            }
        }
    }
    
    
    package com.xxx;
    
    import java.io.File;
    import java.util.ArrayList;
    import java.util.List;
    import java.util.Map;
    
    import org.apache.commons.logging.Log;
    import org.apache.commons.logging.LogFactory;
    import org.springframework.beans.DirectFieldAccessor;
    import org.springframework.context.ApplicationContext;
    import org.springframework.expression.Expression;
    import org.springframework.integration.file.FileReadingMessageSource;
    import org.springframework.integration.file.FileWritingMessageHandler;
    
    /**
     * Displays the names of the input and output directories.
     */
    public final class SpringIntegrationUtils {
    
        private static final Log logger = LogFactory.getLog(SpringIntegrationUtils.class);
    
        private SpringIntegrationUtils() { }
    
        /**
         * Helper Method to dynamically determine and display input and output
         * directories as defined in the Spring Integration context.
         *
         * @param context Spring Application Context
         */
        public static void displayDirectories(final ApplicationContext context) {
    
            final File inDir = (File) new DirectFieldAccessor(context.getBean(FileReadingMessageSource.class)).getPropertyValue("directory");
    
            final Map<String, FileWritingMessageHandler> fileWritingMessageHandlers = context.getBeansOfType(FileWritingMessageHandler.class);
    
            final List<String> outputDirectories = new ArrayList<String>();
    
            for (final FileWritingMessageHandler messageHandler : fileWritingMessageHandlers.values()) {
                final Expression outDir = (Expression) new DirectFieldAccessor(messageHandler).getPropertyValue("destinationDirectoryExpression");
                outputDirectories.add(outDir.getExpressionString());
            }
    
            final StringBuilder stringBuilder = new StringBuilder();
    
            stringBuilder.append("\n=========================================================");
            stringBuilder.append("\n");
            stringBuilder.append("\n    Input directory is : '" + inDir.getAbsolutePath() + "'");
    
            for (final String outputDirectory : outputDirectories) {
                stringBuilder.append("\n    Output directory is: '" + outputDirectory + "'");
            }
    
            stringBuilder.append("\n\n=========================================================");
    
            logger.info(stringBuilder.toString());
    
        }
    
    }
    

    Java类

    log4j2.xml

    <?xml version="1.0" encoding="UTF-8"?> <Configuration> <Appenders> <Console name="STDOUT" target="SYSTEM_OUT"> <PatternLayout pattern="%d{ISO8601} [%t] [%-5p] (%c) - %m%n" /> </Console> <RollingRandomAccessFile name="fileAppenderMessages" fileName="C:/Users/xxxxx/Desktop/fileadapter-test/usecase3.txt"> <PatternLayout pattern="%m %n" /> </RollingRandomAccessFile> </Appenders> <Loggers> <!-- The Wire-Tap and logging-channel-adapter in the Spring cfg file will use this category name --> <Logger name="fileLogger" additivity="false"> <AppenderRef ref="fileAppenderMessages" /> </Logger> <Root level="info"> <AppenderRef ref="STDOUT" /> </Root> </Loggers> </Configuration> 配置文件

    spring-integration-context-usecases.xml

    <?xml version="1.0" encoding="UTF-8"?> <beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:context="http://www.springframework.org/schema/context" xmlns:int="http://www.springframework.org/schema/integration" xmlns:int-file="http://www.springframework.org/schema/integration/file" xmlns:int-stream="http://www.springframework.org/schema/integration/stream" xmlns:batch="http://www.springframework.org/schema/batch" xmlns:task="http://www.springframework.org/schema/task" xsi:schemaLocation="http://www.springframework.org/schema/jms http://www.springframework.org/schema/jms/spring-jms.xsd http://www.springframework.org/schema/integration http://www.springframework.org/schema/integration/spring-integration.xsd http://www.springframework.org/schema/integration/file http://www.springframework.org/schema/integration/file/spring-integration-file.xsd http://www.springframework.org/schema/integration/stream http://www.springframework.org/schema/integration/stream/spring-integration-stream.xsd http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd http://www.springframework.org/schema/task http://www.springframework.org/schema/task/spring-task.xsd"> <int:inbound-channel-adapter id="fileAdapter" ref="fileReadingMessageSource" method="receive" auto-startup="true" channel="files" > <int:poller fixed-delay="#{T(java.lang.Integer).MAX_VALUE}"/> </int:inbound-channel-adapter> <bean id="fileReadingMessageSource" class="org.springframework.integration.file.FileReadingMessageSource"> <property name="directory" value="C:/Users/xxxxx/Desktop/tmg-exchange-gateway-nam/t2"/> </bean> <int:channel id="files"/> <int:splitter input-channel="files" output-channel="stringMessages"> <bean class="com.xxx.FileSplitter" /> </int:splitter> <int:channel id="stringMessages"/> <int:transformer expression="payload + '#{systemProperties['line.separator']}'" output-channel="file" auto-startup="true" input-channel="stringMessages"/> <int-file:outbound-channel-adapter id="file" mode="APPEND" charset="UTF-8" directory="C:/Users/xxxxx/Desktop/fileadapter-test" auto-create-directory="true" filename-generator-expression="'usecase2.txt'"/> </beans> 档案

    1. java -Dspring.expression.compiler.mode=OFF com.xxx.Main
    Leave context file unchanged.
    2. java -Dspring.expression.compiler.mode=MIXED com.xxx.Main
    Change context file to have expression="payload.toString() + '#{systemProperties['line.separator']}'"
    3. java -Dspring.expression.compiler.mode=OFF com.xxx.Main
    Comment out transformer and outbound-channel-adapter.
    Change logging-channel-adapter   auto-startup="true"
    4. java -DLog4jContextSelector=org.apache.logging.log4j.core.async.AsyncLoggerContextSelector  -Dspring.expression.compiler.mode=OFF com.xxx.Main
    Comment out transformer and outbound-channel-adapter.
    Change logging-channel-adapter   auto-startup="true"
    5. java -DLog4jContextSelector=org.apache.logging.log4j.core.async.AsyncLoggerContextSelector  -Dspring.expression.compiler.mode=MIXED com.xxx.Main
    Comment out transformer and outbound-channel-adapter.
    Change logging-channel-adapter   auto-startup="true"
    

    可以使用以下设置运行测试:

    {{1}}

1 个答案:

答案 0 :(得分:2)

感谢您的广泛分析。

说实话,APPEND模式是出站适配器的相对新增功能,尚未优化。

我怀疑成本仅仅是因为每次写入时使用流都关闭(使用FileCopy.copy()),这会刷新到磁盘。

我们绝对应该考虑保持BufferedOutputStream开放的选项。这有点棘手,因为适配器支持为每条消息写入不同的文件。我假设你的用例是你总是写入同一个文件,或一些基于时间戳的文件名。我们可以提供一些优化来保持文件打开,直到有不同文件的请求进入,或者甚至打开几个文件缓冲区。

但是,在某些情况下,如果在一段时间过后没有新消息到达,我们会想要刷新缓冲区。这增加了一些复杂性(但不是很多)。

当然,缺点是当您在内存中缓冲数据时,如果发生电源故障,则存在数据丢失的风险。这是一个经典的权衡 - 性能比。可靠性;现在这个适配器对后者不利。

与往常一样,随时可以打开JIRA问题,我们会一起来看看。