Spring集成:如何按顺序处理文件

时间:2015-12-17 11:03:12

标签: spring-integration

我使用" int-file:inbound-channel-adapter"加载存在于目录中的文件。我喜欢按顺序处理文件:这意味着当第一个文件的处理完成后,我加载第二个文件......等等。

我看到sample,但我无法预测处理一个文件所需的时间,这取决于其大小。

我的源代码:

    <int-file:inbound-channel-adapter
    directory="${directory.files.local}" id="filesIn" channel="channel.filesIn">
    <int:poller fixed-delay="1000" max-messages-per-poll="1" />

</int-file:inbound-channel-adapter>

文件的过程是文件:inbound-channel-adapter ---&gt; transformer ---&gt; splitter ----&gt; http:outbound-gateway ---&gt; outbound-mail-adapter- ---&gt;文件处理完成,所以此时,我要处理下一个文件。

我的项目配置太复杂了。 下面,我向您展示更多配置: 配置的第一部分是:

<int-sftp:inbound-channel-adapter id="sftpInbondAdapter"
    auto-startup="true" channel="receiveChannel" session-factory="sftpSessionFactory"
    local-directory="file:${directory.files.local}" remote-directory="${directory.files.remote}"
    auto-create-local-directory="true" delete-remote-files="true"
    filename-regex=".*\.txt$">
    <int:poller fixed-delay="${sftp.interval.request}"
        max-messages-per-poll="-1" />
</int-sftp:inbound-channel-adapter>
<!-- <int:poller cron="0 * 17 * * ?"></int:poller> -->

<int-file:inbound-channel-adapter
    filter="compositeFileFilter" directory="${directory.files.local}" id="filesIn"
    channel="channel.filesIn" prevent-duplicates="true">
    <int:poller fixed-delay="1000" max-messages-per-poll="1" />
</int-file:inbound-channel-adapter>

<int:transformer input-channel="channel.filesIn"
    output-channel="channel.file.router" ref="fileTransformer" method="transform" />

<int:recipient-list-router id="fileRouter"
    input-channel="channel.file.router">

    <int:recipient channel="channel.empty.files"
        selector-expression="payload.length()==0" />
    <int:recipient channel="channel.filesRejected"
        selector-expression="payload.toString().contains('rejected')" />
    <int:recipient channel="toSplitter"
        selector-expression="(payload.length()>0) and(!payload.toString().contains('rejected'))" />

</int:recipient-list-router>

然后从通道tosplitter,我的程序逐行读取文件:

    <int-file:splitter input-channel="toSplitter"
    output-channel="router" requires-reply="false" />

<int:recipient-list-router id="recipentRouter"
    input-channel="router">

    <int:recipient channel="channelA"
        selector-expression="headers['file_name'].startsWith('${filenameA.prefix}')" />

    <int:recipient channel="channelB"
        selector-expression="headers['file_name'].startsWith('${filenameB.prefix}')" />

</int:recipient-list-router>

每个通道A和B应为每一行调用两个不同的WS。 每个文件对文件A使用asynch调用ws代码:

<int:header-enricher input-channel="channelA"
    output-channel="channelA.withHeader">
    <int:header name="content-type" value="application/json" />
    <int:header name="key1" expression="payload.split('${line.column.separator}')[0]" />
    <int:header name="key2" expression="payload"></int:header>
</int:header-enricher>

<int:transformer input-channel="channelA.withHeader"
    output-channel="channelA.request" ref="imsiMsgTransformer"
    method="transform">
</int:transformer>


<int:channel id="channelA.request">
    <int:queue capacity="10" />

<int-http:outbound-gateway id="maspUpdatorSimChangedGateway"
    request-channel="channelA.request" 
    url="${url}"
    http-method="PUT" expected-response-type="java.lang.String"       charset="UTF-8"
    reply-timeout="${ws.reply.timeout}" reply-channel="channelA.reply">
    <int-http:uri-variable name="foo" expression="headers['key1']" />
    <int:poller fixed-delay="1000" error-channel="channelA.error"
        task-executor="executorA" />

        <int-http:request-handler-advice-chain>
        <int:retry-advice max-attempts="${ws.max.attempts}"
            recovery-channel="recovery.channelA">
            <int:fixed-back-off interval="${ws.interval.attempts}" />
        </int:retry-advice>
    </int-http:request-handler-advice-chain>

</int-http:outbound-gateway>

<int:service-activator input-channel="recovery.channelA"
    ref="updateImsiHttpResponseErrorHandler" method="handleMessage" output-channel="updateImsi.channel.error.toenricher">
</int:service-activator>

<int:service-activator input-channel="channelA.reply"
    ref="updateImsiHttpResponseMessageHandler" method="handleMessage">
    <int:poller fixed-delay="1000"></int:poller>
</int:service-activator>

在(回复频道和恢复频道)的每个激活器中,我计算文件的进度,直到文件完成此时我应该加载第二个文件A2或文件B ......等等。

1 个答案:

答案 0 :(得分:1)

这是默认行为,只要

  1. 投票人没有task-executor(你的没有)。
  2. 在适配器的下游仅使用DirectChannel s(默认值) - 这表示没有QueueChannelExecutorChannel(即没有task-executor<queue/>在频道上。)
  3. 在这种情况下,直到当前轮询结束时才会考虑下一个轮询 - 流程在轮询器线程上运行,并且一次只能进行一次轮询。

    在完全处理当前文件之前,fixed-delay才会启动。

    修改

    如果您需要对流使用异步处理,则需要使用Conditional Poller或简单PollSkipAdvice

    您将提供一个PollSkipStrategy实现,在文件完成之前将返回false。

    这样,随后的民意调查将被跳过,直到你决定。

    <强> EDIT2

    像这样......

    /*
     * Copyright 2015 the original author or authors.
     *
     * Licensed under the Apache License, Version 2.0 (the "License");
     * you may not use this file except in compliance with the License.
     * You may obtain a copy of the License at
     *
     *      http://www.apache.org/licenses/LICENSE-2.0
     *
     * Unless required by applicable law or agreed to in writing, software
     * distributed under the License is distributed on an "AS IS" BASIS,
     * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
     * See the License for the specific language governing permissions and
     * limitations under the License.
     */
    package org.springframework.integration.scheduling;
    
    /**
     * @author Gary Russell
     * @since 4.3
     *
     */
    public class SimplePollSkipStrategy implements PollSkipStrategy {
    
        private volatile boolean skip;
    
        @Override
        public boolean skipPoll() {
            return this.skip;
        }
    
        public void skipPolls() {
            this.skip = true;
        }
    
        public void reset() {
            this.skip = false;
        }
    }
    
    • 将其添加为<bean/>到您的上下文。
    • 使用PollSkipAdvice
    • 将其添加到轮询器的建议链中
    • 如果您想跳过民意调查,请致电skipPolls()
    • 完成文件后,请致电reset()