Question

我有一个文件系统目录，我想要轮询文件，然后同时处理每个文件，每个文件只有一个线程。我的印象是，在封面下，InboundFileAdapter将每个文件放入队列中，因此我可以使用下游的执行程序通道来同时处理以后的调用。我在Java Config中实现了如下：

return IntegrationFlows
            .from(s -> s.file(inboundMessageDirectory.toFile(), Comparator.comparing(File::lastModified)) // serve oldest first
                            .scanner(directoryScanner) // we know the directory structure, so we can take advantage of that with a custom scanner
                            .filter(new AcceptOnceFileListFilter<>(MAX_FILTER_CAPACITY)), // limit number of references in memory
                    e -> e.poller(Pollers
                            .fixedDelay(fileSystemPollDelay)
                            .get()))
            .channel(MessageChannels.executor(executor).get())
            .transform(File::toPath)
            .enrichHeaders(cleanUpConfigurer)
            .get()

执行程序通道下游的每个通道本身都是直接通道。

但是，我看到下游服务中的并发性很差。使用缓存的线程池，我看到同一个线程基本上是串行执行下游代码，而如果我使用固定的池执行器，我会看到不同的线程因为仍然是串行执行而进行权衡。

我也试过在轮询器和执行器通道之间建立桥梁，但无济于事。

Answer 1

That's just because of SourcePollingChannelAdapterFactoryBean under hood: if (this.pollerMetadata.getMaxMessagesPerPoll() == Integer.MIN_VALUE){ // the default is 1 since a source might return // a non-null and non-interruptible value every time it is invoked this.pollerMetadata.setMaxMessagesPerPoll(1); } So, each your .fixedDelay(fileSystemPollDelay) only one File is polled from the queue to process. So, just increase .maxMessagesPerPoll() to some appropriate for your system value and have fun with the concurrency! BTW, there is no reason to introduce ExecutorChannel just after the polling adapter. You simply can use .taskExecutor() for the .poller() exactly for the same concurrency reason.

使用spring入站文件适配器进行并发处理

1 个答案: