Question

我试图使用Spring的Async异步调用API，并在我的Thread Config中使用ThreadPoolTaskExecutor：

@Configuration
@EnableAsync
public class ThreadConfig extends AsyncConfigurerSupport {

@Value("${core.pool.size}")
private int corePoolSize;

@Value("${max.pool.size}")
private int maxPoolSize;

@Value("${queue.capacity}")
private int queueCapacity;

@Override
@Bean
public Executor getAsyncExecutor() {

    ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
    executor.setCorePoolSize(corePoolSize);
    executor.setMaxPoolSize(maxPoolSize);
    executor.setQueueCapacity(queueCapacity);
    executor.setThreadNamePrefix("default_task_executor_thread");
    executor.initialize();

    return executor;

}

此处的设置为：

corePoolSize = 5;
maxPoolSize = 10;
QueueCapacity = 10;

我按如下方式调用异步服务：

for (String path : testList) {
    Future<Boolean> pro = services.invokeAPI(path);
}

testList有大约50条记录。

当我运行它时，编译器处理10个线程并调用invokeAPI方法10次，之后它给出：

org.springframework.core.task.TaskRejectedException: Executor[java.util.concurrent.ThreadPoolExecutor@3234ad78[Running, pool size = 10, active threads = 10, queued tasks = 10, completed tasks = 0]] did not accept task: org.springframework.aop.interceptor.AsyncExecutionInterceptor$1@5c17b70

我假设它将等待当前任务完成并重新分配线程而不是抛弃异常并终止程序。

如何让我的所有50条记录调用invokeAPI方法？

编辑：testList中的记录数可以更改。

有什么建议吗？

Answer 1

这种情况正在发生，因为您使用的是游泳池的大小。由于队列的大小为10，并且您可以拥有的最大线程数为10，因此在执行20个任务（10个运行和10个队列）后，执行程序开始拒绝任务。

有多种方法可以解决这个问题。

使用无界队列。即，不要指定队列的大小，因此它将能够保存所有任务。一旦线程空闲，任务就会被提交。
提供RejectedExecutionHandler，它可以完成任务。即在调用者线程上运行它们或丢弃它们或其他东西（取决于用例）。其中一些已由Java提供，如CallerRunsPolicy，AbortPolicy，DiscardPolicy和DiscardOldestPolicy。您可以使用executor#setRejectedExecutionHandler。
提供您自己的阻塞线程池执行程序，它会阻塞，直到有更多的任务空间（使用信号量）。

以下是阻止执行程序的示例

public class BlockingExecutor extends ThreadPoolExecutor {

    private final Semaphore semaphore;

    public BlockingExecutor(final int corePoolSize, final int poolSize, final int queueSize) {
        super(corePoolSize, poolSize, 0L, TimeUnit.MILLISECONDS, new LinkedBlockingQueue<>());
        semaphore = new Semaphore(poolSize + queueSize);
    }

    @Override
    public void execute(final Runnable task) {
        boolean acquired = false;
        do {
            try {
                semaphore.acquire();
                acquired = true;
            } catch (final InterruptedException e) {
                //do something here
            }
        } while (!acquired);

        try {
            super.execute(task);
        } catch (final RejectedExecutionException e) {
            semaphore.release();
            throw e;
        }
    }

    protected void afterExecute(final Runnable r, final Throwable t) {
        super.afterExecute(r, t);
        semaphore.release();
    }
}

Answer 2

一种解决方法是通过实施RejectedExecutionHandler策略，如下所示

import java.util.concurrent.RejectedExecutionException;
import java.util.concurrent.RejectedExecutionHandler;
import java.util.concurrent.ThreadPoolExecutor;

public class BlockCallerExecutionPolicy implements RejectedExecutionHandler {

    @Override
    public void rejectedExecution(Runnable r, ThreadPoolExecutor executor) {
        try {
            // based on the BlockingQueue documentation below should block until able to place on the queue... 
            executor.getQueue().put(r);
        }
        catch (InterruptedException e) {
            throw new RejectedExecutionException("Unexpected InterruptedException while waiting to add Runnable to ThreadPoolExecutor queue...", e);
        }
    }
}

此操作导致调用线程（可能是主线程）等待，直到阻塞队列中有空间为止。

Answer 3

嘿@Akshay Chopra，

根据@Shaize的回复：

import java.util.concurrent.RejectedExecutionException;
import java.util.concurrent.RejectedExecutionHandler;
import java.util.concurrent.ThreadPoolExecutor;

public class RejectedExecutionHandlerImpl implements RejectedExecutionHandler {

    @Override
    public void rejectedExecution(Runnable r, ThreadPoolExecutor executor) {
        try {
            executor.getQueue().put(r);
        }
        catch (InterruptedException e) {
            throw new RejectedExecutionException("There was an unexpected InterruptedException whilst waiting to add a Runnable in the executor's blocking queue", e);
        }
    }
}

为了拥有不带TaskRejectedException的多线程功能，您必须实现TaskRejectedHnadler，如下所示：

@Configuration
@EnableAsync
public class ThreadConfig extends AsyncConfigurerSupport {

    @Value("${core.pool.size}")
    private int corePoolSize;

    @Value("${max.pool.size}")
    private int maxPoolSize;

    @Value("${queue.capacity}")
    private int queueCapacity;

    @Override
    @Bean
    public Executor getAsyncExecutor() {

        ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
        executor.setCorePoolSize(corePoolSize);
        executor.setMaxPoolSize(maxPoolSize);
        executor.setQueueCapacity(queueCapacity);
        executor.setThreadNamePrefix("default_task_executor_thread");

        // add a rejected execution handler
        executor.setRejectedExecutionHandler(new RejectedExecutionHandlerImpl());

        executor.initialize();

        return executor;
    }
}

另一种方法是将反应性 FLUX 与同步方法而不是异步一起使用：

Flux.just(dtos.toArray()) // in my case an ArrayList
    .parallel(corePoolSize) // 8 in my case
    .runOn(Schedulers.parallel())
    .subscribe(dto -> computeService.compute((CastYourObject) dto));

您已经完成。

最好。

ThreadPoolTaskExecutor中的TaskRejectedException

3 个答案:

ThreadPoolTask​​Executor中的TaskRejectedException

3 个答案:

ThreadPoolTaskExecutor中的TaskRejectedException