为什么Spring Webflux仅并行接受256个请求？

时间：2018-10-23 09:36:55

标签： spring-boot netty spring-webflux reactor

在默认配置中，Spring Webflux似乎将并行请求的数量限制为256。

我的设置有一个非常简单的控制器：

@RestController
public class SimpleRestController {
  private final Log logger = LogFactory.getLog(getClass());

  private AtomicLong countEnter = new AtomicLong(0);
  private AtomicLong countExit = new AtomicLong(0);

  @GetMapping(value = "/delayed")
  public Mono<String> delayed() {
    logger.info("delayed ENTER " + countEnter.incrementAndGet());

    return Mono.just("result").delayElement(Duration.ofSeconds(60))

          .doOnNext(s -> logger.info("delayed EXIT " + countExit.incrementAndGet()));
  }
}

该配置仅启用WebFlux：

@SpringBootConfiguration
@EnableWebFlux
public class SearchServiceConfiguration {
}

依赖项很少：

  <properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
    <java.version>10</java.version>
  </properties>

  <dependencies>
    <dependency>
      <groupId>org.springframework.boot</groupId>
      <artifactId>spring-boot-starter-webflux</artifactId>
    </dependency>

    <dependency>
      <groupId>org.projectlombok</groupId>
      <artifactId>lombok</artifactId>
      <optional>true</optional>
    </dependency>
  </dependencies>

  <build>
    <plugins>
      <plugin>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-maven-plugin</artifactId>
      </plugin>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-compiler-plugin</artifactId>
        <version>3.8.0</version>
        <configuration>
          <source>10</source>
          <target>10</target>
          <encoding>UTF-8</encoding>
        </configuration>
      </plugin>
    </plugins>
  </build>

我用此脚本并行调用端点：

#!/usr/bin/env bash

for p in `seq 1 1000`;
do
    curl -s http://localhost:8080/delayed &
done

wait
echo "All done"

对于第257个请求，返回Resource temporarily unavailable。

哪个设置限制了并行请求的数量？

这是产生的日志输出：

2018-10-26 14:27:27.253  INFO 23728 --- [           main] s.w.r.r.m.a.RequestMappingHandlerMapping : Mapped "{[/sleep],methods=[GET],produces=[application/stream+json]}" onto public java.lang.String controller.SimpleRestController.sleep() throws java.lang.InterruptedException
2018-10-26 14:27:27.253  INFO 23728 --- [           main] s.w.r.r.m.a.RequestMappingHandlerMapping : Mapped "{[/delayed],methods=[GET]}" onto public reactor.core.publisher.Mono<java.lang.String> controller.SimpleRestController.delayed()
2018-10-26 14:27:27.315  INFO 23728 --- [           main] o.s.w.r.r.m.a.ControllerMethodResolver   : Looking for @ControllerAdvice: org.springframework.boot.web.reactive.context.AnnotationConfigReactiveWebServerApplicationContext@3cdf2c61: startup date [Fri Oct 26 14:27:26 CEST 2018]; root of context hierarchy
2018-10-26 14:27:27.639  INFO 23728 --- [           main] o.s.j.e.a.AnnotationMBeanExporter        : Registering beans for JMX exposure on startup
2018-10-26 14:27:27.938  INFO 23728 --- [ctor-http-nio-1] r.ipc.netty.tcp.BlockingNettyContext     : Started HttpServer on /0:0:0:0:0:0:0:0:8080
2018-10-26 14:27:27.939  INFO 23728 --- [           main] o.s.b.web.embedded.netty.NettyWebServer  : Netty started on port(s): 8080
2018-10-26 14:27:27.943  INFO 23728 --- [           main] d.h.s.s.SimpleApplication                : Started SimpleApplication in 1.548 seconds (JVM running for 2.396)
2018-10-26 14:27:34.295  INFO 23728 --- [ctor-http-nio-5] d.h.s.s.controller.SimpleRestController  : delayed ENTER 2
2018-10-26 14:27:34.295  INFO 23728 --- [ctor-http-nio-3] d.h.s.s.controller.SimpleRestController  : delayed ENTER 1
2018-10-26 14:27:34.295  INFO 23728 --- [ctor-http-nio-4] d.h.s.s.controller.SimpleRestController  : delayed ENTER 4
2018-10-26 14:27:34.295  INFO 23728 --- [ctor-http-nio-2] d.h.s.s.controller.SimpleRestController  : delayed ENTER 3
2018-10-26 14:27:34.324  INFO 23728 --- [ctor-http-nio-6] d.h.s.s.controller.SimpleRestController  : delayed ENTER 5
2018-10-26 14:27:34.351  INFO 23728 --- [ctor-http-nio-7] d.h.s.s.controller.SimpleRestController  : delayed ENTER 6
2018-10-26 14:27:34.381  INFO 23728 --- [ctor-http-nio-8] d.h.s.s.controller.SimpleRestController  : delayed ENTER 7
2018-10-26 14:27:34.413  INFO 23728 --- [ctor-http-nio-1] d.h.s.s.controller.SimpleRestController  : delayed ENTER 8
2018-10-26 14:27:34.438  INFO 23728 --- [ctor-http-nio-2] d.h.s.s.controller.SimpleRestController  : delayed ENTER 9
2018-10-26 14:27:34.466  INFO 23728 --- [ctor-http-nio-3] d.h.s.s.controller.SimpleRestController  : delayed ENTER 10
2018-10-26 14:27:34.497  INFO 23728 --- [ctor-http-nio-4] d.h.s.s.controller.SimpleRestController  : delayed ENTER 11
2018-10-26 14:27:34.526  INFO 23728 --- [ctor-http-nio-5] d.h.s.s.controller.SimpleRestController  : delayed ENTER 12
2018-10-26 14:27:34.561  INFO 23728 --- [ctor-http-nio-6] d.h.s.s.controller.SimpleRestController  : delayed ENTER 13
2018-10-26 14:27:34.594  INFO 23728 --- [ctor-http-nio-7] d.h.s.s.controller.SimpleRestController  : delayed ENTER 14
...
2018-10-26 14:27:42.675  INFO 23728 --- [ctor-http-nio-6] d.h.s.s.controller.SimpleRestController  : delayed ENTER 253
2018-10-26 14:27:42.711  INFO 23728 --- [ctor-http-nio-7] d.h.s.s.controller.SimpleRestController  : delayed ENTER 254
2018-10-26 14:27:42.745  INFO 23728 --- [ctor-http-nio-8] d.h.s.s.controller.SimpleRestController  : delayed ENTER 255
2018-10-26 14:27:42.774  INFO 23728 --- [ctor-http-nio-1] d.h.s.s.controller.SimpleRestController  : delayed ENTER 256
2018-10-26 14:28:04.312  INFO 23728 --- [     parallel-3] d.h.s.s.controller.SimpleRestController  : delayed EXIT 1
2018-10-26 14:28:04.312  INFO 23728 --- [     parallel-2] d.h.s.s.controller.SimpleRestController  : delayed EXIT 2
2018-10-26 14:28:04.321  INFO 23728 --- [     parallel-2] d.h.s.s.controller.SimpleRestController  : delayed EXIT 3
2018-10-26 14:28:04.321  INFO 23728 --- [     parallel-2] d.h.s.s.controller.SimpleRestController  : delayed EXIT 4
2018-10-26 14:28:04.333  INFO 23728 --- [     parallel-4] d.h.s.s.controller.SimpleRestController  : delayed EXIT 5
2018-10-26 14:28:04.359  INFO 23728 --- [     parallel-5] d.h.s.s.controller.SimpleRestController  : delayed EXIT 6
2018-10-26 14:28:04.389  INFO 23728 --- [     parallel-6] d.h.s.s.controller.SimpleRestController  : delayed EXIT 7
2018-10-26 14:28:04.421  INFO 23728 --- [     parallel-7] d.h.s.s.controller.SimpleRestController  : delayed EXIT 8
2018-10-26 14:28:04.446  INFO 23728 --- [     parallel-8] d.h.s.s.controller.SimpleRestController  : delayed EXIT 9
2018-10-26 14:28:04.474  INFO 23728 --- [     parallel-1] d.h.s.s.controller.SimpleRestController  : delayed EXIT 10
2018-10-26 14:28:04.505  INFO 23728 --- [     parallel-2] d.h.s.s.controller.SimpleRestController  : delayed EXIT 11
...
2018-10-26 14:28:12.570  INFO 23728 --- [     parallel-1] d.h.s.s.controller.SimpleRestController  : delayed EXIT 250
2018-10-26 14:28:12.607  INFO 23728 --- [     parallel-2] d.h.s.s.controller.SimpleRestController  : delayed EXIT 251
2018-10-26 14:28:12.643  INFO 23728 --- [     parallel-3] d.h.s.s.controller.SimpleRestController  : delayed EXIT 252
2018-10-26 14:28:12.683  INFO 23728 --- [     parallel-4] d.h.s.s.controller.SimpleRestController  : delayed EXIT 253
2018-10-26 14:28:12.719  INFO 23728 --- [     parallel-5] d.h.s.s.controller.SimpleRestController  : delayed EXIT 254
2018-10-26 14:28:12.752  INFO 23728 --- [     parallel-6] d.h.s.s.controller.SimpleRestController  : delayed EXIT 255
2018-10-26 14:28:12.782  INFO 23728 --- [     parallel-7] d.h.s.s.controller.SimpleRestController  : delayed EXIT 256

2 个答案:

答案 0 :(得分：6)

这不是我在本地设置中看到的内容。将您的控制器更改为以下内容会打印“ onNext 1”到“ onNext 1000”：

textvalue

在您的评论之一中，您提到AtomicLong count = new AtomicLong(0); @GetMapping("/delayed") public Mono<String> delayed() { return Mono.just("result").delayElement(Duration.ofSeconds(30)) .doOnNext(s -> { logger.info("onNext " + count.incrementAndGet()); }); } ，它是Servlet特定的。这告诉我您不是运行Spring WebFlux应用程序，而是实际上是Spring MVC应用程序。如果您正在运行Spring Boot，并且在类路径上有Spring MVC，则Spring Boot will configure a Spring MVC application for you。

默认情况下，Spring MVC将使用CommonsRequestLoggingFilter来调度那些请求的异步处理（Spring MVC支持反应性返回类型，但会将它们视为SimpleAsyncTaskExecutor请求，asynchronous but not non-blocking）。我认为@Async默认没有并发限制。也许您在应用程序中配置了将并发限制为256的任务执行器？

我尝试使用Spring MVC应用程序重现该问题，但仍然在日志中处理了999个请求和一个已解决的异常：

SimpleAsyncTaskExecutor

答案 1 :(得分：1)

服务器不是您的限制因素。该错误表明由于资源限制，创建后台进程失败。有几种可能的原因：

  • 内存-每个进程都需要内存来加载curl。
  • 进程-您可以创建的进程数可能会受到限制
  • 打开文件-每个进程打开多个“文件”。用引号引起来，因为“文件”包括网络连接和curl加载的动态库。一次最多可以打开多少个系统范围的限制。
相关问题
最新问题