Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Application fails to start when server.tomcat.threads.max < 10 #40945

Closed
fabian-froehlich opened this issue May 29, 2024 · 5 comments
Closed
Assignees
Labels
type: regression A regression from a previous release
Milestone

Comments

@fabian-froehlich
Copy link

fabian-froehlich commented May 29, 2024

Since spring-boot 3.3.0 our application fails to start with a similar exception as below. Increasing the max threads of tomcat from 5 to 10 resolves that exception.
The problem occurs as soon as there are under 10 threads configured.
spring-boot 3.2.6 is not affected.
Both versions (3.2.6 and 3.3.0) use the same tomcat version (10.1.24).

Steps to reproduce:

  1. Create basic Spring-Boot Project, that uses Apache Tomcat as we Web-Server (https://start.spring.io/ + spring-boot-starter-web dependency)
  2. Set server.tomcat.threads.max=9
  3. Application fails to start
2024-05-29T17:21:30.629+02:00  INFO 22712 --- [demo] [           main] com.example.demo.DemoApplication         : Starting DemoApplication using Java 21.0.3 with ...
2024-05-29T17:21:30.631+02:00  INFO 22712 --- [demo] [           main] com.example.demo.DemoApplication         : No active profile set, falling back to 1 default profile: "default"
2024-05-29T17:21:31.285+02:00  INFO 22712 --- [demo] [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat initialized with port 8080 (http)
2024-05-29T17:21:31.295+02:00  INFO 22712 --- [demo] [           main] o.apache.catalina.core.StandardService   : Starting service [Tomcat]
2024-05-29T17:21:31.295+02:00  INFO 22712 --- [demo] [           main] o.apache.catalina.core.StandardEngine    : Starting Servlet engine: [Apache Tomcat/10.1.24]
2024-05-29T17:21:31.348+02:00  INFO 22712 --- [demo] [           main] o.a.c.c.C.[Tomcat].[localhost].[/]       : Initializing Spring embedded WebApplicationContext
2024-05-29T17:21:31.349+02:00  INFO 22712 --- [demo] [           main] w.s.c.ServletWebServerApplicationContext : Root WebApplicationContext: initialization completed in 674 ms
2024-05-29T17:21:31.383+02:00  INFO 22712 --- [demo] [           main] o.apache.catalina.core.StandardService   : Stopping service [Tomcat]
2024-05-29T17:21:31.388+02:00  WARN 22712 --- [demo] [           main] ConfigServletWebServerApplicationContext : Exception encountered during context initialization - cancelling refresh attempt: org.springframework.context.ApplicationContextException: Unable to start web server
2024-05-29T17:21:31.394+02:00  INFO 22712 --- [demo] [           main] .s.b.a.l.ConditionEvaluationReportLogger : 

Error starting ApplicationContext. To display the condition evaluation report re-run your application with 'debug' enabled.
2024-05-29T17:21:31.408+02:00 ERROR 22712 --- [demo] [           main] o.s.boot.SpringApplication               : Application run failed

org.springframework.context.ApplicationContextException: Unable to start web server
	at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.onRefresh(ServletWebServerApplicationContext.java:165) ~[spring-boot-3.3.0.jar:3.3.0]
	at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:618) ~[spring-context-6.1.8.jar:6.1.8]
	at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:146) ~[spring-boot-3.3.0.jar:3.3.0]
	at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:754) ~[spring-boot-3.3.0.jar:3.3.0]
	at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:456) ~[spring-boot-3.3.0.jar:3.3.0]
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:335) ~[spring-boot-3.3.0.jar:3.3.0]
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1363) ~[spring-boot-3.3.0.jar:3.3.0]
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1352) ~[spring-boot-3.3.0.jar:3.3.0]
	at com.example.demo.DemoApplication.main(DemoApplication.java:12) ~[classes/:na]
Caused by: org.springframework.boot.web.server.WebServerException: Unable to start embedded Tomcat
	at org.springframework.boot.web.embedded.tomcat.TomcatWebServer.initialize(TomcatWebServer.java:147) ~[spring-boot-3.3.0.jar:3.3.0]
	at org.springframework.boot.web.embedded.tomcat.TomcatWebServer.<init>(TomcatWebServer.java:107) ~[spring-boot-3.3.0.jar:3.3.0]
	at org.springframework.boot.web.embedded.tomcat.TomcatServletWebServerFactory.getTomcatWebServer(TomcatServletWebServerFactory.java:516) ~[spring-boot-3.3.0.jar:3.3.0]
	at org.springframework.boot.web.embedded.tomcat.TomcatServletWebServerFactory.getWebServer(TomcatServletWebServerFactory.java:222) ~[spring-boot-3.3.0.jar:3.3.0]
	at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.createWebServer(ServletWebServerApplicationContext.java:188) ~[spring-boot-3.3.0.jar:3.3.0]
	at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.onRefresh(ServletWebServerApplicationContext.java:162) ~[spring-boot-3.3.0.jar:3.3.0]
	... 8 common frames omitted
Caused by: org.apache.catalina.LifecycleException: Failed to start component [org.apache.catalina.core.StandardThreadExecutor@6c931d35]
	at org.apache.catalina.util.LifecycleBase.handleSubClassException(LifecycleBase.java:419) ~[tomcat-embed-core-10.1.24.jar:10.1.24]
	at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:186) ~[tomcat-embed-core-10.1.24.jar:10.1.24]
	at org.apache.catalina.core.StandardService.startInternal(StandardService.java:419) ~[tomcat-embed-core-10.1.24.jar:10.1.24]
	at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:171) ~[tomcat-embed-core-10.1.24.jar:10.1.24]
	at org.apache.catalina.core.StandardServer.startInternal(StandardServer.java:874) ~[tomcat-embed-core-10.1.24.jar:10.1.24]
	at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:171) ~[tomcat-embed-core-10.1.24.jar:10.1.24]
	at org.apache.catalina.startup.Tomcat.start(Tomcat.java:437) ~[tomcat-embed-core-10.1.24.jar:10.1.24]
	at org.springframework.boot.web.embedded.tomcat.TomcatWebServer.initialize(TomcatWebServer.java:128) ~[spring-boot-3.3.0.jar:3.3.0]
	... 13 common frames omitted
Caused by: java.lang.IllegalArgumentException: null
	at org.apache.tomcat.util.threads.ThreadPoolExecutor.<init>(ThreadPoolExecutor.java:1357) ~[tomcat-embed-core-10.1.24.jar:10.1.24]
	at org.apache.tomcat.util.threads.ThreadPoolExecutor.<init>(ThreadPoolExecutor.java:1279) ~[tomcat-embed-core-10.1.24.jar:10.1.24]
	at org.apache.catalina.core.StandardThreadExecutor.startInternal(StandardThreadExecutor.java:117) ~[tomcat-embed-core-10.1.24.jar:10.1.24]
	at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:171) ~[tomcat-embed-core-10.1.24.jar:10.1.24]
	... 19 common frames omitted
@spring-projects-issues spring-projects-issues added the status: waiting-for-triage An issue we've not yet triaged label May 29, 2024
@fabian-froehlich fabian-froehlich changed the title Application fails to start when server.tomcat.threads.max below 10 Application fails to start when server.tomcat.threads.max < 10 May 29, 2024
@wilkinsona wilkinsona self-assigned this May 29, 2024
@wilkinsona
Copy link
Member

Thanks for the report. The change in behavior is due to an unanticipated side-effect of #36087.

The failure's occurring because the maximum number of threads is less than the minimum number of spare threads. You can avoid it by setting server.tomcat.threads.min-spare=9.

This is necessary with Spring Boot 3.3.0 as Tomcat's no longer creating its executor. When it does create the executor, it calls org.apache.tomcat.util.net.AbstractEndpoint.getMinSpareThreads() which does Math.min(getMinSpareThreadsInternal(), getMaxThreads()).

I'm not sure if we should add similar logic to our creation of the executor. Arguably, having the max threads set to something that's less than min-spare threads is a configuration error and I'm not 100% sure that silently adapting to it is the right thing to do. The counter-argument is that it used to work and it's what Tomcat does by default so we should probably reinstate the old behavior.

@wilkinsona wilkinsona added the for: team-meeting An issue we'd like to discuss as a team to make progress label May 29, 2024
@fabian-froehlich
Copy link
Author

@wilkinsona Thanks, your suggestion works. Our application starts again.

I would prefer a solution that either

  • makes the application startable after upgrading the dependency right away
    or
  • add a paragraph in the releasenotes / migration guide

@philwebb philwebb added type: regression A regression from a previous release and removed status: waiting-for-triage An issue we've not yet triaged for: team-meeting An issue we'd like to discuss as a team to make progress labels Jun 6, 2024
@philwebb philwebb added this to the 3.3.x milestone Jun 6, 2024
@philwebb
Copy link
Member

philwebb commented Jun 6, 2024

We're going to align with Tomcat default behavior.

@mhalbritter
Copy link
Contributor

I've opened https://bz.apache.org/bugzilla/show_bug.cgi?id=69133 - if this is resolved, we can switch back to the default executor and set the queue size for #36087 via a normal setter.

@wilkinsona wilkinsona removed their assignment Jun 12, 2024
@wilkinsona wilkinsona added the status: on-hold We can't start working on this issue yet label Jun 12, 2024
@mhalbritter
Copy link
Contributor

Tomcat has added a setter to set the maximum queue size. When using this setter, this issue should go away. I've opened #41093 for that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: regression A regression from a previous release
Projects
None yet
Development

No branches or pull requests

5 participants