'TCP health check failed' the last 6 days

Troubleshooting and help
,
1

Hi everyone,
I’ve had a problem with my deployments from the past 6 days.
i’m trying to redeploy my web service (Kotlin, spring boot & spring security)

i can build, launch & access to it in local.
this is my local container log :

INFO  n.n.Nico2cambackApplicationKt            : Starting Nico2cambackApplicationKt using Java 18.0.2.1 with PID 360215 (/home/nscherman/perso/nico2cam/nico2camback/target/classes started by nscherman in /home/nscherman/perso/nico2cam/nico2camback)
INFO  n.n.Nico2cambackApplicationKt            : The following 1 profile is active: "local"
INFO  .s.d.r.c.RepositoryConfigurationDelegate : Bootstrapping Spring Data JDBC repositories in DEFAULT mode.
INFO  .s.d.r.c.RepositoryConfigurationDelegate : Finished Spring Data repository scanning in 57 ms. Found 0 JDBC repository interfaces.
INFO  o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat initialized with port 8080 (http)
INFO  o.apache.catalina.core.StandardService   : Starting service [Tomcat]
INFO  o.apache.catalina.core.StandardEngine    : Starting Servlet engine: [Apache Tomcat/10.1.19]
INFO  o.a.c.c.C.[Tomcat].[localhost].[/]       : Initializing Spring embedded WebApplicationContext
INFO  w.s.c.ServletWebServerApplicationContext : Root WebApplicationContext: initialization completed in 2522 ms
INFO  com.zaxxer.hikari.HikariDataSource       : HikariPool-1 - Starting...
INFO  com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Added connection org.postgresql.jdbc.PgConnection@42c12b3e
INFO  com.zaxxer.hikari.HikariDataSource       : HikariPool-1 - Start completed.
INFO  liquibase.database                       : Set default schema name to public
INFO  liquibase.changelog                      : Reading from public.databasechangelog
INFO  liquibase.ui                             : Database is up to date, no changesets to execute
INFO  liquibase.changelog                      : Reading from public.databasechangelog
INFO  liquibase.util                           : UPDATE SUMMARY
INFO  liquibase.util                           : Run:                          0
INFO  liquibase.util                           : Previously run:               2
INFO  liquibase.util                           : Filtered out:                 0
INFO  liquibase.util                           : -------------------------------
INFO  liquibase.util                           : Total change sets:            2
INFO  liquibase.util                           : Update summary generated
INFO  liquibase.command                        : Command execution complete
INFO  o.s.s.web.DefaultSecurityFilterChain     : Will secure any request with [org.springframework.security.web.session.DisableEncodeUrlFilter@446cc036, org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter@33b2ba25, org.springframework.security.web.context.SecurityContextHolderFilter@796642ae, org.springframework.security.web.header.HeaderWriterFilter@20a9fb01, org.springframework.web.filter.CorsFilter@6ad198c9, org.springframework.security.web.authentication.logout.LogoutFilter@6e7f29d5, nico2cam.nico2camback.infrastrucure.security.JwtAuthenticationFilter@5d6b42cf, org.springframework.security.web.savedrequest.RequestCacheAwareFilter@6f0ae0ee, org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter@6de9bba2, org.springframework.security.web.authentication.AnonymousAuthenticationFilter@481bae3f, org.springframework.security.web.access.ExceptionTranslationFilter@266a47fd, org.springframework.security.web.access.intercept.AuthorizationFilter@4cb773c5]
INFO  o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat started on port 8080 (http) with context path ''
INFO  n.n.Nico2cambackApplicationKt            : Started Nico2cambackApplicationKt in 13.171 seconds (process running for 14.119)

this is the log on koyeb :

INFO  n.n.Nico2cambackApplicationKt            : Starting Nico2cambackApplicationKt v0.0.1-SNAPSHOT using Java 17-ea with PID 1 (/app.jar started by root in /)
INFO  n.n.Nico2cambackApplicationKt            : The following 1 profile is active: "prod"
INFO  .s.d.r.c.RepositoryConfigurationDelegate : Bootstrapping Spring Data JDBC repositories in DEFAULT mode.
INFO  .s.d.r.c.RepositoryConfigurationDelegate : Finished Spring Data repository scanning in 33 ms. Found 0 JDBC repository interfaces.
INFO  o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat initialized with port 8070 (http)
INFO  o.apache.catalina.core.StandardService   : Starting service [Tomcat]
INFO  o.apache.catalina.core.StandardEngine    : Starting Servlet engine: [Apache Tomcat/10.1.19]
INFO  o.a.c.c.C.[Tomcat].[localhost].[/]       : Initializing Spring embedded WebApplicationContext
INFO  w.s.c.ServletWebServerApplicationContext : Root WebApplicationContext: initialization completed in 3030 ms
INFO  com.zaxxer.hikari.HikariDataSource       : HikariPool-1 - Starting...
INFO  com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Added connection org.postgresql.jdbc.PgConnection@6d617b07
INFO  com.zaxxer.hikari.HikariDataSource       : HikariPool-1 - Start completed.
INFO  liquibase.database                       : Set default schema name to public
INFO  liquibase.changelog                      : Reading from public.databasechangelog
INFO  liquibase.ui                             : Database is up to date, no changesets to execute
INFO  liquibase.changelog                      : Reading from public.databasechangelog
INFO  liquibase.util                           : UPDATE SUMMARY
INFO  liquibase.util                           : Run:                          0
INFO  liquibase.util                           : Previously run:               2
INFO  liquibase.util                           : Filtered out:                 0
INFO  liquibase.util                           : -------------------------------
INFO  liquibase.util                           : Total change sets:            2
INFO  liquibase.util                           : Update summary generated
INFO  liquibase.command                        : Command execution complete
TCP health check failed on port 8070. Retrying...

The difference between the logs is the non-appearance of these 2 logs:

INFO  o.s.s.web.DefaultSecurityFilterChain     : Will secure any request with [org.springframework.security.web.session.DisableEncodeUrlFilter@446cc036, org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter@33b2ba25, org.springframework.security.web.context.SecurityContextHolderFilter@796642ae, org.springframework.security.web.header.HeaderWriterFilter@20a9fb01, org.springframework.web.filter.CorsFilter@6ad198c9, org.springframework.security.web.authentication.logout.LogoutFilter@6e7f29d5, nico2cam.nico2camback.infrastrucure.security.JwtAuthenticationFilter@5d6b42cf, org.springframework.security.web.savedrequest.RequestCacheAwareFilter@6f0ae0ee, org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter@6de9bba2, org.springframework.security.web.authentication.AnonymousAuthenticationFilter@481bae3f, org.springframework.security.web.access.ExceptionTranslationFilter@266a47fd, org.springframework.security.web.access.intercept.AuthorizationFilter@4cb773c5]
INFO  o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat started on port 8080 (http) with context path ''
INFO  n.n.Nico2cambackApplicationKt            : Started Nico2cambackApplicationKt in 13.171 seconds (process running for 14.119)

I already :

  • try to increase the Health checks grace period from 5 seconds to 120 seconds. The same error appears.
  • try to deploy an older and health docker image. The same error appears.
  • destroyed my service, and when I recreated it, log NFO o.s.s.web.DefaultSecurityFilterChain reappeared but TCP health check failed on port 8070. is still present.

how can i unlock, do you need more information ?

2

Hi!

There is also this event added to your logs:

Application exited with code 9. This usually occurs when the Instance runs out of memory (OOM). Upgrade your Service to a larger Instance to mitigate this issue.

Did you try running your service on a bigger instance than eNano?

4

that was indeed the problem. I’m curious, though: why did the old images, where everything was fine, no longer work?