问题
I have a springboot application which connects to a postgres DB.Connection pool used is Hikari. I have a select query running every 5 minutes.Though it gets executed properly most of the times, i encounter a PSQLException intermittently
org.postgresql.util.PSQLException: An I/O error occurred while sending to the backend.
at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:333) ~[postgresql-42.2.2.jar!/:42.2.2]
at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:441) ~[postgresql-42.2.2.jar!/:42.2.2]
at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:365) ~[postgresql-42.2.2.jar!/:42.2.2]
at org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:155) ~[postgresql-42.2.2.jar!/:42.2.2]
at org.postgresql.jdbc.PgPreparedStatement.executeUpdate(PgPreparedStatement.java:132) ~[postgresql-42.2.2.jar!/:42.2.2]
at org.postgresql.jdbc.PgConnection.isValid(PgConnection.java:1364) ~[postgresql-42.2.2.jar!/:42.2.2]
at com.zaxxer.hikari.pool.PoolBase.isConnectionAlive(PoolBase.java:153) [HikariCP-3.2.0.jar!/:na]
at com.zaxxer.hikari.pool.HikariPool.getConnection(HikariPool.java:179) [HikariCP-3.2.0.jar!/:na]
at com.zaxxer.hikari.pool.HikariPool.getConnection(HikariPool.java:155) [HikariCP-3.2.0.jar!/:na]
at com.zaxxer.hikari.HikariDataSource.getConnection(HikariDataSource.java:128) [HikariCP-3.2.0.jar!/:na]
at org.springframework.jdbc.datasource.DataSourceUtils.fetchConnection(DataSourceUtils.java:157) [spring-jdbc-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:115) [spring-jdbc-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:78) [spring-jdbc-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:371) [spring-jdbc-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:452) [spring-jdbc-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:462) [spring-jdbc-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
at com.***.***.batch.job.CacheRefreshWatcher.cacheRefreshSchedular(CacheRefreshWatcher.java:38) [classes!/:na]
at sun.reflect.GeneratedMethodAccessor344.invoke(Unknown Source) ~[na:na]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_212]
at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_212]
at org.springframework.scheduling.support.ScheduledMethodRunnable.run(ScheduledMethodRunnable.java:84) [spring-context-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54) [spring-context-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
at org.springframework.scheduling.concurrent.ReschedulingRunnable.run(ReschedulingRunnable.java:93) [spring-context-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_212]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_212]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_212]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) [na:1.8.0_212]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_212]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_212]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_212]
Caused by: java.net.SocketException: Connection reset
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:115) ~[na:1.8.0_212]
at java.net.SocketOutputStream.write(SocketOutputStream.java:155) ~[na:1.8.0_212]
at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82) ~[na:1.8.0_212]
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140) ~[na:1.8.0_212]
at org.postgresql.core.PGStream.flush(PGStream.java:514) ~[postgresql-42.2.2.jar!/:42.2.2]
at org.postgresql.core.v3.QueryExecutorImpl.sendSync(QueryExecutorImpl.java:1363) ~[postgresql-42.2.2.jar!/:42.2.2]
at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:304) ~[postgresql-42.2.2.jar!/:42.2.2]
... 29 common frames omitted
2019-12-14 07:55:00.015 WARN 1 --- [ scheduling-1] com.zaxxer.hikari.pool.PoolBase : HikariPool-1 - Failed to validate connection org.postgresql.jdbc.PgConnection@48cf96e3 (This connection has been closed.). Possibly consider using a shorter maxLifetime value.
2019-12-14 22:00:00.017 WARN 1 --- [ scheduling-1] org.postgresql.jdbc.PgConnection : Validating connection.
org.postgresql.util.PSQLException: An I/O error occurred while sending to the backend.
at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:333) ~[postgresql-42.2.2.jar!/:42.2.2]
at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:441) ~[postgresql-42.2.2.jar!/:42.2.2]
at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:365) ~[postgresql-42.2.2.jar!/:42.2.2]
at org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:155) ~[postgresql-42.2.2.jar!/:42.2.2]
at org.postgresql.jdbc.PgPreparedStatement.executeUpdate(PgPreparedStatement.java:132) ~[postgresql-42.2.2.jar!/:42.2.2]
at org.postgresql.jdbc.PgConnection.isValid(PgConnection.java:1364) ~[postgresql-42.2.2.jar!/:42.2.2]
at com.zaxxer.hikari.pool.PoolBase.isConnectionAlive(PoolBase.java:153) [HikariCP-3.2.0.jar!/:na]
at com.zaxxer.hikari.pool.HikariPool.getConnection(HikariPool.java:179) [HikariCP-3.2.0.jar!/:na]
at com.zaxxer.hikari.pool.HikariPool.getConnection(HikariPool.java:155) [HikariCP-3.2.0.jar!/:na]
at com.zaxxer.hikari.HikariDataSource.getConnection(HikariDataSource.java:128) [HikariCP-3.2.0.jar!/:na]
at org.springframework.jdbc.datasource.DataSourceUtils.fetchConnection(DataSourceUtils.java:157) [spring-jdbc-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:115) [spring-jdbc-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:78) [spring-jdbc-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:371) [spring-jdbc-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:452) [spring-jdbc-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:462) [spring-jdbc-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
at com.adessa.promiseengine.batch.job.CacheRefreshWatcher.cacheRefreshSchedular(CacheRefreshWatcher.java:38) [classes!/:na]
at sun.reflect.GeneratedMethodAccessor344.invoke(Unknown Source) ~[na:na]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_212]
at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_212]
at org.springframework.scheduling.support.ScheduledMethodRunnable.run(ScheduledMethodRunnable.java:84) [spring-context-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54) [spring-context-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
at org.springframework.scheduling.concurrent.ReschedulingRunnable.run(ReschedulingRunnable.java:93) [spring-context-5.1.5.RELEASE.jar!/:5.1.5.RELEASE]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_212]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_212]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_212]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) [na:1.8.0_212]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_212]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_212]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_212]
org.postgresql.util.PSQLException: An I/O error occurred while sending to the backend.
Caused by: java.net.SocketException: Connection reset
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:115) ~[na:1.8.0_212]
at java.net.SocketOutputStream.write(SocketOutputStream.java:155) ~[na:1.8.0_212]
at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82) ~[na:1.8.0_212]
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140) ~[na:1.8.0_212]
at org.postgresql.core.PGStream.flush(PGStream.java:514) ~[postgresql-42.2.2.jar!/:42.2.2]
at org.postgresql.core.v3.QueryExecutorImpl.sendSync(QueryExecutorImpl.java:1363) ~[postgresql-42.2.2.jar!/:42.2.2]
at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:304) ~[postgresql-42.2.2.jar!/:42.2.2]
... 29 common frames omitted
2019-12-14 22:00:00.018 WARN 1 --- [ scheduling-1] com.zaxxer.hikari.pool.PoolBase : HikariPool-1 - Failed to validate connection org.postgresql.jdbc.PgConnection@3029eb50 (This connection has been closed.). Possibly consider using a shorter maxLifetime value.
Below is my hikari properties
hikari:
minimum-idle: 5
maximum-pool-size: 10
idle-timeout: 600000
max-lifetime: 550000
connection-timeout: 600000
Application is hosted in an on premise server connected to an on prem db, also its hosted in AKS connecting to a Azure Postgres DB. Error is seen in both deployments. Can someone help me understand why is this happening and suggest a solution.
来源：https://stackoverflow.com/questions/59530069/intermittent-connection-issues-from-springboot-to-postgres