前提说明:
服务环境,用的是阿里云的 ECS云服务。 共用三台服务,用docker 搭建 3 主3 从共6个集群环境。 使用springCloud做分布式框架。 使用 lettuce 做连接池。
安装搭建参考:
【SpringCloud分布式框架搭建】docker-compose安装redis5 集群。并集成到springCloud中
错误信息如下:
org.springframework.dao.DataAccessResourceFailureException: Unable to connect to 172.16.75.43:6300; nested exception is io.lettuce.core.RedisConnectionException: Unable to connect to 172.16.75.43:6300
at org.springframework.data.redis.connection.lettuce.LettuceClusterConnection$LettuceClusterNodeResourceProvider.getResourceForSpecificNode(LettuceClusterConnection.java:697) ~[spring-data-redis-2.1.10.RELEASE.jar!/:2.1.10.RELEASE]
at org.springframework.data.redis.connection.lettuce.LettuceClusterConnection$LettuceClusterNodeResourceProvider.getResourceForSpecificNode(LettuceClusterConnection.java:674) ~[spring-data-redis-2.1.10.RELEASE.jar!/:2.1.10.RELEASE]
at org.springframework.data.redis.connection.ClusterCommandExecutor.executeCommandOnSingleNode(ClusterCommandExecutor.java:140) ~[spring-data-redis-2.1.10.RELEASE.jar!/:2.1.10.RELEASE]
at org.springframework.data.redis.connection.ClusterCommandExecutor.executeCommandOnSingleNode(ClusterCommandExecutor.java:123) ~[spring-data-redis-2.1.10.RELEASE.jar!/:2.1.10.RELEASE]
at org.springframework.data.redis.connection.ClusterCommandExecutor.executeCommandOnArbitraryNode(ClusterCommandExecutor.java:111) ~[spring-data-redis-2.1.10.RELEASE.jar!/:2.1.10.RELEASE]
at org.springframework.data.redis.connection.lettuce.LettuceClusterServerCommands.time(LettuceClusterServerCommands.java:317) ~[spring-data-redis-2.1.10.RELEASE.jar!/:2.1.10.RELEASE]
at org.springframework.data.redis.connection.DefaultedRedisConnection.time(DefaultedRedisConnection.java:1218) ~[spring-data-redis-2.1.10.RELEASE.jar!/:2.1.10.RELEASE]
at sun.reflect.GeneratedMethodAccessor176.invoke(Unknown Source) ~[na:na]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_232]
at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_232]
at org.springframework.data.redis.core.CloseSuppressingInvocationHandler.invoke(CloseSuppressingInvocationHandler.java:61) ~[spring-data-redis-2.1.10.RELEASE.jar!/:2.1.10.RELEASE]
at com.sun.proxy.$Proxy208.time(Unknown Source) ~[na:na]
at com.keda.util.redis.RedisService$1.doInRedis(RedisService.java:840) ~[keda-util-1.1.2.jar!/:na]
at org.springframework.data.redis.core.RedisTemplate.execute(RedisTemplate.java:224) ~[spring-data-redis-2.1.10.RELEASE.jar!/:2.1.10.RELEASE]
at org.springframework.data.redis.core.RedisTemplate.execute(RedisTemplate.java:184) ~[spring-data-redis-2.1.10.RELEASE.jar!/:2.1.10.RELEASE]
at org.springframework.data.redis.core.RedisTemplate.execute(RedisTemplate.java:171) ~[spring-data-redis-2.1.10.RELEASE.jar!/:2.1.10.RELEASE]
at com.keda.util.redis.RedisService.currtTimeForRedis(RedisService.java:837) ~[keda-util-1.1.2.jar!/:na]
at com.keda.util.redis.RedisService.getNowLocalDateTime(RedisService.java:873) ~[keda-util-1.1.2.jar!/:na]
at com.keda.orderbuy.scheduled.timeTask.myTask(timeTask.java:50) ~[classes!/:0.0.1-SNAPSHOT]
at com.keda.orderbuy.scheduled.timeTask$$FastClassBySpringCGLIB$$689dd3ff.invoke(<generated>) ~[classes!/:0.0.1-SNAPSHOT]
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) ~[spring-core-5.1.3.RELEASE.jar!/:5.1.3.RELEASE]
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:749) ~[spring-aop-5.1.3.RELEASE.jar!/:5.1.3.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ~[spring-aop-5.1.3.RELEASE.jar!/:5.1.3.RELEASE]
at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:88) ~[spring-aop-5.1.3.RELEASE.jar!/:5.1.3.RELEASE]
at org.springframework.cloud.sleuth.instrument.scheduling.TraceSchedulingAspect.traceBackgroundThread(TraceSchedulingAspect.java:73) ~[spring-cloud-sleuth-core-2.1.3.RELEASE.jar!/:2.1.3.RELEASE]
at sun.reflect.GeneratedMethodAccessor184.invoke(Unknown Source) ~[na:na]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_232]
at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_232]
at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:644) ~[spring-aop-5.1.3.RELEASE.jar!/:5.1.3.RELEASE]
at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:633) ~[spring-aop-5.1.3.RELEASE.jar!/:5.1.3.RELEASE]
at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70) ~[spring-aop-5.1.3.RELEASE.jar!/:5.1.3.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.1.3.RELEASE.jar!/:5.1.3.RELEASE]
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:93) ~[spring-aop-5.1.3.RELEASE.jar!/:5.1.3.RELEASE]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.1.3.RELEASE.jar!/:5.1.3.RELEASE]
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:688) ~[spring-aop-5.1.3.RELEASE.jar!/:5.1.3.RELEASE]
at com.keda.orderbuy.scheduled.timeTask$$EnhancerBySpringCGLIB$$27fb77d1.myTask(<generated>) ~[classes!/:0.0.1-SNAPSHOT]
at sun.reflect.GeneratedMethodAccessor183.invoke(Unknown Source) ~[na:na]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_232]
at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_232]
at org.springframework.scheduling.support.ScheduledMethodRunnable.run(ScheduledMethodRunnable.java:84) ~[spring-context-5.1.3.RELEASE.jar!/:5.1.3.RELEASE]
at net.javacrumbs.shedlock.core.DefaultLockingTaskExecutor.executeWithLock(DefaultLockingTaskExecutor.java:54) ~[shedlock-core-2.3.0.jar!/:na]
at net.javacrumbs.shedlock.core.DefaultLockingTaskExecutor.executeWithLock(DefaultLockingTaskExecutor.java:39) ~[shedlock-core-2.3.0.jar!/:na]
at net.javacrumbs.shedlock.core.DefaultLockManager.executeWithLock(DefaultLockManager.java:51) ~[shedlock-core-2.3.0.jar!/:na]
at net.javacrumbs.shedlock.core.LockableRunnable.run(LockableRunnable.java:35) ~[shedlock-core-2.3.0.jar!/:na]
at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54) ~[spring-context-5.1.3.RELEASE.jar!/:5.1.3.RELEASE]
at org.springframework.scheduling.concurrent.ReschedulingRunnable.run(ReschedulingRunnable.java:93) [spring-context-5.1.3.RELEASE.jar!/:5.1.3.RELEASE]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_232]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_232]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_232]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) [na:1.8.0_232]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_232]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_232]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_232]
Caused by: io.lettuce.core.RedisConnectionException: Unable to connect to 172.16.75.43:6300
at io.lettuce.core.RedisConnectionException.create(RedisConnectionException.java:78) ~[lettuce-core-5.1.8.RELEASE.jar!/:na]
at io.lettuce.core.RedisConnectionException.create(RedisConnectionException.java:56) ~[lettuce-core-5.1.8.RELEASE.jar!/:na]
at io.lettuce.core.cluster.PooledClusterConnectionProvider.getConnection(PooledClusterConnectionProvider.java:361) ~[lettuce-core-5.1.8.RELEASE.jar!/:na]
at io.lettuce.core.cluster.PooledClusterConnectionProvider.getConnection(PooledClusterConnectionProvider.java:346) ~[lettuce-core-5.1.8.RELEASE.jar!/:na]
at io.lettuce.core.cluster.StatefulRedisClusterConnectionImpl.getConnection(StatefulRedisClusterConnectionImpl.java:160) ~[lettuce-core-5.1.8.RELEASE.jar!/:na]
at org.springframework.data.redis.connection.lettuce.LettuceClusterConnection$LettuceClusterNodeResourceProvider.getResourceForSpecificNode(LettuceClusterConnection.java:695) ~[spring-data-redis-2.1.10.RELEASE.jar!/:2.1.10.RELEASE]
... 52 common frames omitted
Caused by: io.lettuce.core.RedisConnectionException: Unable to connect to 172.16.75.43:6300
at io.lettuce.core.RedisConnectionException.create(RedisConnectionException.java:78) ~[lettuce-core-5.1.8.RELEASE.jar!/:na]
at io.lettuce.core.RedisConnectionException.create(RedisConnectionException.java:56) ~[lettuce-core-5.1.8.RELEASE.jar!/:na]
at io.lettuce.core.cluster.PooledClusterConnectionProvider.lambda$getConnectionAsync$6(PooledClusterConnectionProvider.java:328) ~[lettuce-core-5.1.8.RELEASE.jar!/:na]
at java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:836) ~[na:1.8.0_232]
at java.util.concurrent.CompletableFuture$UniHandle.tryFire(CompletableFuture.java:811) ~[na:1.8.0_232]
at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) ~[na:1.8.0_232]
at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1990) ~[na:1.8.0_232]
at io.lettuce.core.AbstractRedisClient.lambda$initializeChannelAsync0$4(AbstractRedisClient.java:330) ~[lettuce-core-5.1.8.RELEASE.jar!/:na]
at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:500) ~[netty-common-4.1.39.Final.jar!/:4.1.39.Final]
at io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:493) ~[netty-common-4.1.39.Final.jar!/:4.1.39.Final]
at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:472) ~[netty-common-4.1.39.Final.jar!/:4.1.39.Final]
at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:413) ~[netty-common-4.1.39.Final.jar!/:4.1.39.Final]
at io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:538) ~[netty-common-4.1.39.Final.jar!/:4.1.39.Final]
at io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:531) ~[netty-common-4.1.39.Final.jar!/:4.1.39.Final]
at io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:111) ~[netty-common-4.1.39.Final.jar!/:4.1.39.Final]
at io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:631) ~[netty-transport-native-epoll-4.1.39.Final-linux-x86_64.jar!/:4.1.39.Final]
at io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:650) ~[netty-transport-native-epoll-4.1.39.Final-linux-x86_64.jar!/:4.1.39.Final]
at io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:524) ~[netty-transport-native-epoll-4.1.39.Final-linux-x86_64.jar!/:4.1.39.Final]
at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:414) ~[netty-transport-native-epoll-4.1.39.Final-linux-x86_64.jar!/:4.1.39.Final]
at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:326) ~[netty-transport-native-epoll-4.1.39.Final-linux-x86_64.jar!/:4.1.39.Final]
at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918) ~[netty-common-4.1.39.Final.jar!/:4.1.39.Final]
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[netty-common-4.1.39.Final.jar!/:4.1.39.Final]
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[netty-common-4.1.39.Final.jar!/:4.1.39.Final]
... 1 common frames omitted
Caused by: java.util.concurrent.CompletionException: io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection timed out: /172.16.75.43:6300
at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:292) ~[na:1.8.0_232]
at java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:308) ~[na:1.8.0_232]
at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:607) ~[na:1.8.0_232]
at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:591) ~[na:1.8.0_232]
... 19 common frames omitted
Caused by: io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection timed out: /172.16.75.43:6300
Caused by: java.net.ConnectException: finishConnect(..) failed: Connection timed out
at io.netty.channel.unix.Errors.throwConnectException(Errors.java:124) ~[netty-transport-native-unix-common-4.1.39.Final.jar!/:4.1.39.Final]
at io.netty.channel.unix.Socket.finishConnect(Socket.java:243) ~[netty-transport-native-unix-common-4.1.39.Final.jar!/:4.1.39.Final]
at io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:667) ~[netty-transport-native-epoll-4.1.39.Final-linux-x86_64.jar!/:4.1.39.Final]
at io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:644) ~[netty-transport-native-epoll-4.1.39.Final-linux-x86_64.jar!/:4.1.39.Final]
at io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:524) ~[netty-transport-native-epoll-4.1.39.Final-linux-x86_64.jar!/:4.1.39.Final]
at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:414) ~[netty-transport-native-epoll-4.1.39.Final-linux-x86_64.jar!/:4.1.39.Final]
at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:326) ~[netty-transport-native-epoll-4.1.39.Final-linux-x86_64.jar!/:4.1.39.Final]
at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918) ~[netty-common-4.1.39.Final.jar!/:4.1.39.Final]
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[netty-common-4.1.39.Final.jar!/:4.1.39.Final]
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[netty-common-4.1.39.Final.jar!/:4.1.39.Final]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_232]
一直被这问题,折磨了几天。 反复检查自己的 redis.conf 配置文件,并没有任何问题。
重点是,我用 RedisDesktopManager客户端,可以连接 所有的 6个节点,并且能set key,get key .
为此,我还写了一个 100次的for循环,操作redis。 成功get/set了85次。 连接超时,导致的错误次数为 15次。
瞬间人就不好了。然后各种百度,都无解。 但是,有个网友提醒了一句:
“是不是docker redis 使用的Ip地址,没有设置成外网可访问的地址”
贴上我的 redis-cluster代码:
redis-cli -a KedaX3@1 \
--cluster create \
--cluster-replicas 1 \
101.xx.xx.160:6300 101.xx.xx.160:6301 \
47.xx.xx.148:6300 47.xx.xx.148:6301 \
47.xx.xx.44:6300 47.xx.xx.44:6301
自认为,配置的Ip。 都是外网能访问的。 但是问题是,springCloud端,访问的时候。 连接的怎么会是 docker 的 ip地址。即(172.16.75.43:6300)。
只能猜,在创建集群的时候。 使用到了 内网地址。 所以打算去看,节点配置文件。即 nodes-6300.conf(文件名,是在 redis.conf 中配置的。如下:)
# 每个集群节点都有集群配置文件,这个文件不是用来提供给使用者编辑,
# 它将由集群节点自身创建并通过集群通信后修改。
# 每个集群节点如果在同一个宿主机上都要有不同的文件,否则将会被覆盖。
#
cluster-config-file nodes-6300.conf
nodes-6300.conf 配置文件内容如下:
c362f3db937xxxxxxxxxxxxxxxxxxxxxxxx3f2fe 47.xx.xx.148:6301@16301 slave 8fc4eabf8dcxxxxxxxxxxxxxxxxxxxxxxxxfe831 0 1576051320000 2 connected
8fc4eabf8dxxxxxxxxxxxxxxxxxxxxxxxxxfe831 172.16.75.43:6300@16300 master - 0 1576051320607 2 connected 5461-10922
67b016927xxxxxxxxxxxxxxxxxxxxxxxxxxx65f2 101.xx.xx.160:6301@16301 master - 0 1576051320000 5 connected 0-5460
fc485e421xxxxxxxxxxxxxxxxxxxxxxxxxfc9b 47.xx.xx.44:6301@16301 master - 0 1576051320662 4 connected 10923-16383
5eb87df6xxxxxxxxxxxxxxxxxxxxxxxxxf6e2f 101.xx.xx.160:6300@16300 slave fc485e4xxxxxxxxxxxxxxxxxxxxxxxxxx342fc9b 0 1576051320427 4 connected
d905ba8xxxxxxxxxxxxxxxxxxxxxxxxxxc69321d 47.xx.xx.148:6300@16300 myself,slave 67b0169xxxxxxxxxxxxxxxxxxxxxxxxxxc465f2 0 1576051320000 1 connected
vars currentEpoch 5 lastVoteEpoch 4
可以一眼就看到,第二行就是 内网地址,即(172.16.75.43:6300@16300 master),并不是 外网可访问的:47.xx.xx.44:6300
修改 成 47.xx.xx.44:6300
然后依次,查看其他5台 redis的node-xxxx.conf文件。 并全部修改成对应的 公网Ip.重启 redis docker.
问题完美解决了!!!!!
来源:CSDN
作者:老新人
链接:https://blog.csdn.net/weixin_42697074/article/details/103574112