[JCS-231] Lateral TCP cache goes in loop and consumes all available resources

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: jcs-3.1
• Fix Version/s: jcs-3.2
• Environment: Linux, Tomcat 9, Adoption JDK 11
• Assignee: Thomas Vandahl
• Reporter: Andrea Vettori
• Created: 16/Apr/22 13:11
• Updated: 18/Sep/23 17:34
• Resolved: 22/Apr/22 09:32

Description

Hello, we upgraded from 3.0 to 3.1 and had to go back for the following problem.

With this LTCP configuration

jcs.auxiliary.LTCP=org.apache.commons.jcs3.auxiliary.lateral.socket.tcp.LateralTCPCacheFactory
jcs.auxiliary.LTCP.attributes=org.apache.commons.jcs3.auxiliary.lateral.socket.tcp.TCPLateralCacheAttributes
jcs.auxiliary.LTCP.attributes.TcpListenerPort=1120
jcs.auxiliary.LTCP.attributes.UdpDiscoveryAddr=228.5.6.8
jcs.auxiliary.LTCP.attributes.UdpDiscoveryPort=6782
jcs.auxiliary.LTCP.attributes.UdpDiscoveryEnabled=true
jcs.auxiliary.LTCP.attributes.Receive=true
jcs.auxiliary.LTCP.attributes.AllowGet=false
jcs.auxiliary.LTCP.attributes.IssueRemoveOnPut=false

 

Most of the time JCS starts with the following logs

16-Apr-2022 14:31:04.523 INFO [main] org.apache.commons.jcs3.auxiliary.AuxiliaryCacheConfigurator. Using standard serializer [org.apache.commons.jcs3.utils.serialization.StandardSerializer@22aa6570] for auxiliary [jcs.auxiliary.LTCP]16-Apr-2022 14:31:04.524 INFO [main] org.apache.commons.jcs3.auxiliary.lateral.socket.tcp.LateralTCPCacheFactory. Instance for [] is null, creating16-Apr-2022 14:31:04.524 INFO [main] org.apache.commons.jcs3.auxiliary.lateral.socket.tcp.LateralTCPCacheFactory. Creating TCP service, lca = :112016-Apr-2022 14:31:04.531 SEVERE [main] org.apache.commons.jcs3.auxiliary.lateral.socket.tcp.LateralTCPService. Could not create sender to [] – Invalid address []16-Apr-2022 14:31:04.531 SEVERE [main] org.apache.commons.jcs3.auxiliary.lateral.socket.tcp.LateralTCPCacheFactory. Failure, lateral instance will use zombie service        java.io.IOException: Invalid address []                at org.apache.commons.jcs3.auxiliary.lateral.socket.tcp.LateralTCPSender.<init>(LateralTCPSender.java:104)                at org.apache.commons.jcs3.auxiliary.lateral.socket.tcp.LateralTCPService.<init>(LateralTCPService.java:96)                at org.apache.commons.jcs3.auxiliary.lateral.socket.tcp.LateralTCPCacheFactory.lambda$getCSNLInstance$0(LateralTCPCacheFactory.java:271)                at java.base/java.util.concurrent.ConcurrentHashMap.compute(ConcurrentHashMap.java:1908)                at org.apache.commons.jcs3.auxiliary.lateral.socket.tcp.LateralTCPCacheFactory.getCSNLInstance(LateralTCPCacheFactory.java:251)                at org.apache.commons.jcs3.auxiliary.lateral.socket.tcp.LateralTCPCacheFactory.createCacheNoWait(LateralTCPCacheFactory.java:143)                at org.apache.commons.jcs3.auxiliary.lateral.socket.tcp.LateralTCPCacheFactory.createCache(LateralTCPCacheFactory.java:110)                at org.apache.commons.jcs3.auxiliary.lateral.socket.tcp.LateralTCPCacheFactory.createCache(LateralTCPCacheFactory.java:56)                at org.apache.commons.jcs3.engine.control.CompositeCacheConfigurator.parseAuxiliary(CompositeCacheConfigurator.java:450)                at org.apache.commons.jcs3.engine.control.CompositeCacheConfigurator.parseRegion(CompositeCacheConfigurator.java:234)                at org.apache.commons.jcs3.engine.control.CompositeCacheConfigurator.parseRegion(CompositeCacheConfigurator.java:157)                at org.apache.commons.jcs3.engine.control.CompositeCacheConfigurator.parseRegions(CompositeCacheConfigurator.java:135)                at org.apache.commons.jcs3.engine.control.CompositeCacheManager.doConfigure(CompositeCacheManager.java:455)                at org.apache.commons.jcs3.engine.control.CompositeCacheManager.configure(CompositeCacheManager.java:406)                at org.apache.commons.jcs3.engine.control.CompositeCacheManager.configure(CompositeCacheManager.java:365)                at org.apache.commons.jcs3.engine.control.CompositeCacheManager.configure(CompositeCacheManager.java:347)                at org.apache.commons.jcs3.engine.control.CompositeCacheManager.configure(CompositeCacheManager.java:336)                at org.apache.commons.jcs3.engine.control.CompositeCacheManager.getInstance(CompositeCacheManager.java:188)                at org.apache.commons.jcs3.engine.control.CompositeCacheManager.getInstance(CompositeCacheManager.java:163)                at org.apache.commons.jcs3.JCS.getCacheManager(JCS.java:126)                at org.apache.commons.jcs3.JCS.getInstance(JCS.java:144)

 

and then goes into a loop that after some time consumes all system resources{}

16-Apr-2022 14:31:24.544 INFO [JCS-LateralCacheMonitor] org.apache.commons.jcs3.auxiliary.lateral.LateralCacheMonitor. Found LateralCacheNoWait in error, data16-Apr-2022 14:31:24.544 INFO [JCS-LateralCacheMonitor] org.apache.commons.jcs3.auxiliary.lateral.socket.tcp.LateralTCPCacheFactory. Disposing of zombie service instance for []16-Apr-2022 14:31:24.545 INFO [JCS-LateralCacheMonitor] org.apache.commons.jcs3.auxiliary.lateral.socket.tcp.LateralTCPCacheFactory. Instance for [] is null, creating16-Apr-2022 14:31:24.545 INFO [JCS-LateralCacheMonitor] org.apache.commons.jcs3.auxiliary.lateral.socket.tcp.LateralTCPCacheFactory. Creating TCP service, lca = :112016-Apr-2022 14:31:24.545 SEVERE [JCS-LateralCacheMonitor] org.apache.commons.jcs3.auxiliary.lateral.socket.tcp.LateralTCPService. Could not create sender to [] – Invalid address []16-Apr-2022 14:31:24.545 SEVERE [JCS-LateralCacheMonitor] org.apache.commons.jcs3.auxiliary.lateral.socket.tcp.LateralTCPCacheFactory. Failure, lateral instance will use zombie service        java.io.IOException: Invalid address []                at org.apache.commons.jcs3.auxiliary.lateral.socket.tcp.LateralTCPSender.<init>(LateralTCPSender.java:104)                at org.apache.commons.jcs3.auxiliary.lateral.socket.tcp.LateralTCPService.<init>(LateralTCPService.java:96)                at org.apache.commons.jcs3.auxiliary.lateral.socket.tcp.LateralTCPCacheFactory.lambda$getCSNLInstance$0(LateralTCPCacheFactory.java:271)                at java.base/java.util.concurrent.ConcurrentHashMap.compute(ConcurrentHashMap.java:1932)                at org.apache.commons.jcs3.auxiliary.lateral.socket.tcp.LateralTCPCacheFactory.getCSNLInstance(LateralTCPCacheFactory.java:251)                at org.apache.commons.jcs3.auxiliary.lateral.LateralCacheMonitor.lambda$doWork$0(LateralCacheMonitor.java:124)                at java.base/java.util.concurrent.ConcurrentHashMap.forEach(ConcurrentHashMap.java:1603)                at org.apache.commons.jcs3.auxiliary.lateral.LateralCacheMonitor.doWork(LateralCacheMonitor.java:113)                at org.apache.commons.jcs3.auxiliary.AbstractAuxiliaryCacheMonitor.run(AbstractAuxiliaryCacheMonitor.java:180)

 

Even when JCS starts starts with no exception, after some time the second exception above begins to appear on the logs and takes the server down.