[NIFI-5830] RedisConnectionPoolService does not work with Standalone Redis using non-localhost deployment

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.8.0
• Fix Version/s: 1.9.0
• Component/s: Core Framework
• Environment: Ubuntu 16 LTS, NiFi 1.8.0
• Assignee: Unassigned
• Reporter: Alexander Bukarev
• Created: 17/Nov/18 06:42
• Updated: 06/Dec/18 22:32
• Resolved: 06/Dec/18 22:32

Description

The controller service RedisConnectionPoolService does not work with Standalone Redis which is deployed on a host other than localhost (or if Redis uses the port other than 6379). So the only way to use RedisConnectionPoolService is to deploy Redis on localhost and run it with default port 6379.

Steps:
Let's assume our Redis is deployed on host redis (not localhost) and it listens port 6379 (I use docker for that)

1. Create a PutDistributedMapCache processor
2. Configure the processor with RedisDistributedMapCacheClientService
3. Create a new controller service: RedisConnectionPoolService
   • Choose Standalone Redis Mode
   • Use redis:6379 as a Connection String
4. Connect some incoming flow to the PutDistributedMapCache processor (I've used GetFile as a producer) and run the whole flow. Allow GetFile to consume some file (if you use GetFile ro reproduce) and wait some time till the PutDistributedMapCache will be schedulled.

Result:
The processor is failed to run. And we can see the error in logs:

2018-11-17 06:06:47,572 WARN [Timer-Driven Process Thread-2] o.a.n.controller.tasks.ConnectableTask Administratively Yielding PutDistribu
tedMapCache[id=2041c81f-0167-1000-c82f-d7da2155dfb4] due to uncaught Exception: org.springframework.data.redis.RedisConnectionFailureExce
ption: Cannot get Jedis connection; nested exception is redis.clients.jedis.exceptions.JedisConnectionException: Could not get a resource
from the pool
org.springframework.data.redis.RedisConnectionFailureException: Cannot get Jedis connection; nested exception is redis.clients.jedis.exceptions.JedisConnectionException: Could not get a resource from the pool
at org.springframework.data.redis.connection.jedis.JedisConnectionFactory.fetchJedisConnector(JedisConnectionFactory.java:281)
at org.springframework.data.redis.connection.jedis.JedisConnectionFactory.getConnection(JedisConnectionFactory.java:464)
at org.apache.nifi.redis.service.RedisConnectionPoolService.getConnection(RedisConnectionPoolService.java:89)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:84)
at com.sun.proxy.$Proxy128.getConnection(Unknown Source)
at org.apache.nifi.redis.service.RedisDistributedMapCacheClientService.withConnection(RedisDistributedMapCacheClientService.java:343)
at org.apache.nifi.redis.service.RedisDistributedMapCacheClientService.put(RedisDistributedMapCacheClientService.java:189)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:84)
at com.sun.proxy.$Proxy124.put(Unknown Source)
at org.apache.nifi.processors.standard.PutDistributedMapCache.onTrigger(PutDistributedMapCache.java:202)

Issue Links

• relates to: NIFI-5795 RedisDistributedMapCacheClientService put missing option
• links to: GitHub Pull Request #3176