[RANGER-4761] Reduce memory footprint of hbase plugin

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: Ranger
• Assignee: Fateh Singh
• Reporter: Fateh Singh
• Created: 27/Mar/24 00:47
• Updated: 06/May/24 21:34

Description

• Map<String, Set> getColumnFamilies(Map<byte[], ? extends Collection<?>> families) becomes a bottleneck in multiget and multiput workloads wherein hundred/thousands of columns may be accessed together in a single request as it is a computationally heavy function converting bytes to string and type-casting Collection to set of strings.

The implementation needs to be revisited to reduce memory footprint

make lazy memory allocation for family map instead of ahead of time memory allocation for family map of type Map<String, Set>. With this approach, memory footprint will get reduced even further when enabling column auth optimization supported by RANGER-4670