[HBASE-2925] LRU of HConnectionManager.HBASE_INSTANCES breaks if HBaseConfiguration is changed

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.20.3, 0.90.0
• Fix Version/s: 0.90.0
• Hadoop Flags: Reviewed
• Tags: noob
• Assignee: Unassigned
• Reporter: Robert Mahfoud
• Created: 17/Aug/10 21:27
• Updated: 20/Nov/15 12:42
• Resolved: 06/Sep/10 15:52

Description

HConnectionManager.getConnection(config) caches the created TableServer in HBASE_INSTANCES (a LinkedHashMap ) which is keyed by the configuration instance itself.
Given the current implementation of hashCode() (and equals()) of HBaseConfiguration, the hash code of the configuration is changed if any of its properties are changed, which will cause the keys of HBASE_INSTANCES to be inconsistent with the hashtable that contains them, making some entries unreachable.
In this case, when the map's LRU strategy needs to remove the oldest entry, it tries to remove it based on the oldest key, which no longer gives the original hash code, therefore the lookup in HBASE_INSTANCES.remove(oldest) doesn't actually remove anything.

This has been observed to lead to OOM errors in long running clients.

Issue Links

• is a clone of: HBASE-2027 HConnectionManager.HBASE_INSTANCES leaks TableServers
• is related to: HBASE-1251 HConnectionManager.getConnection(HBaseConfiguration) returns same HConnection for different HBaseConfigurations
• is related to: HBASE-1976 HBaseConfiguration implements hashCode() without implementing equals()
• is related to: HBASE-3773 Set ZK max connections much higher in 0.90
• is related to: HBASE-3777 Redefine Identity Of HBase Configuration