[HADOOP-12475] Replace guava Cache with ConcurrentHashMap for caching Connection in ipc Client

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.8.0, 3.0.0-alpha1
• Component/s: conf, io, ipc
• Hadoop Flags: Reviewed
• Assignee: Walter Su
• Reporter: Walter Su
• Created: 14/Oct/15 06:07
• Updated: 30/Aug/16 01:22
• Resolved: 15/Oct/15 18:55

Description

quote daryn from HADOOP-11772:

CacheBuilder is obscenely expensive for concurrent map, and it requires generating unnecessary garbage even just to look up a key. Replace it with ConcurrentHashMap.

I identified this issue that impaired my own perf testing under load. The slowdown isn't just the sync. It's the expensive of Connection's ctor stalling other connections. The expensive of ConnectionId#equals causes delays. Synch'ing on connections causes unfair contention unlike a sync'ed method. Concurrency simply hides this.

BTW, guava Cache is heavyweight. Per local test, ConcurrentHashMap has better overal performance.