[HIVE-10128] BytesBytesMultiHashMap does not allow concurrent read-only access

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: llap, 1.2.0

Release Note

HIVE-10128: BytesBytesMultiHashMap does not allow concurrent read-only access (Sergey Shelukhin, reviewed by Gunther Hagleitner)

• Assignee: Sergey Shelukhin Assign to me
• Reporter: Gopal Vijayaraghavan
• Created: 28/Mar/15 00:18
• Updated: 18/May/15 19:56
• Resolved: 05/Apr/15 04:53

Description

The multi-threaded performance takes a serious hit when LLAP shares hashtables between the probe threads running in parallel.

This is an explicit synchronized block inside ReusableRowContainer which triggers this particular pattern.

Looking deeper into the code, the synchronization seems to be caused due to the fact that WriteBuffers.setReadPoint modifies the otherwise read-only hashtable.

To generate this sort of result, run LLAP at a WARN log-level, to avoid all the log synchronization that otherwise affects the thread sync.

Issue Links

• duplicates: HIVE-10112 LLAP: query 17 tasks fail due to mapjoin issue
• relates to: HIVE-10219 BytesBytesMultiHashMap::clear() can't be concurrently used with reads
• supercedes: HIVE-10112 LLAP: query 17 tasks fail due to mapjoin issue