[HADOOP-7183] WritableComparator.get should not cache comparator objects

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 0.22.0
• Fix Version/s: 0.21.1, 0.22.0
• Hadoop Flags: Reviewed
• Assignee: Thomas White
• Reporter: Todd Lipcon
• Created: 11/Mar/11 18:22
• Updated: 12/Dec/11 06:19
• Resolved: 06/May/11 05:09

Description

HADOOP-6881 modified WritableComparator.get such that the constructed WritableComparator gets saved back into the static map. This is fine for stateless comparators, but some comparators have per-instance state, and thus this becomes thread-unsafe and causes errors in the shuffle where multiple threads are doing comparisons. An example of a Comparator with per-instance state is WritableComparator itself.

Issue Links

• is related to: HADOOP-7219 Reuse WritableComparator instances using ThreadLocals