[MAHOUT-1035] Hotspot in recommenditembased – UnsymmetrifyMapper job

• Type: Improvement
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 0.7
• Fix Version/s: 0.8
• Assignee: Sebastian Schelter
• Reporter: Bhaskar Devireddy
• Created: 29/Jun/12 02:45
• Updated: 31/Jan/24 22:11
• Resolved: 29/Jun/12 21:51

Description

While profiling the unsymmetrify mapper job in recommendations we noticed an hotspot consuming 90% of the CPU runtime in org.apache.mahout.math.map.OpenIntDoubleHashMap.keys method for the first map task. We used the script provided in mahout examples for running ASF Email recommendations for profiling. The attached patch addresses the hotspot by reducing the number of for loop iterations in OpenIntDoubleHashMap.keys method by changing the initialization of transposedPartial. This patch while retaining functionality(verified the output with and without patch) speeds up the unsymmetrify mapper task by more than 4X on x86 architectures.