[MAHOUT-1042] Hotspot in RecommenderJob-PartialMultiplyMapper-Reducer

• Type: Improvement
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 0.6, 0.7
• Fix Version/s: 0.8
• Assignee: Sebastian Schelter
• Reporter: Bhaskar Devireddy
• Created: 09/Jul/12 21:31
• Updated: 31/Jan/24 22:11
• Resolved: 10/Jul/12 15:20

Description

While profiling PartialMultiplyMapper-Reducer job we noticed a hotspot consuming more than 40% of the CPU time in org.apache.mahout.math.RandomAccessSparseVector.assign method for the reducer task. We used the script provided in mahout examples for running ASF Email recommendations for profiling. The hotspot is coming from the use of Vector.plus(Vector x) method in AggregateAndRecommendReducerc class. The pattern used is VectorA = VectorA.plus(VectorB). In this case VectorA doesn't have to be cloned using assign method. The attached patch addresses the hotspot by eliminating cloning in the above case for plus and times methods. This patch while retaining functionality (verified the output with and without patch), speeds up execution time of PartialMultiplyMapper-Reducer job by more than 10X on x86 architectures.