[SPARK-2152] the error of comput rightNodeAgg about Decision tree algorithm in Spark MLlib

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.0.0
• Fix Version/s: 1.0.2, 1.1.0
• Labels: features

Environment

windows7 ,32 operator,and 3G mem

• Assignee: Jon Sondag
• Reporter: caoli
• Created: 16/Jun/14 09:14
• Updated: 05/Aug/14 19:32
• Resolved: 09/Jul/14 02:18

Description

the error of comput rightNodeAgg about Decision tree algorithm in Spark MLlib about the function extractLeftRightNodeAggregates() ,when compute rightNodeAgg used bindata index is error. in the DecisionTree.scala file about Line 980:

rightNodeAgg(featureIndex)(2 * (numBins - 2 - splitIndex)) =
binData(shift + (2 * (numBins - 2 - splitIndex))) +
rightNodeAgg(featureIndex)(2 * (numBins - 1 - splitIndex))

the binData(shift + (2 * (numBins - 2 - splitIndex))) index compute is error, so the result of rightNodeAgg include repeated data about "bins"

Issue Links

• is duplicated by: SPARK-2160 error of Decision tree algorithm in Spark MLlib
• is related to: SPARK-2417 Decision tree tests are failing