[SPARK-31216] RDDs in GradientBoostedTress can be unpersisted earlier for saving memory

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 2.4.5
• Component/s: ML, MLlib
• Assignee: Unassigned
• Reporter: IcySanwitch
• Created: 22/Mar/20 15:04
• Updated: 22/Mar/20 15:04

Description

In ml.tree.impl.GradientBoostedTrees.boost(), predErrorCheckpointer and input are unpersisted at last. Actually, they can be unpersisted before the if(validate) block, so it can save memory for the computation in if block.