[SPARK-8392] RDDOperationGraph: getting cached nodes is slow

• Type: Improvement
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 1.4.0
• Fix Version/s: 1.4.1, 1.5.0
• Component/s: Spark Core
• Target Version/s: 1.4.1, 1.5.0
• Assignee: meiyoula
• Reporter: meiyoula
• Created: 16/Jun/15 09:16
• Updated: 18/Jun/15 05:32
• Resolved: 18/Jun/15 05:32

Description

def getAllNodes: Seq[RDDOperationNode] =

{ _childNodes ++ _childClusters.flatMap(_.childNodes) }

when the _childClusters has so many nodes, the process will hang on. I think we can improve the efficiency here.

Issue Links

• links to: [Github] Pull Request #6839 (XuTingjun)