[SPARK-27583] Optimize strongly connected components

• Type: Improvement
• Status: In Progress
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 3.1.0
• Component/s: GraphX
• Assignee: Unassigned
• Reporter: jiamuzhou
• Created: 27/Apr/19 12:03
• Updated: 16/Mar/20 22:53

Description

Currently, when I use function of strongly connected components in graph (0.1billion vertices and 0.15 billion edges) found that memory storage of RDDs is growing during the iteration, which lead to running very slow or failure of the job.

 

Issue Links

• links to: [Github] Pull Request #24479 (jmzhoulab)
• links to: GitHub Pull Request #24479