Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-27583

Optimize strongly connected components

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: In Progress
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 3.1.0
    • Fix Version/s: None
    • Component/s: GraphX
    • Labels:
      None

      Description

      Currently, when I use function of strongly connected components in graph (0.1billion vertices and 0.15 billion edges) found that memory storage of RDDs is growing during the iteration, which lead to running very slow or failure of the job.

       

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              jmzhou jiamuzhou
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: