Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-29877

static PageRank allow checkPoint from previous computations

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 2.3.0
    • 3.0.0
    • GraphX
    • None

    Description

      It would be really helpful to have the possibility, when computing staticPageRank to use a previous computation as a checkpoint to continue the iterations.

      I have done a small code proposal, but there is a problem because at the end of the iterations a normalization step is performed.

      Therefore, if we use this normalized graph as a checkpoint for a new set of iterations, the algorithm will not restart in a coherent way.

      I created a branch in my fork: https://github.com/JoanFM/spark/tree/pageRank_checkPoint

      I hope you can consider this feature, and if you do that you can implement yourself or propose me a way to handle this identified problem

      Thank you very much,

      Best regards,

      Joan

      Attachments

        Activity

          People

            JoanFM Joan Fontanals
            JoanFM Joan Fontanals
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: