Uploaded image for project: 'Giraph (Retired)'
  1. Giraph (Retired)
  2. GIRAPH-293

Should aggregators be checkpointed?

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • None
    • None
    • None

    Description

      As I understand, we don't include aggregators in checkpoints because they are kept in the Zookeeper.

      One of our bootcampers is working on fixing TestManualCheckpoint, which currently involves starting a new job from a checkpoint from a previous job*.
      If this is a functionality we want going forward, then persistent aggregators should be checkpointed.

      [*] That test relies on the fact that either aggregators are checkpointed or they are always reset at each superstep. None of these is happening, but the error cancels out with the fact that we are not actually resuming from a checkpoint, but re-running the job from scratch.

      Attachments

        1. GIRAPH-293.patch
          61 kB
          Maja Kabiljo
        2. GIRAPH-293.patch
          61 kB
          Maja Kabiljo
        3. GIRAPH-293.patch
          61 kB
          Maja Kabiljo
        4. GIRAPH-293.patch
          61 kB
          Maja Kabiljo
        5. GIRAPH-293.patch
          61 kB
          Maja Kabiljo

        Issue Links

          is related to

          Bug - A problem which impairs or prevents the functions of the product. GIRAPH-270 Improve TestManualCheckpoint

          • Major - Major loss of function.
          • Open

          Bug - A problem which impairs or prevents the functions of the product. GIRAPH-259 TestBspBasic.testBspPageRank is broken

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. GIRAPH-297 Checkpointing on master is done one superstep later

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. GIRAPH-355 Partition.readFields crashes

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. GIRAPH-273 Aggregators shouldn't use Zookeeper

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              majakabiljo Maja Kabiljo
              apresta Alessandro Presta
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: