Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-22681

Accumulator should only be updated once for each task in result stage

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.2.0
    • Fix Version/s: 2.3.0
    • Component/s: Spark Core
    • Labels:
      None

      Description

      As the doc says "For accumulator updates performed inside actions only, Spark guarantees that each task’s update to the accumulator will only be applied once, i.e. restarted tasks will not update the value."
      But currently the code doesn't guarantee this.

        Attachments

          Activity

            People

            • Assignee:
              carsonwang Carson Wang
              Reporter:
              carsonwang Carson Wang
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: