Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-11047

Internal accumulators miss the internal flag when replaying events in the history server

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 1.5.0
    • Fix Version/s: 1.5.2, 1.6.0
    • Component/s: Spark Core
    • Labels:
      None

      Description

      Internal accumulators don't write the internal flag to event log. So on the history server Web UI, all accumulators are not internal. This causes incorrect peak execution memory and unwanted accumulator table displayed on the stage page.

        Attachments

          Activity

            People

            • Assignee:
              carsonwang Carson Wang
              Reporter:
              carsonwang Carson Wang
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: