Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-2868

Support named accumulators in Python

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Resolved
    • Major
    • Resolution: Incomplete
    • None
    • None
    • PySpark

    Description

      SPARK-2380 added this for Java/Scala. To allow this in Python we'll need to make some additional changes. One potential path is to have a 1:1 correspondence with Scala accumulators (instead of a one-to-many). A challenge is exposing the stringified values of the accumulators to the Scala code.

      Attachments

        Issue Links

          is related to

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-2380 Support displaying accumulator contents in the web UI

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved

          Activity

            People

              Unassigned Unassigned
              pwendell Patrick Wendell
              Votes:
              2 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: