Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-3994

countByKey / countByValue do not go through Aggregator

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 1.0.0
    • 1.2.0
    • Spark Core
    • None

    Description

      The implementations of these methods are historical remnants of Spark from a time when the shuffle may have been nonexistent. Now, they can be simplified by plugging into reduceByKey(), potentially seeing performance and stability improvements.

      Attachments

        Activity

          People

            ilikerps Aaron Davidson
            ilikerps Aaron Davidson
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: