Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-3655

Support sorting of values in addition to keys (i.e. secondary sort)

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Resolved
    • Major
    • Resolution: Auto Closed
    • 1.1.0, 1.2.0
    • None
    • Spark Core

    Description

      Now that spark has a sort based shuffle, can we expect a secondary sort soon? There are some use cases where getting a sorted iterator of values per key is helpful.

      Attachments

        Issue Links

          is related to

          New Feature - A new feature of the product, which has yet to be developed. SPARK-2045 Sort-based shuffle implementation

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-10405 Support takeOrdered and topK values per key

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          relates to

          Sub-task - The sub-task of the issue PIG-4504 Enable Secondary key sort feature in spark mode

          • Major - Major loss of function.
          • Closed

          New Feature - A new feature of the product, which has yet to be developed. SPARK-15798 Secondary sort in Dataset/DataFrame

          • Major - Major loss of function.
          • Resolved
          links to

          Pull request #3632 [Github] Pull Request #3632 (koertkuipers)

          Activity

            People

              koertkuipers Koert Kuipers
              koert koert kuipers
              Votes:
              7 Vote for this issue
              Watchers:
              38 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: