Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-3655

Support sorting of values in addition to keys (i.e. secondary sort)

    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Major
    • Resolution: Auto Closed
    • Affects Version/s: 1.1.0, 1.2.0
    • Fix Version/s: None
    • Component/s: Spark Core
    • Labels:

      Description

      Now that spark has a sort based shuffle, can we expect a secondary sort soon? There are some use cases where getting a sorted iterator of values per key is helpful.

        Attachments

          Issue Links

          is related to

          New Feature - A new feature of the product, which has yet to be developed. SPARK-2045 Sort-based shuffle implementation

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-10405 Support takeOrdered and topK values per key

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          relates to

          Sub-task - The sub-task of the issue PIG-4504 Enable Secondary key sort feature in spark mode

          • Major - Major loss of function.
          • Closed

          New Feature - A new feature of the product, which has yet to be developed. SPARK-15798 Secondary sort in Dataset/DataFrame

          • Major - Major loss of function.
          • Resolved
          links to

          Pull request #3632 [Github] Pull Request #3632 (koertkuipers)

            Activity

              People

              • Assignee:
                koertkuipers Koert Kuipers
                Reporter:
                koert koert kuipers
              • Votes:
                7 Vote for this issue
                Watchers:
                39 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: