Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-3655

Support sorting of values in addition to keys (i.e. secondary sort)

          Details

          • Type: New Feature
          • Status: In Progress
          • Priority: Major
          • Resolution: Unresolved
          • Affects Version/s: 1.1.0, 1.2.0
          • Fix Version/s: None
          • Component/s: Spark Core
          • Labels:
            None

            Description

            Now that spark has a sort based shuffle, can we expect a secondary sort soon? There are some use cases where getting a sorted iterator of values per key is helpful.

              Attachments

                Issue Links

                is related to

                New Feature - A new feature of the product, which has yet to be developed. SPARK-2045 Sort-based shuffle implementation

                • Major - Major loss of function.
                • Resolved

                Improvement - An improvement or enhancement to an existing feature or task. SPARK-10405 Support takeOrdered and topK values per key

                • Minor - Minor loss of function, or other problem where easy workaround is present.
                • Resolved
                relates to

                Sub-task - The sub-task of the issue PIG-4504 Enable Secondary key sort feature in spark mode

                • Major - Major loss of function.
                • Closed

                New Feature - A new feature of the product, which has yet to be developed. SPARK-15798 Secondary sort in Dataset/DataFrame

                • Major - Major loss of function.
                • Open
                links to

                Pull request #3632 [Github] Pull Request #3632 (koertkuipers)

                  Activity

                    People

                    • Assignee:
                      koertkuipers Koert Kuipers
                      Reporter:
                      koert koert kuipers
                    • Votes:
                      7 Vote for this issue
                      Watchers:
                      39 Start watching this issue

                      Dates

                      • Created:
                        Updated: