Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-7292 Hive on Spark
  3. HIVE-9561

SHUFFLE_SORT should only be used for order by query [Spark Branch]

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 1.2.0
    • Spark
    • None

    Description

      The sortByKey shuffle launches probe jobs. Such jobs can hurt performance and are difficult to control. So we should limit the use of sortByKey to order by query only.

      Attachments

        1. HIVE-9561.6-spark.patch
          66 kB
          Xuefu Zhang
        2. HIVE-9561.5-spark.patch
          59 kB
          Xuefu Zhang
        3. HIVE-9561.4-spark.patch
          65 kB
          Rui Li
        4. HIVE-9561.3-spark.patch
          75 kB
          Rui Li
        5. HIVE-9561.2-spark.patch
          90 kB
          Rui Li
        6. HIVE-9561.1-spark.patch
          3 kB
          Rui Li

        Issue Links

          is related to

          Sub-task - The sub-task of the issue HIVE-9649 Skip orderBy or sortBy for intermediate result [Spark Branch]

          • Major - Major loss of function.
          • Open
          relates to

          Sub-task - The sub-task of the issue HIVE-9666 Improve some qtests

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed
          links to

          Web Link RB entry

          Activity

            People

              lirui Rui Li
              lirui Rui Li
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: