Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-17276

Check max shuffle size when converting to dynamically partitioned hash join

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 3.0.0
    • 3.0.0
    • Physical Optimizer

    Description

      Currently we only check that the max number of entries in the hashmap for a MapJoin surpasses a certain threshold to decide whether to execute a dynamically partitioned hash join.

      We would like to factor the size of the large input that we will shuffle for the dynamically partitioned hash join into the cost model too.

      Attachments

        1. HIVE-17276.patch
          14 kB
          jcamachorodriguez
        2. HIVE-17276.03.patch
          15 kB
          jcamachorodriguez
        3. HIVE-17276.02.patch
          15 kB
          jcamachorodriguez
        4. HIVE-17276.01.patch
          14 kB
          jcamachorodriguez

        Issue Links

          breaks

          Bug - A problem which impairs or prevents the functions of the product. HIVE-17464 Fix to be able to disable max shuffle size DHJ config

          • Major - Major loss of function.
          • Closed
          relates to

          Bug - A problem which impairs or prevents the functions of the product. HIVE-24308 FIX conditions used for DPHJ conversion

          • Major - Major loss of function.
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. HIVE-12492 MapJoin: 4 million unique integers seems to be a probe plateau

          • Major - Major loss of function.
          • Closed

          Activity

            People

              jcamacho Jesús Camacho Rodríguez
              jcamacho Jesús Camacho Rodríguez
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: