Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-3996

Correctly enforce the memory limit on the multi-table map-join

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.11.0
    • Fix Version/s: 0.11.0
    • Component/s: Query Processor
    • Labels:
      None

      Description

      Currently with HIVE-3784, the joins are converted to map-joins based on checks of the table size against the config variable: hive.auto.convert.join.noconditionaltask.size.

      However, the current implementation will also merge multiple mapjoin operators into a single task regardless of whether the sum of the table sizes will exceed the configured value.

        Attachments

        1. HIVE-3996.patch
          10 kB
          Vikram Dixit K
        2. HIVE-3996_2.patch
          56 kB
          Vikram Dixit K
        3. HIVE-3996_3.patch
          21 kB
          Vikram Dixit K
        4. HIVE-3996_4.patch
          22 kB
          Vikram Dixit K
        5. HIVE-3996_5.patch
          155 kB
          Vikram Dixit K
        6. HIVE-3996_6.patch
          155 kB
          Vikram Dixit K
        7. HIVE-3996_7.patch
          155 kB
          Vikram Dixit K
        8. HIVE-3996_8.patch
          158 kB
          Vikram Dixit K
        9. HIVE-3996_9.patch
          114 kB
          Vikram Dixit K
        10. hive.3996.9.patch-nohcat
          141 kB
          Namit Jain

          Issue Links

            Activity

              People

              • Assignee:
                vikram.dixit Vikram Dixit K
                Reporter:
                vikram.dixit Vikram Dixit K
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: