Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-9277

Hybrid Hybrid Grace Hash Join

    XMLWordPrintableJSON

    Details

      Description

      We are proposing an enhanced hash join algorithm called “hybrid hybrid grace hash join”.

      We can benefit from this feature as illustrated below:

      • The query will not fail even if the estimated memory requirement is slightly wrong
      • Expensive garbage collection overhead can be avoided when hash table grows
      • Join execution using a Map join operator even though the small table doesn't fit in memory as spilling some data from the build and probe sides will still be cheaper than having to shuffle the large fact table

      The design was based on Hadoop’s parallel processing capability and significant amount of memory available.

        Attachments

        1. HIVE-9277.15.patch
          138 kB
          Wei Zheng
        2. HIVE-9277.14.patch
          138 kB
          Wei Zheng
        3. HIVE-9277.13.patch
          138 kB
          Wei Zheng
        4. HIVE-9277.08.patch
          114 kB
          Wei Zheng
        5. HIVE-9277.07.patch
          88 kB
          Wei Zheng
        6. HIVE-9277.06.patch
          86 kB
          Wei Zheng
        7. HIVE-9277.05.patch
          87 kB
          Wei Zheng
        8. HIVE-9277.04.patch
          95 kB
          Wei Zheng
        9. HIVE-9277.03.patch
          84 kB
          Wei Zheng
        10. HIVE-9277.02.patch
          86 kB
          Wei Zheng
        11. HIVE-9277.01.patch
          89 kB
          Wei Zheng
        12. High-leveldesignforHybridHybridGraceHashJoinv1.0.pdf
          453 kB
          Wei Zheng

          Issue Links

            Activity

              People

              • Assignee:
                wzheng Wei Zheng
                Reporter:
                wzheng Wei Zheng
              • Votes:
                0 Vote for this issue
                Watchers:
                12 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: