Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-7292 Hive on Spark
  3. HIVE-7613

Research optimization of auto convert join to map join [Spark branch]

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • None
    • 1.1.0
    • Spark
    • None

    Description

      ConvertJoinMapJoin is an optimization the replaces a common join(aka shuffle join) with a map join(aka broadcast or fragment replicate join) when possible. we need to research how to make it workable with Hive on Spark.

      Attachments

        1. small_table_broadcasting.pdf
          44 kB
          Xuefu Zhang
        2. HIve on Spark Map join background.docx
          123 kB
          Szehon Ho
        3. Hive on Spark Join Master Design.pdf
          1.32 MB
          Szehon Ho

        Issue Links

          is depended upon by

          Sub-task - The sub-task of the issue HIVE-8406 Research on skewed join [Spark Branch]

          • Major - Major loss of function.
          • Resolved
          is part of

          Improvement - An improvement or enhancement to an existing feature or task. HIVE-7292 Hive on Spark

          • Major - Major loss of function.
          • Resolved
          is related to

          Sub-task - The sub-task of the issue HIVE-8243 clone SparkWork for join optimization

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HIVE-8183 make ConvertJoinMapJoin optimization pluggable for different execution frameworks

          • Major - Major loss of function.
          • Open
          relates to

          Sub-task - The sub-task of the issue HIVE-8616 convert joinOp to MapJoinOp and generate MapWorks only [Spark Branch]

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue HIVE-8621 Dump small table join data for map-join [Spark Branch]

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue HIVE-8622 Split map-join plan into 2 SparkTasks in 3 stages [Spark Branch]

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue HIVE-8623 Implement HashTableLoader for Spark map-join [Spark Branch]

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HIVE-8699 Enable support for common map join [Spark Branch]

          • Major - Major loss of function.
          • Open

          Activity

            People

              ssatish Suhas Satish
              chengxiang li Chengxiang Li
              Votes:
              0 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: