Hive
  1. Hive
  2. HIVE-3652

Join optimization for star schema

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Duplicate
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Query Processor
    • Labels:
      None

      Description

      Currently, if we join one fact table with multiple dimension tables, it results in multiple mapreduce jobs for each join with dimension table, because join would be on different keys for each dimension.
      Usually all the dimension tables will be small and can fit into memory and so map-side join can used to join with fact table.

      In this issue I want to look at optimizing such query to generate single mapreduce job sothat mapper loads dimension tables into memory and joins with fact table on different keys as well.

      1. HIVE-3652-tests.patch
        17 kB
        Amareshwari Sriramadasu
      2. HIVE-3652-tests.patch
        23 kB
        Amareshwari Sriramadasu

        Issue Links

          Activity

          Amareshwari Sriramadasu created issue -
          Amareshwari Sriramadasu made changes -
          Field Original Value New Value
          Description Currently, if we join one fact table with multiple dimension tables, it results in multiple mapreduce jobs for each join with dimension table, because join would be on different keys for each dimension.
          Usually all the dimension tables will be small and can hit into memory and so map-side join can used to join with fact table.

          In this issue I want to look at optimizing such query to generate single mapreduce job sothat mapper loads dimension tables into memory and joins with fact table on different keys as well.
          Currently, if we join one fact table with multiple dimension tables, it results in multiple mapreduce jobs for each join with dimension table, because join would be on different keys for each dimension.
          Usually all the dimension tables will be small and can fit into memory and so map-side join can used to join with fact table.

          In this issue I want to look at optimizing such query to generate single mapreduce job sothat mapper loads dimension tables into memory and joins with fact table on different keys as well.
          Amareshwari Sriramadasu made changes -
          Assignee Amareshwari Sriramadasu [ amareshwari ]
          Vikram Dixit K made changes -
          Assignee Vikram Dixit K [ vikram.dixit ]
          Vikram Dixit K made changes -
          Link This issue duplicates HIVE-3784 [ HIVE-3784 ]
          Vikram Dixit K made changes -
          Status Open [ 1 ] Resolved [ 5 ]
          Fix Version/s 0.11.0 [ 12323587 ]
          Resolution Duplicate [ 3 ]
          Amareshwari Sriramadasu made changes -
          Resolution Duplicate [ 3 ]
          Status Resolved [ 5 ] Reopened [ 4 ]
          Amareshwari Sriramadasu made changes -
          Attachment HIVE-3652-tests.patch [ 12569159 ]
          Amareshwari Sriramadasu made changes -
          Attachment HIVE-3652-tests.patch [ 12569162 ]
          Amareshwari Sriramadasu made changes -
          Link This issue is duplicated by HIVE-3784 [ HIVE-3784 ]
          Amareshwari Sriramadasu made changes -
          Status Reopened [ 4 ] Resolved [ 5 ]
          Resolution Duplicate [ 3 ]
          Owen O'Malley made changes -
          Fix Version/s 0.11.0 [ 12323587 ]

            People

            • Assignee:
              Vikram Dixit K
              Reporter:
              Amareshwari Sriramadasu
            • Votes:
              0 Vote for this issue
              Watchers:
              13 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development